Working with Nested XML Structures

To reach a nested target, the path must describe the hierarchy from that starting element down to the target.

Programming

Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

A Search That Follows a Route

When learners first encounter findall(), they may expect it to search the entire XML document for every element with a particular name. ElementTree.findall() works differently. It follows the path supplied to it from the element where the method is called. To reach a nested target, the path must describe the hierarchy from that starting element down to the target.

What exact sequence of nested elements must findall() follow to reach user, and which root element is omitted from the path?

The Path Rule

Suppose the XML tree has a root element named stuff. Inside stuff is a users element, and inside users are user elements. When the search starts with stuff, the correct path to the user elements is users/user. The root name, stuff, is not included because it is already the element from which findall() begins searching.

Path	What the path asks for	Result for the stuff tree
users/user	Find users below stuff, then find user below each users element	Finds 2 elements
user	Find user as a direct child of stuff	Finds 0 elements

The same starting element and target name can produce different results when the path changes.

Definition

A complete findall() path is the slash-separated sequence of parent and child element names needed to travel from the starting element to the target. The starting element itself is omitted from the path.

Tracing a Successful Lookup

Worked example

Finding nested user elements

An XML tree has stuff as its root, users inside stuff, and two user elements inside users. Which path should be supplied when findall() is called on stuff?

1
Identify the starting element
The search begins at stuff, so stuff is the root of this search context and is omitted from the path.
2
Identify the intermediate parent
The target user elements are not directly inside stuff. They are inside the users element, so users must appear in the path.
3
Add the target element
The final element to locate is user, so the complete path is users/user.
4
Follow the route
findall() first follows users below stuff and then looks for user elements below each users element.

Result

The correct path is users/user, and the source example reports that this path finds 2 elements.

What do you think happens?

If the search starts at stuff, what does the path user ask findall() to look for?

A user element anywhere in the tree
A user element directly under stuff
A user element inside users

Reveal answer

Answer:

A user element directly under stuff

The path contains only user, so findall() checks for user as a direct child of the starting element. In the source structure, user is nested inside users, so the result is an empty list.

How does findall() use users/user to reach the target element?

Reading Slashes as Levels

Read a findall() path from left to right. Each slash-separated name represents the next element level that findall() must follow. In users/user, users is the first level below the starting element and user is the next level below users. The path is therefore a description of the XML hierarchy, not merely a name to search for.

How does each slash-separated tag correspond to a level in the XML structure?

In the real world

For a generated structure with a root named catalog, a group inside catalog, and an item inside group, a search starting at catalog would use group/item. The same rule applies: omit the starting root and include the intermediate parent before the target.

Diagnosing an Empty Result

What happens when the requested path skips the intermediate users element?

Common mistake
Using only the target name
What it looks like
Calling findall() from stuff with the path user
Why it fails
This asks for user as a direct child of stuff, but the user elements are inside users.
Do this instead
Include the intermediate parent and use users/user.
Common mistake
Including the root name in the path
What it looks like
Using stuff/users/user when the search already starts at stuff
Why it fails
The starting root is omitted from the path in this search context.
Do this instead
Begin with the first child level below the starting element: users/user.
Common mistake
Ignoring capitalization
What it looks like
Using Users/user when the XML element is named users
Why it fails
XML element names are case-sensitive.
Do this instead
Match each element name's capitalization exactly.

1
Confirm the element name you want to find.
2
Start at the element on which findall() is called.
3
Trace the XML tree from that starting element to the target.
4
Write every parent-level element in order, excluding the starting root.
5
Check capitalization in every path component.
6
If the result is empty, compare the path with the actual nesting before assuming the target elements are absent.

Practice and Takeaways

MEDIUM

A generated XML structure has root inventory, a section inside inventory, a shelf inside section, and a book inside shelf. If the search starts at inventory, write the complete findall() path to book. Then explain which element is omitted and why.

Hints

List the elements from the starting element to the target.
Exclude the starting root from the path.
Separate the remaining element names with slashes.

Your attempt (not saved)

1
findall() follows a supplied parent-to-child route rather than searching globally. A path must include every ancestor between the starting element and the target, except the starting root itself. The path users/user succeeds when searching from stuff because it follows users before looking for user. The shorter path user fails because it asks for a direct child that is not present. When findall() returns an empty list, compare the path with the XML hierarchy, verify that no parent was skipped, and check element-name capitalization.

At a glance

findall() is a path-following function, not a global search function.
Include every parent-level element between the search context and the target, except the starting root.
Use slash-separated names to represent successive XML nesting levels.
An incomplete path commonly produces an empty list because findall() checks the wrong parent-child relationship.
Check hierarchy and capitalization whenever a lookup returns no elements.

More concepts starting with W