Concepts / Iterating Through Multiple XML Nodes

Iterating Through Multiple XML Nodes

Imagine the XML document as a tree.

  • Programming
Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

  1. The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
  2. Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
  3. findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
  4. find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
  5. Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

The Empty-List Puzzle

A findall() call can return an empty list even when the requested elements clearly exist in the XML document. The usual reason is not that the elements are missing. It is that the path given to findall() does not match the document's hierarchy. The key idea is simple: findall() follows a path; it does not search globally for every element with a matching name.

What do you think happens?

An XML root named stuff contains users, and users contains two user elements. Which path should findall() use to locate both user elements?

  • user

  • users/user

  • stuff/users/user

Reveal answer
Answer:

users/user

The path begins with the root's child, users, and then descends to user. The root element, stuff, is excluded from the path.

Following the XML Hierarchy

Imagine the XML document as a tree. The root element is at the top. Its children are beneath it, and deeper elements are beneath their own parents. When findall() receives a path, it starts from the element on which the method is called and follows each path segment in order. For a root element named stuff containing users and then user, the path users/user means: first move from stuff to users, then move from users to user. Every parent level between the starting element and the target must appear in the path.

stuffrootusersparentusertargetcontainscontains
How does findall() follow each level from the root's child to the target element?
Key point

The root element is the starting point, not a path segment. Include all parent-level elements below the root, followed by the target element. For this structure, use users/user rather than stuff/users/user or user.

Constructing a Nested Path

A nested findall() path lists element names in hierarchy order and separates them with a slash. The first name identifies a child of the element on which findall() is called. Each following name identifies the next level down. Therefore, users/user describes two levels below stuff: users first, then user.

usersfirst child level/next levelusertarget levelthenthen
How do the XML nesting levels map to the element names and separator in a findall() path?
Worked example

Locating Nested User Elements

A document has the hierarchy stuff, then users, then multiple user elements. Determine the path that locates the user elements.

  1. 1
    Start at the method's element

    The method is called on stuff, so stuff is the starting element and is not included in the path.

  2. 2
    Add the intermediate parent

    The user elements are inside users, so users must be the first path segment.

  3. 3
    Add the target

    The target elements are named user, so user follows users after the slash.

Result

The complete path is users/user. It matches the document hierarchy and finds the two user elements.

Tracing Success and Failure

Consider the same XML tree: stuff contains users, and users contains two user elements. Calling findall() with users/user succeeds because the path matches both levels below the root. Calling it with user fails because user is not a direct child of stuff. In that second case, findall() is not looking one level deeper automatically; it is checking for user elements at the direct-child level requested by the path.

userresult: 0 elementsdirect child of stuffnot foundusers/userresult: 2 elementsuserinside usersexpectsmatches
Why does findall() return an empty list when the requested path does not match the XML hierarchy exactly?
PathWhat it asks forResult
users/userFind users, then find user elements inside themFinds 2 elements
userFind user elements directly inside stuffFinds 0 elements

The two paths begin from the same root element but describe different hierarchy levels.

Key point

A successful result depends on structural agreement, not just on the target name. Both paths mention user, but only users/user describes where user actually appears.

Diagnosing an Empty Result

  • Common mistake

    Treating findall() as a global search

    What it looks like

    Looking for user with the path user when user is nested inside users

    Why it fails

    findall() follows the supplied hierarchy instead of searching the entire XML tree for every matching name.

    Do this instead

    Write the complete path below the root: users/user.

  • Common mistake

    Omitting an intermediate parent

    What it looks like

    Using user when the actual structure is users followed by user

    Why it fails

    The requested user element is not a direct child of the starting element.

    Do this instead

    Include every ancestor between the starting element and the target, except the root element.

  • Common mistake

    Including the root in the path

    What it looks like
    Using stuff/users/user when findall() is called on stuff
    Why it fails

    The method already starts at stuff, so the path begins with its child level.

    Do this instead

    Leave out stuff and use users/user.

  • Common mistake

    Changing the capitalization of an element name

    What it looks like

    Using User when the XML element is named user

    Why it fails

    XML element names are case-sensitive, so User and user are different names.

    Do this instead

    Match the capitalization used in the XML exactly.

  1. 1
    Check the XML hierarchy from the starting element to the target.
  2. 2
    List every parent-level element below the root.
  3. 3
    Place the names in order and separate adjacent levels with a slash.
  4. 4
    Confirm that the root element itself is excluded.
  5. 5
    Check the spelling and capitalization of every element name.

Using the Returned Nodes

Once the complete path matches the XML hierarchy, findall() can locate multiple elements at the final path level. In the source example, users/user locates two user elements. Those returned elements are the nodes that can then be processed one by one through iteration. The important separation is this: the path determines which nodes are selected, and iteration moves through the nodes that were successfully selected.

MEDIUM

A root element named catalog contains section, and section contains multiple item elements. What path should be supplied to findall() when it is called on catalog? Explain why the root name is or is not included.

Hints
  • Start with the child level immediately below the root.
  • Include section before item.
  • Do not include catalog because it is the element on which findall() starts.
Best practice

Before writing a path, trace the nesting visually from the starting element to the target. Then translate each level below the root into one path segment. This makes the path reflect the XML structure instead of relying on the assumption that findall() searches globally.

Path Rules to Remember

  1. 1
    findall() follows a specified XML path; it does not perform a global search.
  2. 2
    Include every parent-level element between the root and the target.
  3. 3
    Exclude the root element because findall() starts from the element on which it is called.
  4. 4
    Use slash-separated element names in hierarchy order, such as users/user.
  5. 5
    If the result is an empty list, check the hierarchy, omitted parents, root inclusion, and capitalization first.

Key Takeaways

  • findall() is a path-following function rather than a global element-name search.
  • A nested target requires every parent-level element in the path except the root.
  • For stuff containing users containing user elements, users/user succeeds while user returns an empty list.
  • Element names must match the XML exactly, including capitalization.
  • When findall() returns no results, compare the requested path with the actual XML hierarchy.

Related Topics

More concepts starting with I