Concepts / Handling Nested XML Structures

Handling Nested XML Structures

The result of findall is a list of Element objects, not one combined XML node. The position of an element in that list corresponds to its position among the matching nodes returned by the search.

  • Programming
Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

  1. The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
  2. Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
  3. findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
  4. find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
  5. Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

From Repeating XML to a Python List

When an XML document contains several similar records, extracting only one record is not enough. A repeating structure such as a users container with several user elements needs a pattern that gathers every matching user and then processes them one at a time. The findall method performs the gathering step: it returns a Python list containing the XML elements that match a specified path.

usersuserChuckElementposition 0userBrentlstPython listElementposition 1containscontainsfindall resultfindall result
What data structure does findall return, and how do the matching XML nodes relate to their positions in the returned list?
Key point

The result of findall is a list of Element objects, not one combined XML node. The position of an element in that list corresponds to its position among the matching nodes returned by the search.

Tracing the Loop

After findall returns its list, a for loop visits each Element in sequence. The loop variable holds the current XML node for one iteration. When the list contains two matching user elements, the loop body runs twice: first with the Element for Chuck, then with the Element for Brent. The same statements run each time, but the current node contains different data.

lsttwo ElementsuserChuckloop enduserBrentiteration 1next itemall items processed
How does the loop move from one XML subtree to the next, and what node is being processed at each iteration?
Worked example

Following Two User Elements

A parsed XML tree contains a users container with two user elements. Determine what the loop variable represents on each pass.

  1. 1
    Collect

    findall returns a list containing the two matching user Element objects.

  2. 2
    First pass

    The loop variable item refers to the first user Element, representing Chuck with id 001 and x equal to 2.

  3. 3
    Second pass

    The loop variable item refers to the second user Element, representing Brent with id 009 and x equal to 7.

  4. 4
    Stop

    After the second Element has been processed, the loop automatically stops because there are no more items in the list.

Result

The loop body runs twice, once for each matching user Element.

Reading Child Text and Attributes

Once item refers to one user Element, extraction depends on where the XML data is stored. Text content is the value between an element's opening and closing tags. To obtain it, use item.find('name').text: find locates the name child inside the current user, and text retrieves that child's content. An attribute is a key-value pair on the opening tag itself, so use item.get('x') to retrieve the x attribute directly from the current user Element.

usercurrent Elementnamechild elementChucktext contentxattribute2attribute valuefind('name')textget('x')returns
How does find move from the current subtree to a child element, and where does the child element's text value come from?
XML data locationOperationWhat it accesses
Child elementitem.find('name').textText inside the name child element
Attribute on current elementitem.get('x')The value of the x attribute on the current user element

Use find followed by text for child-element content; use get for an attribute on the current element.

python
Output
Chuck 2
Brent 7

Checking Failed Iterations

When iteration produces no output or fails inside the loop, inspect the result of findall before changing the loop. Check the list length and inspect the first item. A length of zero means that the path did not match the XML structure you expected. A correct length means that findall located nodes, so the next investigation belongs inside the loop: inspect each item and verify the child-element and attribute names.

  • Common mistake

    Treating findall as though it returns one Element

    What it looks like

    Using the result as a single user node instead of a list of matching user Elements.

    Why it fails

    findall returns a Python list, and the individual Elements are obtained by processing the list.

    Do this instead

    Use a for loop so the loop variable refers to one Element on each iteration.

  • Common mistake

    Assuming an empty loop means the loop itself is broken

    What it looks like

    The for loop runs but produces no output.

    Why it fails

    The list returned by findall may have length zero because the search path does not match the XML structure.

    Do this instead

    Check the length of the returned list immediately after calling findall.

  • Common mistake

    Using the wrong child or attribute name

    What it looks like

    Searching for Name when the XML child is named name, or requesting a different attribute name.

    Why it fails

    XML names must match exactly, and XML is case-sensitive.

    Do this instead

    Compare the names used in find and get with the XML element and attribute names.

  • Common mistake

    Reading an attribute as child text

    What it looks like

    Trying to locate x with find instead of retrieving it from the current user Element.

    Why it fails

    The x value is an attribute on the user element, not a child element.

    Do this instead
    Use item.get('x') for the attribute.
Debugging

A useful debugging sequence is to check the returned list before extracting fields. First inspect len(lst) to determine whether any matching nodes were found. Then inspect the first item when the list is not empty. If the list contains the expected number of Elements but extraction fails, place a diagnostic print inside the loop and verify the exact child and attribute names.

single Elementnot the findall resultfor loopprocesses list itemslist of Elementsloop runs for each itemno loop bodywhen list is emptyempty listloop runs zero timeseach Elementzero items
What does the code actually receive from findall, and how does each result affect the loop?

Practice the Extraction Pattern

EASY

Suppose findall has returned a list named records. Each current Element contains a title child element and a code attribute. Write the loop statements that retrieve the title text and the code attribute for every record.

Hints
  • Use a for loop with one variable representing the current Element.
  • Use find followed by text for the title child.
  • Use get directly on the current Element for the code attribute.
Worked example

Solution Pattern

Retrieve the title text and code attribute from each Element in records.

  1. 1
    Iterate

    Bind record to one Element at a time with a for loop.

  2. 2
    Read child text

    Call record.find('title').text to locate the title child and retrieve its text content.

  3. 3
    Read the attribute

    Call record.get('code') because the code value is an attribute on the current Element.

Result
for record in records:
    title = record.find('title').text
    code = record.get('code')

The Complete Mental Model

  1. 1
    findall searches for matching XML subtrees and returns a Python list of Element objects.
  2. 2
    A for loop visits the returned Elements one at a time, with the loop variable holding the current node.
  3. 3
    Use item.find('child_name').text to read text from a child element.
  4. 4
    Use item.get('attribute_name') to read an attribute from the current Element.
  5. 5
    When iteration fails, check the list length, inspect an item, and verify XML names exactly.

Key Takeaways

  • findall returns a list of all matching XML Element objects.
  • The for loop processes each matching subtree in sequence.
  • Child text and attributes require different access methods.
  • find followed by text reads child-element content, while get reads an attribute.
  • Debugging begins by checking whether the returned list contains the expected nodes.

Related Topics

Related concepts

More concepts starting with H