Concepts / Understanding XML Structure and Hierarchy

Understanding XML Structure and Hierarchy

The findall method retrieves a list of XML nodes using a path that must include all parent-level elements (except the root) separated by forward slashes.

  • Programming
Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

  1. The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
  2. Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
  3. findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
  4. find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
  5. Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

A Path That Finds Nothing

A findall statement can fail quietly. If user elements are nested inside a users element, findall('user') does not search everywhere for user elements. It looks for user elements directly under the current node. Because that location does not exist in this structure, the result is an empty list, and a loop over that list produces no output.

What do you think happens?

Suppose the current root contains users, and users contains user elements. What does findall('user') return?

  • All user elements anywhere in the tree

  • The direct user children of the current node

  • An error before the loop starts

Reveal answer
Answer:

The direct user children of the current node

In this structure, user is not a direct child of the current node. The result is therefore an empty list, and a for loop over it never executes.

Following Parent Elements

findall retrieves a list of XML nodes by following a path. Each parent-level element between the current node and the target must appear in the path, separated by forward slashes. The root is the starting point for the search, so its name is omitted from the path. For user elements nested inside users, the path is users/user. This means: begin at the current node, move into users, and collect its user children.

rootusersparent elementuserfirst matchusersecond matchfind childcollectcollect
How does findall('users/user') move through the XML hierarchy to select the intended nodes?
python

The result stored in lst is a list of matching user nodes. The list contains the user nodes in the order in which they appear in the XML document. The path selects the nodes; it does not yet extract the text inside their children or the attributes attached to them.

Reading Each Matched Node

Worked example

Extracting Names, IDs, and Attributes

Process the user nodes returned by findall('users/user'). For each node, retrieve the name text, the id text, and the x attribute.

  1. 1
    Retrieve the list

    findall('users/user') returns the matching user nodes as a list.

  2. 2
    Visit one node

    The for loop assigns one user node at a time to the variable item.

  3. 3
    Read child text

    item.find('name').text retrieves the text inside the name child element. item.find('id').text does the same for id.

  4. 4
    Read the attribute

    item.get('x') retrieves the value of the x attribute attached directly to the current user node.

Result

The loop processes each user once, extracting child-element text and the current node's attribute value.

Code walkthrough
lst = stuff.findall('users/user')
for item in lst:
    name = item.find('name').text
    user_id = item.find('id').text
    attribute_x = item.get('x')
  1. 1
    The first line retrieves the user nodes using the complete path from the current node.
  2. 2
    The for statement takes the nodes from lst one at a time.
  3. 3
    item.find('name').text locates the name child and extracts its text content.
  4. 4
    item.find('id').text locates the id child and extracts its text content.
  5. 5
    item.get('x') retrieves the x attribute from the current user node.
Output
Chuck 001 2
Brent 009 7

The loop visits the first user node, extracting Chuck, 001, and 2. It then visits the second user node, extracting Brent, 009, and 7. After the second iteration, the list has no remaining nodes, so the loop ends.

Text and Attributes

A child element's text and a node's attribute are stored in different parts of the XML structure. Text such as Chuck appears inside a child element such as name, so it is accessed with find('name').text. An attribute such as x="2" appears on the opening tag of the user element, so it is accessed with get('x'). Use find for a child element and its text; use get for an attribute on the current node.

XML locationAccess patternValue in the source example
Child element contentitem.find('name').textChuck
Child element contentitem.find('id').text001
Attribute on the current nodeitem.get('x')2 or 7

find followed by text reads child-element content; get reads an attribute on the current node.

userx2nameChuckid001get('x')find('name').textfind('id').text
What is the difference between child text and an attribute while processing one user node?

Path Mistakes and Repairs

Before
lst = stuff.findall('user')
for item in lst:
    process(item)

The path searches for direct user children and returns an empty list.

After
lst = stuff.findall('users/user')
for item in lst:
    process(item)

The path includes the users parent and returns the matching user nodes.

userincomplete pathempty listusers/usercomplete pathuser nodesreturnsreturns
What changes when the users parent is omitted from the findall path?
  • Common mistake

    Leaving out the parent element

    What it looks like
    findall('user')
    Why it fails

    The search looks for user elements directly under the current node, but the user elements are children of users.

    Do this instead
    Use findall('users/user').
  • Common mistake

    Assuming findall searches the entire tree from any shortened name

    What it looks like
    findall('user')
    Why it fails

    The path describes the required hierarchy from the current node; it is not an unrestricted search.

    Do this instead

    Write every parent-level element between the current node and the target, separated by forward slashes.

  • Common mistake

    Using get for child-element content

    What it looks like
    item.get('name')
    Why it fails

    name is a child element in the source structure, not an attribute on the user node.

    Do this instead
    Use item.find('name').text.
  • Common mistake

    Using find to read an attribute

    What it looks like
    item.find('x')
    Why it fails

    x is an attribute attached to the user node, not a child element.

    Do this instead
    Use item.get('x').

Check Your Path

EASY

An XML tree has a users element containing user elements. Write the findall path that retrieves the user nodes, then state which method reads a child name element and which method reads the x attribute on the current user node.

Hints
  • Include the parent element before the target element.
  • Use find followed by text for the name child.
  • Use get with the attribute name for x.
Worked example

Practice Solution

Retrieve user nodes nested inside users, then access each user's name text and x attribute.

  1. 1
    Build the path

    The parent users and target user are written as users/user.

  2. 2
    Iterate

    A for loop processes each node in the list one at a time.

  3. 3
    Read the two locations

    Use item.find('name').text for the child text and item.get('x') for the node attribute.

Result

lst = stuff.findall('users/user'); then use a for loop with item.find('name').text and item.get('x') inside it.

The Complete Mental Model

  1. 1
    findall returns a list of nodes selected by a parent-to-child path.
  2. 2
    The path includes all parent-level elements between the current node and the target, but not the root itself.
  3. 3
    A for loop visits each matching node once and preserves the order in which the nodes appear.
  4. 4
    Use find followed by text for child-element content and get for attributes on the current node.
  5. 5
    An omitted parent can produce an empty list, causing the loop to run without output.

Key Takeaways

  • Describe the XML hierarchy from the current node to the target when writing a findall path.
  • Use users/user rather than user when user elements are nested inside users.
  • Process the returned list with a for loop, one node at a time.
  • Read child text with find(...).text and attributes with get(...).
  • Treat an empty list and a loop with no output as possible signs of an incomplete path.

Related Topics

Related concepts

More concepts starting with U