Concepts / Navigating XML Trees with findall() and XPath

Navigating XML Trees with findall() and XPath

ET.fromstring() converts an XML string into a tree structure of Element objects, making the hierarchical data queryable.

  • Programming
Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

  1. The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
  2. Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
  3. findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
  4. find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
  5. Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

From Text to Tree

XML begins as text, but its nested tags describe a hierarchy. ET.fromstring() converts that XML string into a tree structure made of Element objects. After conversion, you work with the hierarchy rather than treating the data as one undivided string.

person XMLlinear textnamechild Elementpersonroot Elementphonechild Elementemailchild ElementET.fromstring()containscontainscontains
How does nested XML text become a root Element with child Elements after ET.fromstring()?

Suppose the XML describes a person with name, phone, and email elements. After tree = ET.fromstring(data), tree is an Element object representing the person root. The tree contains the complete hierarchy, including the three child Elements. It is not simply the original string printed in another form; it is a structured object that you query with methods.

Tracing the Root Element

Code walkthrough
tree = ET.fromstring(data)
  1. 1
    Before this line, data is an XML string.
  2. 2
    During this line, ET.fromstring() converts the string into a tree structure.
  3. 3
    After this line, tree represents the person root and its descendants.
  • ET.fromstring(data) parses the XML string.
  • tree becomes the root Element of the parsed hierarchy.
  • For the source example, the root Element is person.
dataXML stringET.fromstring()conversionpersonroot Elementinputreturns
What path does the XML string follow before you can retrieve an element?

Selecting an Element

The find() method searches the tree for the first element matching a given tag name. Starting with the root Element, you provide the tag name you want to retrieve. For example, tree.find('name') searches for the name element in the person hierarchy.

personroot Elementfind('name')tag searchnamefirst matching Elementsearchesreturns
What path does find() follow, and which single element does it return?
python
Output
Chuck

The two lines separate retrieval from extraction. First, find() produces the name Element. Then .text reads the text inside that Element. In the source example, tree.find('name').text returns Chuck.

Reading Text and Attributes

An Element can hold text content and attributes. Use the .text property to retrieve the text inside an element. Use .get(attribute_name) to retrieve the value of a named attribute. These are two different kinds of information: text belongs to the element's content, while an attribute value is retrieved by its attribute name.

emailElementemail text.textyesattribute valuehide.get('hide')containshas attributereturns
Where are an element's text value and attribute key-value pairs located, and how are they accessed?
python
ExpressionWhat it accessesSource result
tree.find('name').textText content of the name ElementChuck
tree.find('email').get('hide')Value of the hide attribute on the email Elementyes

The source example separates element text from attribute retrieval.

XPath and Multiple Matches

The topic of this article also includes findall() and XPath as ways of navigating XML trees. The supplied reference facts explain the conversion step, find(), .text, and .get() in detail, but do not specify the exact findall() or XPath syntax or matching rules. Keep the verified distinction clear: find() is documented here as returning the first element matching a tag name.

personrootnamechildphonechildemailchildpath targetpath targetpath target
How can a navigation query be understood as moving through the XML hierarchy toward named elements?
Best practice

When reading XML code, trace it in this order: identify the Element that represents the current tree, identify the tag or navigation expression being used, identify the selected Element, and only then read .text or .get() to determine the extracted value.

Mistakes Beginners Make

  • Common mistake

    Treating the result of ET.fromstring() as an ordinary string

    What it looks like

    Assuming tree is still the original XML text after tree = ET.fromstring(data)

    Why it fails

    The conversion produces an Element object representing the root and its hierarchy.

    Do this instead

    Use Element methods and properties such as find(), .text, and .get().

  • Common mistake

    Confusing the Element with its text content

    What it looks like

    Using the result of tree.find('name') as if it were already the text Chuck

    Why it fails

    find() returns an element, while .text retrieves the content inside that element.

    Do this instead

    Use tree.find('name').text when you need the text value.

  • Common mistake

    Using .text to retrieve an attribute

    What it looks like

    Trying to read the hide value on the email element with .text

    Why it fails

    The source example retrieves the hide attribute with .get('hide').

    Do this instead

    Use tree.find('email').get('hide') for the attribute value.

  • Common mistake

    Ignoring preserved whitespace

    What it looks like

    Using text exactly as returned when the XML contains surrounding whitespace

    Why it fails

    Whitespace in XML text is preserved.

    Do this instead

    Use .strip() when surrounding whitespace needs to be removed.

  • Common mistake

    Assuming find() represents every possible match

    What it looks like
    Treating tree.find('name') as a collection of matching elements
    Why it fails

    The documented behavior here is that find() searches for the first matching element.

    Do this instead

    Interpret the result of find() as one selected Element, and consult the specific API documentation when using findall() or XPath.

Practice the Trace

EASY

Using the source example's person tree, trace each expression and state what kind of result it produces: tree = ET.fromstring(data), tree.find('name'), tree.find('name').text, and tree.find('email').get('hide').

Hints
  • The first expression performs the conversion.
  • The second expression selects an Element.
  • The third expression reads text from the selected Element.
  • The fourth expression reads an attribute value from the email Element.
Worked example

Tracing the Source Example

Determine what each operation represents when the XML root is person with name, phone, and email children.

  1. 1
    Convert

    ET.fromstring(data) turns the XML string into an Element tree whose root is person.

  2. 2
    Select

    tree.find('name') searches the tree for the first element matching the name tag.

  3. 3
    Read text

    Adding .text retrieves the text inside the name Element, which is Chuck in the source example.

  4. 4
    Read attribute

    tree.find('email').get('hide') retrieves the hide attribute value from the email Element, which is yes in the source example.

Result

The conversion creates the structure, find() selects an Element, .text reads element content, and .get() reads an attribute value.

Key Takeaways

  1. 1
    ET.fromstring() converts an XML string into a hierarchy of Element objects.
  2. 2
    The resulting root Element represents the complete XML tree and its descendants.
  3. 3
    find() searches for the first element matching a tag name.
  4. 4
    .text retrieves an element's text content, while .get(attribute_name) retrieves an attribute value.
  5. 5
    XML whitespace is preserved, so use .strip() when surrounding whitespace must be cleaned.

At a glance

  • ET.fromstring() changes linear XML text into a queryable Element tree.
  • The root Element contains the hierarchy represented by the XML.
  • find() returns the first Element matching the requested tag name.
  • Use .text for element content and .get() for attribute values.
  • Remember that XML whitespace is preserved and may require .strip().

Related Topics

Related concepts

More concepts starting with N