Concepts / Parsing XML with Python's ElementTree

Parsing XML with Python's ElementTree

The overall extraction process has three steps: parse the XML string with ET.fromstring, locate an element with find, and extract either its text with .text or an attribute value with .get().

  • Programming
Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

  1. The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
  2. Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
  3. findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
  4. find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
  5. Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

From Text to Structure

When XML data arrives as a plain string, it is only a sequence of characters. You cannot conveniently navigate to a particular piece of information until the string has been transformed into a structure. Python's ElementTree provides the fromstring function for this conversion. It reads the XML string and builds a hierarchical tree whose elements mirror the nesting in the XML.

Key point

The overall extraction process has three steps: parse the XML string with ET.fromstring, locate an element with find, and extract either its text with .text or an attribute value with .get().

Building the Element Tree

The fromstring function returns an Element object representing the root element. The root is the entry point to the complete tree. For example, if the outermost tag is person, the returned Element represents person and gives your code access to the nested elements inside it.

personroot elementnameAdaphone555-0100containscontains
How does a flat XML string become a tree of nested elements?
Code walkthrough
import xml.etree.ElementTree as ET

xml_data = "<person><name>Ada</name><phone>555-0100</phone></person>"
root = ET.fromstring(xml_data)

print(root.tag)
  1. 1
    Import ElementTree using the name ET.
  2. 2
    Store the XML document as a plain string.
  3. 3
    Pass the string to ET.fromstring. The returned Element represents the root element.
  4. 4
    Read the root element's tag with .tag.
Output
person

Finding a Nested Element

After parsing, use the find method to search the tree for a tag name. find returns an Element object for the first matching element. That returned object becomes the specific part of the tree from which you can extract data.

personsearch begins at rootnamematching elementnamereturned by findphoneother elementcontainscontainsfind("name")
How does find locate the requested element in the parsed hierarchy?
python
Output
Ada

The check is important. If no element has the requested tag, find returns None. None is not an Element, so attempting to access .text or .get() on it causes the code to fail. Check the result before reading its properties.

Reading Text and Attributes

XML stores information in two distinct places. Text content appears between an element's opening and closing tags and is accessed through .text. Attributes are key-value pairs in the opening tag and are accessed with .get(). An element may have text, attributes, both, or neither.

nameAda.textAdapersonid="p1".get("id")p1reads contentreads attribute
What is the difference between content inside an element and metadata in its opening tag?
XML locationPython accessWhat it retrieves
Between opening and closing tags.textElement content
Key-value pair in the opening tag.get()Attribute value

Text content and attributes use different access mechanisms.

python
Output
Ada
p1

In this example, name_element.text reads the content inside the name tags. In contrast, root.get("id") reads the id attribute from the person opening tag. The two expressions retrieve different kinds of XML data.

Cleaning Extracted Text

The value returned by .text can include whitespace from the original XML. This is especially noticeable when the XML contains newlines and spaces around the content. When clean text is needed, use the string .strip() method after confirming that you have a valid element.

python

Mistakes in XML Extraction

  • Common mistake

    Using .get() to read element text

    What it looks like
    name_element.get("name")
    Why it fails

    Text content and attributes are separate. .get() accesses an attribute value, not the content between tags.

    Do this instead

    Use name_element.text for content between the opening and closing tags.

  • Common mistake

    Using .text to read an attribute

    What it looks like

    root.text to retrieve the id attribute

    Why it fails

    The id value is stored as an attribute in the opening tag, not as the element's text content.

    Do this instead

    Use root.get("id") to retrieve the attribute value.

  • Common mistake

    Accessing properties without checking find

    What it looks like
    email_element = root.find("email") followed by email_element.text
    Why it fails

    If email does not exist, find returns None, and None does not provide a .text property.

    Do this instead

    Check that email_element is not None before accessing .text or .get().

  • Common mistake

    Treating whitespace as part of the desired value

    What it looks like
    phone_number = phone_element.text
    Why it fails

    The original XML may contribute newlines and spaces to the returned text.

    Do this instead

    Use phone_element.text.strip() when the extracted text needs whitespace removed.

Practice the Extraction Sequence

EASY

Given the XML string "<book code='b7'><title>Python Basics</title></book>", describe the three operations needed to parse it, locate title, and retrieve both the title text and the book code attribute.

Hints
  • Start by converting the string with ET.fromstring.
  • Use find with the title tag.
  • Use .text for the title content and .get("code") for the attribute.
  • Check the result of find before reading .text.
Worked example

Parsing and extracting from one XML string

Extract the title text and code attribute from the book element.

  1. 1
    Parse

    Pass the XML string to ET.fromstring so it becomes an Element tree with book as the root.

  2. 2
    Locate

    Call root.find("title") to obtain the first matching title element.

  3. 3
    Read text

    After checking that the returned element is valid, read title_element.text to obtain Python Basics.

  4. 4
    Read attribute

    Call root.get("code") to obtain the code attribute value b7.

Result

The extraction uses .text for title content and .get() for the code attribute.

Key Takeaways

  1. 1
    ET.fromstring converts an XML string into an Element tree whose root represents the outermost element.
  2. 2
    find locates the first element with a matching tag name and returns an Element object.
  3. 3
    Use .text for content between an element's opening and closing tags.
  4. 4
    Use .get() for attribute values stored in an element's opening tag.
  5. 5
    Check the result of find before accessing properties, and strip whitespace from text when necessary.

At a glance

  • Convert XML text into a navigable tree with ET.fromstring.
  • Use find to locate the first matching element.
  • Use .text for element content and .get() for attribute values.
  • Check for None after find and use .strip() when extracted text contains unwanted whitespace.

Related Topics

More concepts starting with P