Fetching Data from Web APIs

When XML data arrives from a web service, a file, or another system, it arrives as a flat string of characters.

Programming

Interactive lab

Try it: XML Element Trees

How ET.fromstring() turns XML text into a tree of Elements (tag, attributes, .text), and how find(), findall() and iter() walk that tree along a path such as users/user.

How it works

The parser reads start tags, text and end tags left to right, keeping a stack of open elements; each start tag becomes an Element that is a child of the element on top of the stack.
Text before an element's first child becomes its .text (whitespace included); an end tag must match the open element or parsing stops with ParseError.
findall(path) starts at the element it is called on (the root is not part of the path) and follows each path segment down one level of children.
find(path) returns the first match or None; iter(tag) visits every element in document order, including the root.
Reading .text, .tag or .get('attr') on each result gives the extracted values; reading an attribute from None raises AttributeError.

Default run (29 steps): ET.fromstring() receives 183 characters. The parser reads tags left to right and keeps a stack of open elements. … findall() returned a list of 2 elements; reading e.get('x') from each gives ['2', '7'].

Simplified: A small, hand-written XML parser for a subset: elements, attributes, text, self-closing tags, comments, CDATA, processing instructions and the five predefined entities plus numeric character references. DOCTYPE/DTDs and namespaces are not supported. Error messages follow Python's expat parser for these cases (column numbers can differ for non-ASCII documents). Paths support tags, *, ., //, [@attr], [@attr='v'], [tag] and [n].

Educational simulation

Loading the simulation…

From Response Text to Usable Data

When XML data arrives from a web service, a file, or another system, it arrives as a flat string of characters. The string contains tags, nesting, attributes, and values, but Python cannot conveniently search that raw text as a tree. ElementTree provides the parsing step that turns the string into a structured object.

Key point

The central workflow is parse first, then search: pass the XML string to ET.fromstring, use find to locate an element, and then read its text or an attribute.

How does XML data travel from a web API response through parsing and element lookup to the extracted value?

Building the XML Tree

ET.fromstring converts an XML string into a tree structure. ElementTree parses the XML tags, nesting, and attributes, then builds an in-memory object representing the document. The result is no longer merely a sequence of characters: it is a nested hierarchy that Python code can navigate and query.

How does ET.fromstring change a linear XML string into nested elements that can be navigated and queried?

The outermost XML tag becomes the root element. Elements inside it become child elements. Each element can have child elements, text content, and attributes. This structure is what makes later operations such as locating name or email meaningful.

What contains what in the parsed XML tree, and how are the root, child, and nested elements connected?

Finding and Reading Elements

After parsing, find searches the tree for the first element matching a supplied tag name. It returns an element object when a match is found, or None when no matching element is found. The returned object is not the text itself, so you read its text separately with .text.

How does find move from the root to locate a specific element in the tree?

python

import xml.etree.ElementTree as ET

xml_text = "<person><name>Alice</name><email hide=\"true\">alice@example.com</email></person>"
root = ET.fromstring(xml_text)

name_element = root.find("name")
email_element = root.find("email")

print(name_element.text)
print(email_element.get("hide"))

import xml.etree.ElementTree as ET

xml_text = "<person><name>Alice</name><email hide=\"true\">alice@example.com</email></person>"
root = ET.fromstring(xml_text)

name_element = root.find("name")
email_element = root.find("email")

print(name_element.text)
print(email_element.get("hide"))

Output

Alice
true

The first lookup returns the name element object. Accessing name_element.text reads the content between the name tags. The second lookup returns the email element object. Accessing email_element.get("hide") reads the value of the hide attribute. Text content and attributes are separate parts of an XML element and are accessed separately.

Where are an element's text content and attributes stored, and how are they accessed separately?

Tracing the Complete Operation

What do you think happens?

What will the two print statements display after the XML is parsed and searched?

The name element object and the email element object
Alice and true
name and hide

Reveal answer

Answer:

Alice and true

find returns element objects. The .text access extracts Alice from the name element, and .get("hide") extracts the true value from the email element's attribute.

Worked example

Following the XML Through Four States

Explain how an XML string containing person data becomes the printed values Alice and true.

1
1. Receive the string
The XML begins as flat text containing the person, name, and email tags, along with the hide attribute.
2
2. Build the tree
ET.fromstring parses the flat XML string and returns a tree object whose root is the person element.
3
3. Locate elements
root.find("name") locates the name element, while root.find("email") locates the email element.
4
4. Extract values
The name element's .text is Alice. The email element's .get("hide") retrieves the attribute value true.

Result

The parsed tree makes the XML queryable: text is obtained with .text, and the attribute is obtained with .get("hide").

Mistakes in XML Queries

Common mistake
Treating the result of find as though it were already a string
What it looks like
```
name_element = root.find("name")
```
Why it fails
find returns an element object or None, not the element's text value.
Do this instead
Use name_element.text to read the text content.
Common mistake
Reading an attribute with .text
What it looks like
```
email_element.text
```
Why it fails
The email text and the hide attribute are separate parts of the element.
Do this instead
Use email_element.get("hide") to retrieve the hide attribute.
Common mistake
Changing the capitalization of a tag or attribute name
What it looks like
```
root.find("Name") or email_element.get("Hide")
```
Why it fails
XML tag names and attribute names are case-sensitive.
Do this instead
Match the exact spelling used in the XML, such as "name" and "hide".
Common mistake
Assuming every search succeeds
What it looks like
```
missing_element = root.find("phone")
```
Why it fails
find returns None when no matching element is found.
Do this instead
Handle the None case before attempting to access element data.

Best practice

Keep the two stages visible in your code: first create the tree with ET.fromstring, then search the tree with find, and finally choose the correct extraction operation. Use .text for content between tags and .get with the exact attribute name for an attribute.

Practice the Parse-Search-Extract Pattern

MEDIUM

Suppose an XML response contains a root element named person, a child element named name with the text Jordan, and a child element named email with the attribute hide set to false. Describe the three operations needed to obtain Jordan and false.

Hints

Use ET.fromstring for the initial XML string.
Use find with the exact tag names.
Use .text for name and .get("hide") for the attribute.

Your attempt (not saved)

1
Start with the XML string received from the web service or another source.
2
Pass the string to ET.fromstring to create the tree and obtain its root element.
3
Call find with the exact tag name to locate the desired element.
4
Read element text with .text or read an attribute with .get("attribute_name").
5
Check for None when a requested element might not exist.

Key Takeaways

1
An XML response begins as a flat string of characters.
2
ET.fromstring parses that string into a hierarchical ElementTree object.
3
The root element represents the outermost tag, and nested elements form the parent-child structure.
4
find locates the first matching element and returns an element object or None.
5
Use .text for element content and .get("attribute_name") for attributes, while preserving exact case.

At a glance

ET.fromstring transforms flat XML text into a navigable tree.
The parsed tree contains a root element and nested child elements.
find searches for the first element with a matching tag name.
Element text is accessed with .text, while attributes are accessed with .get('attribute_name').
XML names are case-sensitive, and find may return None when no match exists.

More concepts starting with F