Concepts / Pattern Matching Across Multiple Lines

Pattern Matching Across Multiple Lines

The pattern outside the parentheses must still match, but findall() returns the portion inside the parentheses.

  • Programming

A Larger Match, A Smaller Result

A regular expression can match a larger piece of text while you keep only a smaller piece of it. Parentheses control this difference. The pattern outside the parentheses must still match, but findall() returns the portion inside the parentheses.

What do you think happens?

Suppose a pattern recognizes a line beginning with X-, followed by a colon and a number, and places parentheses around the number. What should findall() return when the line matches?

  • The entire line

  • The entire matched text, including X- and the colon

  • Only the number inside the parentheses

Reveal answer
Answer:

Only the number inside the parentheses

The complete pattern determines whether the line matches. The parentheses tell findall() which part of that successful match to extract and return.

Matching Versus Capturing

A capturing group is the part of a regular expression enclosed in parentheses. During matching, the parentheses do not change the text that the pattern recognizes. Their special effect appears when findall() constructs its result: instead of returning the entire matched substring, findall() returns the text enclosed by the parentheses.

X-price: 12.50full matched textfindall() resultlist containing 12.5012.50captured portion12.50returned stringcontainsextractsreturns
Which part of a larger successful match does findall() return when parentheses surround only the number?
Worked example

Keeping Only the Number

A line contains a marker beginning with X-, a colon, and a numeric value. The pattern matches the marker and number together but places parentheses around only the numeric value.

  1. 1
    Match

    The complete pattern must recognize the required marker, colon, and number before the match succeeds.

  2. 2
    Capture

    The parentheses identify the numeric portion as the capturing group.

  3. 3
    Return

    findall() returns a list containing the captured number as a string rather than returning the whole matched text.

Result

The pattern decides whether the text matches; the capturing group decides which part findall() returns.

Parentheses as Extraction Markers

Read a pattern in two layers. First, read all of it as the condition that must match. Then identify the text enclosed in each pair of parentheses. That enclosed text is the part findall() is instructed to keep.

X-must match:must match(number)capture this partfollowed byfollowed by
What role does each part of a pattern play when parentheses surround the value to extract?

Results Across Multiple Lines

A program can read lines from a file and apply the pattern to each line. For a line that matches, findall() produces a list containing the captured portion. For a line that does not match, findall() produces an empty list, so that line contributes nothing to the printed results.

Input linesmatching and nonmatchinglinesRegular expressionmatches each lineCaptured portionsinside parenthesesfindall() liststrings or an empty listapplyextractcollect
How does multiline input move through pattern matching and become the list returned by findall()?
Pattern structurefindall() result
No capturing groupThe entire matched substring
One capturing groupA list of strings containing the captured portions
Multiple capturing groupsA list of tuples containing captured values in pattern order

The presence and number of capturing groups determine the structure of the result.

Multiple Groups and Data Types

One capturing group is useful when each match contains one value to keep. You can also place multiple pairs of parentheses in a pattern to extract several parts of the same line. In that case, findall() returns a list of tuples, and each tuple stores the captured values in the order in which their groups appear in the pattern.

Every extracted value begins as a string, even when it looks numeric. If later work requires arithmetic or numeric comparisons, convert the string explicitly to an appropriate type such as int or float.

Worked example

Extracting Two Values

A matching line contains two pieces of information that are both enclosed in separate capturing groups.

  1. 1
    Identify the groups

    The first pair of parentheses marks the first value to extract, and the second pair marks the second value.

  2. 2
    Read the result shape

    Because there are multiple groups, each match is represented as a tuple rather than as one string.

  3. 3
    Convert when needed

    The captured tuple values are still strings. Convert a numeric value to int or float before using it as a number.

Result

Multiple groups produce tuples, and numeric-looking tuple members remain strings until explicitly converted.

Safe Result Handling

A nonmatching line produces an empty list. Before accessing an element from the result, verify that findall() returned a non-empty list. The source example uses the condition len(x) > 0 so that processing occurs only when a match was found.

Mistakes with Capturing Groups

  • Common mistake

    Expecting findall() to return the entire match when the pattern contains a capturing group.

    What it looks like

    A pattern matches a marker, colon, and number, but parentheses surround only the number.

    Why it fails

    With a capturing group, findall() returns the captured portion rather than the complete matched substring.

    Do this instead

    Separate the question of whether the pattern matches from the question of which text the parentheses identify for extraction.

  • Common mistake

    Treating an extracted numeric-looking value as a number automatically.

    What it looks like

    A captured value looks like a floating-point number.

    Why it fails

    findall() returns extracted values as strings.

    Do this instead

    Convert the value explicitly to float or int when numeric operations or comparisons are required.

  • Common mistake

    Accessing the first result without checking whether a match exists.

    What it looks like

    A line does not satisfy the pattern, so findall() returns an empty list.

    Why it fails

    An empty list has no element to access.

    Do this instead

    Verify that the returned list is non-empty, such as by checking len(x) > 0.

  • Common mistake

    Assuming multiple capturing groups still produce a simple list of strings.

    What it looks like

    A pattern contains two pairs of parentheses.

    Why it fails

    With multiple groups, findall() returns tuples containing the captured values in pattern order.

    Do this instead

    Handle each result as a tuple and remember that its members are strings until converted.

Practice the Trace

MEDIUM

Imagine a multiline input containing several lines. The pattern recognizes lines that begin with X-, contain a colon, and end with a number. Parentheses surround only the number. Predict what findall() returns for a matching line and for a nonmatching line. Then decide whether the returned number is ready for arithmetic or needs conversion.

Hints
  • The full pattern decides whether a line matches.
  • The parentheses decide which portion is returned.
  • A nonmatching line produces an empty list.
  • Captured values are strings, even when they look numeric.

Key Takeaways

  • The entire regular-expression pattern determines whether text matches.
  • Parentheses create capturing groups and tell findall() which matched portions to return.
  • One capturing group produces a list of strings; multiple groups produce a list of tuples.
  • Extracted values are strings and must be converted to int or float when numeric use is required.
  • Check that findall() returned a non-empty list before accessing an element.

Related Topics

Related concepts

More concepts starting with P