Grouping and Capturing with Parentheses

Capturing is useful because the search result preserves more than a yes-or-no answer.

Programming

Interactive lab

Try it: Regular expressions: a backtracking matcher

How Python's re module matches a pattern: it tries each start position, matches tokens one by one, lets greedy quantifiers take as much as possible and give it back when the rest fails (lazy ones take as little as possible), tries alternatives in order and records groups.

How it works

search tries start positions 0, 1, 2 …; match tries only position 0; findall collects every non-overlapping match.
Tokens are matched left to right: literals, ., classes like [a-c] and \d, and zero-width anchors ^, \$, \b.
A greedy quantifier (* + ? {m,n}) takes as many repetitions as it can, then backtracks one at a time; a lazy one (*? +? ??) takes as few as possible and grows only when needed.
Alternatives in a | b are tried in order; the first that lets the whole pattern match wins.
Parentheses capture the text their part matched; captures are undone when the matcher backtracks past them.

Default run (63 steps): re.search('(\\w+)@(\\w+)\\.com', 'mail bo@ikb.com now'): try each start position until one matches. … re.search('(\\w+)@(\\w+)\\.com', 'mail bo@ikb.com now') → <re.Match object; span=(5, 15), match='bo@ikb.com'>

Simplified: A subset of re: no flags, lookarounds, backreferences, named groups or possessive quantifiers; pattern and text up to 24 printable ASCII characters. The trace stops after 1900 attempts ("too many steps") where real Python would keep backtracking.

Educational simulation

Loading the simulation…

A Search Has Two Outcomes

A regular-expression search does not always give back text. When re.search() finds the requested pattern, it returns a match object. When the pattern does not occur in the string, it returns None. That difference is the foundation for understanding grouped and captured search results: first determine whether a match exists, then inspect what the match contains.

What do you think happens?

A search looks for the literal pattern From: in the string Received From: Ada. What kind of result should you expect when the pattern is found?

A match object
None
The integer -1

Reveal answer

Answer:

A match object

re.search() returns a match object when it finds a pattern. None indicates that no match exists. The integer -1 is the failure result associated with the string method find(), not re.search().

Following re.search()

The re module provides re.search(). Calling re.search(pattern, string) makes Python scan the string from left to right. As soon as the pattern is found, the search stops and the function returns a match object containing information about the match. If the pattern never appears, the function returns None rather than raising an error.

What happens next after re.search() runs when the pattern matches versus when it does not?

Best practice

Treat the result of re.search() as something to test before inspecting it. In a conditional, a match object is truthy and None is falsy, so checking the result first tells you whether it is safe to call methods such as group(), start(), or end().

Reading a Match Object

Definition

A match object is the result returned by re.search() when the pattern is found. Its group() method gives the matched text, start() gives the starting index, and end() gives the ending index. The indices use Python's zero-based indexing and slice convention.

What does a match object contain, and how are the full match, captured text, and positions related?

Worked example

Locating From:

Suppose re.search() searches for the literal pattern From: in the string Received From: Ada and returns a match object.

1
Read the full match
The match object's group() method gives From:, the text matched by the pattern.
2
Read the start
The first character of From: is at index 9 because Python counts from index 0.
3
Read the end
The end index is 14, one position after the final character of From:.
4
Check the slice
The slice from index 9 up to, but not including, index 14 gives the same matched text: From:.

Result

group() gives From:, start() gives 9, and end() gives 14.

How do the start and end indices identify the exact substring returned by group()?

Capturing the Exact Span

Capturing is useful because the search result preserves more than a yes-or-no answer. The matched text can be retrieved with group(), while start() and end() identify its location in the original string. Because end() points one position after the final matched character, the same span can be reproduced with string[match.start():match.end()].

How can a None return value be distinguished from a match object during debugging?

Edge case

If the pattern is absent, the result is None. Calling group(), start(), or end() on None causes an AttributeError. This is why a safe debugging sequence always checks the result before using match-object methods.

Regex or String Method

Task	Appropriate choice	Failure result	Reason
Search for an exact fixed string	find() or in	find() returns -1; in produces a Boolean result	These methods are simpler for literal text.
Search for a pattern involving digits, word characters, repetition, or alternatives	re.search()	None	Regular expressions are intended for patterns that are not simple literals.
Check whether text begins with a fixed prefix	startswith()	A false Boolean result	A direct string method expresses this fixed-prefix task.

Choose re.search() when the search pattern needs regular-expression behavior; choose a simpler string method for a simple fixed-string task.

Using re.search() for a literal string such as From: works, but it does not demonstrate the main power of regular expressions. For an exact fixed string, line.find('From:') >= 0 or 'From:' in line is simpler. Use re.search() when the pattern involves features such as finding any digit, any word character, repeated text, or one of several alternatives.

Mistakes During Debugging

Common mistake
Calling group() immediately on the result of a search
What it looks like
The pattern is absent, so the result is None.
Why it fails
None is not a match object and has no group() method. The call causes an AttributeError.
Do this instead
Check whether the result is truthy or whether it is None before calling group(), start(), or end().
Common mistake
Treating re.search() as if it returned only True or False
What it looks like
A successful search returns a match object, not merely the matched text.
Why it fails
A Boolean check tells you whether a match exists, but it does not by itself expose the matched text or its positions.
Do this instead
Keep the match object when you need group(), start(), or end().
Common mistake
Expecting re.search() to return -1 when no match exists
What it looks like
The pattern is absent and the result is None.
Why it fails
The -1 failure convention belongs to find(). re.search() uses None.
Do this instead
Test for None or use the result in a conditional.
Common mistake
Using regular expressions for every fixed-string search
What it looks like
Searching for the literal text From: with re.search().
Why it fails
The search works, but it does not use the pattern-matching power that makes regular expressions useful.
Do this instead
Use find(), in, or startswith() when the task is a simple fixed-string search.

Practice and Transfer

MEDIUM

For each situation, decide what result to expect and which information you would inspect: a search finds From: in Received From: Ada; a search does not find From: in Subject: Update; and a program needs to search for any digit rather than one fixed literal string.

Hints

A successful re.search() produces a match object.
An unsuccessful re.search() produces None.
Use group(), start(), and end() to inspect a successful result.
A pattern involving any digit is a case where regular expressions are more appropriate than a simple literal-string method.

Your attempt (not saved)

1
Identify the pattern and the input string.
2
Decide whether the task is a fixed-string search or a pattern search.
3
If using re.search(), check whether the result is a match object or None.
4
For a match object, inspect group() for the text and start() and end() for its span.
5
Remember that the end index is one position after the final matched character.

Working Rules

1
re.search() scans from left to right and returns a match object at the first match, or None when no match exists.
2
A match object provides the matched text through group(), the starting position through start(), and the ending position through end().
3
Python uses zero-based indexing, and string[match.start():match.end()] reproduces the matched text.
4
Always check the result before calling match-object methods, because calling them on None causes an AttributeError.
5
Use simpler string methods for exact fixed strings, and use re.search() when the task requires a non-literal pattern.

Key Takeaways

re.search() returns a match object when it finds a pattern and None when it does not.
The match object's group(), start(), and end() methods reveal the matched text and its position.
The match span follows Python's zero-based slicing convention.
Check for a match before calling its methods.
Prefer simpler string methods for fixed literal searches and re.search() for genuine pattern matching.

Related concepts

Regular Expression Pattern Matching

More concepts starting with G