Character Classes and Quantifiers in Regex

Raw strings (r'...') are mandatory for regex patterns because they prevent Python from interpreting backslashes as escape sequences, allowing the regex engine to receive the correct pattern.

Programming

Interactive lab

Try it: Regular expressions: a backtracking matcher

How Python's re module matches a pattern: it tries each start position, matches tokens one by one, lets greedy quantifiers take as much as possible and give it back when the rest fails (lazy ones take as little as possible), tries alternatives in order and records groups.

How it works

search tries start positions 0, 1, 2 …; match tries only position 0; findall collects every non-overlapping match.
Tokens are matched left to right: literals, ., classes like [a-c] and \d, and zero-width anchors ^, \$, \b.
A greedy quantifier (* + ? {m,n}) takes as many repetitions as it can, then backtracks one at a time; a lazy one (*? +? ??) takes as few as possible and grows only when needed.
Alternatives in a | b are tried in order; the first that lets the whole pattern match wins.
Parentheses capture the text their part matched; captures are undone when the matcher backtracks past them.

Default run (63 steps): re.search('(\\w+)@(\\w+)\\.com', 'mail bo@ikb.com now'): try each start position until one matches. … re.search('(\\w+)@(\\w+)\\.com', 'mail bo@ikb.com now') → <re.Match object; span=(5, 15), match='bo@ikb.com'>

Simplified: A subset of re: no flags, lookarounds, backreferences, named groups or possessive quantifiers; pattern and text up to 24 printable ASCII characters. The trace stops after 1900 attempts ("too many steps") where real Python would keep backtracking.

Educational simulation

Loading the simulation…

From Python Source to Regex Engine

A regular expression pattern may contain backslashes, but Python processes the pattern before the regular expression engine receives it. In an ordinary Python string, Python treats backslashes as special characters used in escape sequences such as \n and \t. A raw string, written with the r prefix, prevents Python from interpreting those backslashes first. This allows the regex engine to receive the pattern you intended, such as \S+.

What changes as r'\S+' moves from Python source code to the regular expression engine?

python

pattern = r'\S+'

pattern = r'\S+'

Reading \S and + Together

The character class \S matches one non-whitespace character. The plus quantifier means one or more, so \S+ matches one or more consecutive non-whitespace characters. The match continues across adjacent non-whitespace characters and stops when it reaches whitespace or the end of the string.

Which parts of the text does \S+ consume, and where do spaces divide the matches?

What do you think happens?

What substrings will r'\S+' match in the text Send mail today?

Send mail today as one match
Send, mail, and today as three matches
Only Send
Nothing, because the text contains spaces

Reveal answer

Answer:

Send, mail, and today as three matches

Each word is a consecutive run of non-whitespace characters. The spaces stop one \S+ match and separate it from the next.

python

import re

text = 'Send mail today'
matches = re.findall(r'\S+', text)
print(matches)

import re

text = 'Send mail today'
matches = re.findall(r'\S+', text)
print(matches)

Output (expected)

['Send', 'mail', 'today']

Scanning with findall

re.findall() scans the input from left to right. At each position, it tests whether the pattern matches. When it finds a match, it adds the matched substring to a list and continues scanning from the position after that match. The result is the complete list of matches found by the scan.

How does re.findall() move through the input and collect each non-overlapping match?

Worked example

Extracting Non-Whitespace Tokens

Use r'\S+' with re.findall() on the text 'Status: ready now'.

1
Construct the pattern
The raw string r'\S+' gives the regex engine a pattern that means one or more consecutive non-whitespace characters.
2
Scan from the left
The scan first finds Status:, then stops at the space.
3
Continue after whitespace
The scan finds ready and then now as separate non-whitespace runs.

Result

The returned list is ['Status:', 'ready', 'now'].

Finding Email-Like Text

The pattern \S+@\S+ requires at least one non-whitespace character before the @ symbol and at least one non-whitespace character after it. Combined with re.findall(), it extracts email-like substrings from surrounding prose. Text that does not satisfy those requirements remains unmatched.

How does the pattern select email-like substrings while leaving surrounding prose unmatched?

Code walkthrough

import re

text = 'Contact alice@example.com or bob@example.org.'
addresses = re.findall(r'\S+@\S+', text)
print(addresses)

1
Import the re module so the program can use re.findall().
2
Store prose containing two email-like substrings in text.
3
Use the raw pattern r'\S+@\S+' to require non-whitespace text on both sides of @.
4
Call re.findall() to collect every matching substring in a list.
5
Print the returned list.

Output

['alice@example.com', 'bob@example.org.']

Debugging Empty Results

How can you compare the pattern requirements with the input when no email-like match appears?

Common mistake
Leaving out the raw-string prefix.
What it looks like
```
pattern = '\S+'
```
Why it fails
Python processes backslashes in regular strings before the regex engine receives the pattern, which can create a mismatch between the intended pattern and the pattern used.
Do this instead
```
Write pattern = r'\S+'.
```
Common mistake
Expecting \S+ to cross whitespace.
What it looks like
Applying r'\S+' to 'red blue' and expecting 'red blue' as one match.
Why it fails
\S matches non-whitespace, and the + quantifier stops when whitespace is reached.
Do this instead
Expect separate matches for 'red' and 'blue'.
Common mistake
Assuming every email-like string satisfies \S+@\S+.
What it looks like
```
Testing 'name @domain.com'.
```
Why it fails
The pattern requires at least one non-whitespace character immediately before @, but the space breaks that requirement.
Do this instead
Compare the input with the pattern and check that non-whitespace text appears on both sides of @.
Common mistake
Accessing the first result without checking whether a match exists.
What it looks like
```
matches = re.findall(r'\S+@\S+', line) followed by matches[0].
```
Why it fails
re.findall() returns an empty list when there are no matches, so accessing an element that is not present can cause IndexError.
Do this instead
Check that the returned list is not empty before accessing individual elements.

python

import re

line = 'No address is provided here.'
matches = re.findall(r'\S+@\S+', line)

if len(matches) > 0:
    print(matches)
else:
    print('No matches found')

import re

line = 'No address is provided here.'
matches = re.findall(r'\S+@\S+', line)

if len(matches) > 0:
    print(matches)
else:
    print('No matches found')

Best practice

When a match is missing, inspect the problem in stages: confirm that the pattern is a raw string, check what \S and + require, compare those requirements with the input text, and verify that the result list contains an item before reading one.

Practice the Prediction

MEDIUM

Predict the result of re.findall(r'\S+', 'A\tshort\nline') before checking it. Identify each non-whitespace run and explain what causes one match to stop and the next one to begin.

Hints

Treat the tab and line break as whitespace separators.
The + quantifier consumes consecutive non-whitespace characters only.
List the substrings in their left-to-right order.

Your attempt (not saved)

MEDIUM

Use re.findall(r'\S+@\S+', text) on a line that contains one valid email-like substring and one string with a space before @. Predict which substring appears in the returned list, then add a length check before processing the result.

Hints

The pattern needs non-whitespace text immediately before and after @.
re.findall() returns all matches in a list.
Check len(matches) > 0 before accessing an individual result.

Your attempt (not saved)

Key Takeaways

1
Use raw strings such as r'\S+' so Python does not interpret regex backslashes before the regex engine receives them.
2
\S matches one non-whitespace character, while \S+ matches a consecutive run of one or more non-whitespace characters.
3
re.findall() scans from left to right and returns all matches in a list.
4
The pattern \S+@\S+ extracts email-like substrings with non-whitespace text on both sides of @.
5
Check that the result list is not empty before accessing an individual match.

At a glance

Raw strings protect backslashes in regex patterns from Python's string interpretation.
\S+ consumes each consecutive non-whitespace run and stops at whitespace or the end of the input.
re.findall() collects every match while scanning from left to right.
\S+@\S+ is useful for extracting email-like substrings, but it only enforces its stated non-whitespace requirements.
Empty results must be checked before individual list elements are accessed.

Related concepts

Python Strings and Escape Sequences

More concepts starting with C

Try it: Regular expressions: a backtracking matcher

From Python Source to Regex Engine

Reading \S and + Together

Scanning with findall

Finding Email-Like Text

Debugging Empty Results

Practice the Prediction

Key Takeaways

At a glance

Related Topics

Related concepts