Concepts / The re Module and Pattern Matching

The re Module and Pattern Matching

A grep simulator reads a file line by line, tests each line against a user-supplied regular expression, and counts how many lines match the pattern.

  • Programming

A Line-by-Line Search

The Unix grep command searches text for lines that match a pattern. A Python grep simulator uses the same central idea: it reads a file one line at a time, tests each line with a user-supplied regular expression, counts the matching lines, and reports the final count.

Input filelinesCurrent lineone lineRegular expressionuser suppliedMatch testre.searchMatch countfinal totalreadapplytestmatch increments
How does data move from the input file through regex testing to the final match count?

Counter Trace

Consider a five-line file and the pattern ^X, which searches for lines that start with X. The counter begins at 0 before the loop. Each line is tested separately. A nonmatching line leaves the counter unchanged, while a matching line increases it by one.

LineContentMatches ^X?Counter after test
1Author: JohnNo0
2X-Priority: 1Yes1
3Subject: MeetingNo1
4X-Mailer: OutlookYes2
5Date: MondayNo2

The counter changes only when the current line matches the pattern.

Counter 0before loopAuthor: Johncounter 0X-Priority: 1counter 1Subject: Meetingcounter 1X-Mailer: Outlookcounter 2Date: Mondaycounter 2Counter 2after loopreadnext linenext linenext linenext lineloop ends
What changes after each line is read, tested, and either counted or skipped?
Worked example

Tracing the Pattern ^X

Count the lines beginning with X in the five-line file.

  1. 1
    Initialize

    Set the counter to 0 before reading any lines.

  2. 2
    Test the first line

    Author: John does not match ^X, so the counter remains 0.

  3. 3
    Test the second line

    X-Priority: 1 matches ^X, so the counter becomes 1.

  4. 4
    Test the third line

    Subject: Meeting does not match, so the counter remains 1.

  5. 5
    Test the fourth line

    X-Mailer: Outlook matches, so the counter becomes 2.

  6. 6
    Test the fifth line

    Date: Monday does not match, so the counter remains 2.

  7. 7
    Finish

    After all five lines are processed, report the counter.

Result

Two lines matched ^X.

The Grep Algorithm

The algorithm has a deliberate order. First, obtain the pattern. Next, open the file. Then initialize the match counter before the loop. For every line, apply the regular expression. Increment the counter only when the line matches. After the loop ends, report the final count. This ordering matters because the loop must see every line, and the counter must preserve its value from one iteration to the next.

Code walkthrough
import re

pattern = input("Enter a regular expression: ")
filename = input("Enter a file name: ")
count = 0

with open(filename) as file:
    for line in file:
        line = line.strip()
        if re.search(pattern, line):
            count += 1

print(count)
  1. 1
    The program imports re so it can perform regular-expression matching.
  2. 2
    It obtains a pattern and a file name from the user.
  3. 3
    It sets count to 0 before any line is processed.
  4. 4
    It reads the file line by line.
  5. 5
    It strips the current line before testing it.
  6. 6
    It applies the pattern to the current line with re.search.
  7. 7
    It increments count only for a matching line.
  8. 8
    It prints the total after all lines have been tested.
  • The pattern is supplied by the user.
  • The counter is initialized before the loop.
  • The loop processes each line in the file.
  • The newline is removed from the current line.
  • The counter increases only when re.search finds a match.
  • The final count is reported after the loop.
Read lineTest patternre.searchIncrement countmatchKeep countno matchNext linecurrent linematchno matchcontinuecontinue
What happens next when a line matches the pattern versus when it does not?
Output
2

Matching Each Line

The regular expression is applied to the current line, not to an imagined collection of results. The program repeats the same operation as it advances through the file: obtain one line, test it, update the counter if appropriate, and continue. Using re.search is important for this simulator. Replacing it with re.match is a common mistake identified in the source material.

FilePython programPatternuser suppliedMatch resultmatch or no matchone linetestre.search
How is the user-supplied regular expression compared with each line, and what result does the match operation produce?

The same file can produce different totals for different patterns. The source material uses several patterns with mbox.txt to show that the selected regular expression determines which lines are counted. After the basic simulator works, the pattern can describe a more structured line, such as one containing New Revision: followed by a number.

Counter State Changes

0before first line1after first match1after nonmatch2after second matchmatchno matchmatch
How does the match counter change as matching and nonmatching lines are encountered?

When debugging, record two pieces of state after each iteration: the current line and the counter value. If the line does not match, the counter should stay the same. If it matches, the counter should increase by one. This trace reveals whether the program is testing every line and whether the update occurs in the correct branch.

What do you think happens?

A file has two matching lines. If count is initialized inside the loop, what final count might the program report?

Reveal answer
Answer:

The counter can be reset on every iteration, so it cannot preserve the total number of matches. The reported value may reflect only the latest iteration rather than the complete file.

The counter must be initialized before the loop. Initializing it inside the loop resets it whenever a new line is processed.

Mistakes to Catch

ApproachWhat it processesLikely problem
Loop through each lineOne current line at a timeThe intended grep-simulator algorithm
Process the whole file as one valueThe entire file rather than each lineDoes not follow the line-by-line counting process
Apply the pattern to the wrong valueA value other than the current lineThe match result does not describe the line being counted

The simulator must test every individual line and update one persistent counter.

  • Common mistake

    Initializing the counter inside the loop.

    What it looks like

    The counter is set to 0 again each time a new line is read.

    Why it fails

    Previous matches are lost when the counter is reset.

    Do this instead

    Initialize the counter before the loop begins.

  • Common mistake

    Forgetting to strip the newline character.

    What it looks like

    The current line still includes its line-ending character during debugging or matching.

    Why it fails

    The line value is harder to inspect and reason about.

    Do this instead

    Strip the newline character before testing the line.

  • Common mistake

    Using re.match instead of re.search.

    What it looks like

    The program uses a different match operation from the one specified for the simulator.

    Why it fails

    The matching behavior is not the intended re.search behavior.

    Do this instead

    Use re.search for the line test.

  • Common mistake

    Reporting the count inside the loop.

    What it looks like

    The program prints a result before all lines have been processed.

    Why it fails

    The value is not yet the final count for the file.

    Do this instead

    Report the count after the loop ends.

  • Common mistake

    Testing only the first few lines.

    What it looks like

    The loop stops before every line has been examined.

    Why it fails

    Matches later in the file are excluded from the result.

    Do this instead

    Continue the line-by-line loop through the entire file.

Practice and Extension

MEDIUM

Write or inspect a grep simulator that accepts a regular expression and a file name. Trace the value of the current line and the counter for every line in a small file. Then change the pattern and explain why the final count changes.

Hints
  • Check that the counter is initialized before the loop.
  • Write down the current line and counter after every match test.
  • Confirm that the program uses re.search for the line test.
  • Confirm that the final result is reported after the loop.

For an extension, search for a structured line format rather than only a prefix such as ^X. The source material suggests looking for a line containing New Revision: followed by a number. The important design remains unchanged: obtain the pattern, read every line, test the current line, update the counter only for a match, and report the total after the loop.

Key Takeaways

  1. 1
    A grep simulator reads a file one line at a time and tests every line with a user-supplied regular expression.
  2. 2
    Initialize the match counter before the loop, increment it only when the current line matches, and report it after the loop.
  3. 3
    Tracing the current line and counter after each iteration is an effective way to debug the program.
  4. 4
    Strip the newline character, use re.search for the intended line test, and avoid resetting or reporting the counter at the wrong point.
  5. 5
    The same algorithm can support more complex patterns describing structured lines.

At a glance

  • The grep-simulation algorithm combines file iteration, regular-expression matching, and counting.
  • Each line must be tested separately, and only matching lines increase the counter.
  • The counter belongs outside the loop and the final report belongs after the loop.
  • Execution tracing makes counter resets, incomplete iteration, and incorrect match operations easier to find.

Related Topics

Related concepts

More concepts starting with T