Loops: Iterating Through Files

When a loop reads a file one line at a time, each line can be tested for a substring.

Programming

Interactive lab

Try it: Reading and Writing Text Files

How open() returns a file object with a position pointer, how read(), read(n), readline(), readlines() and for line in fh advance it (and return '' at the end), how 'w' and 'a' modes change the file, and why a closed file (after close() or a with block) can no longer be read.

How it works

open(name, 'r') starts the pointer at 0; 'w' empties the file first; 'a' starts at the end.
read() returns everything from the pointer to the end; read(n) at most n characters; readline() up to and including the next \n; each moves the pointer past what it returned.
At the end of the file every read returns '' (an empty string), which is how loops know to stop; for line in fh reads line by line the same way.
write() text waits in a buffer and reaches the file when it is closed (or the program ends); reading a 'w'/'a' file or writing an 'r' file raises io.UnsupportedOperation.
close(), or leaving a with block, closes the file; any further read raises ValueError: I/O operation on closed file.

Default run (10 steps): Ready: notes.txt holds 4 lines (51 characters). Nothing is open yet. … Finished. notes.txt now holds "From: stephen\nSubject: meeting\nFrom: louis\nThanks!\n".

Simplified: The lab's own small Python interpreter runs a fixed, hand-written script (you choose the variant and the values, never the code); nothing is executed as real Python. Every script was run in CPython 3.12 with the same inputs and the lab reproduces its output and error messages. One virtual text file with \n line endings (as on Linux/macOS), positions counted in characters (ASCII only), and a write buffer that is only flushed by close() or the end of the program.

Educational simulation

Loading the simulation…

A Search Result with Two Meanings

When a loop reads a file one line at a time, each line can be tested for a substring. The find() method gives the test a precise result: it returns the starting index of the substring when the substring is present, and it returns -1 when the substring is absent. That single integer lets a program decide whether a line should be kept or skipped.

What do you think happens?

What does find() return when the substring is not present in the string?

0
-1
The length of the string
The substring itself

Reveal answer

Answer:

-1

find() returns the starting index when it finds the substring. If it cannot find the substring, it returns -1.

Mapping a Match to an Index

String positions are zero-indexed, so the first character is at position 0. When find() locates a substring, it reports the position of that substring's first character. For example, in the generated string "alice@example.com", the at sign begins at position 5. Calling find("@") therefore returns 5, not the position after the at sign and not the number of characters in the string.

What does find() report when a substring starts at a known character position, and what does it report when no match exists?

Worked example

Tracing the at sign

Determine the result of searching the generated string "alice@example.com" for the substring "@".

1
Start counting
Use zero-based positions. The first character, a, is at position 0.
2
Locate the match
The at sign follows the five characters a, l, i, c, and e, so it begins at position 5.
3
Interpret the return value
Because the substring is present, find("@") returns the starting index 5.

Result

find("@") returns 5.

Preparing Each File Line

A line read from a file includes the newline character at its end. Before searching, call rstrip() on the line. rstrip() removes trailing whitespace, including the newline, so the search operates on the actual line content. The source material recommends calling rstrip() before searching because leaving trailing whitespace in place can make a search fail unexpectedly or behave strangely.

Code walkthrough

line = line.rstrip()
position = line.find("@")

1
The first line removes trailing whitespace from the line currently being processed.
2
The second line searches for the at sign and stores either its starting index or -1 in position.

Remove trailing whitespace from the current line.
Store the integer returned by find().

Filtering Lines with a Conditional

The useful distinction is between the two possible kinds of result. A return value of -1 means the substring was not found, so the line can be skipped. Any other returned integer means the substring was found at that index, so the line can be printed or collected. A conditional makes this decision for every line in the file.

How does control flow differ when find() returns -1 compared with an index?

python

for line in file:
    line = line.rstrip()
    if line.find("@") == -1:
        continue
    print(line)

for line in file:
    line = line.rstrip()
    if line.find("@") == -1:
        continue
    print(line)

This pattern first removes trailing whitespace. It then searches the current line for the at sign. If find() returns -1, continue skips the rest of the loop body and moves to the next line. If find() returns an index, the condition is false and the line is printed.

Walking Through the File

The loop applies the same three-stage process to every line: prepare the line with rstrip(), search for the chosen substring, and either skip or keep the line. The loop does not need to know the result from the previous line. Each current line receives its own find() result.

How does the loop move through lines, test each one, and produce matching lines?

How does each line move from the file through cleanup and substring testing into the extracted result?

python

file = open("mbox-short.txt")
matching_lines = []

for line in file:
    line = line.rstrip()
    if line.find("@") == -1:
        continue
    matching_lines.append(line)

file = open("mbox-short.txt")
matching_lines = []

for line in file:
    line = line.rstrip()
    if line.find("@") == -1:
        continue
    matching_lines.append(line)

In this generated variation, a list named matching_lines receives each line whose find() result is not -1. The same filtering rule can instead print the line immediately. The important mechanism is unchanged: each line is cleaned, searched, and either skipped or retained.

Tracing an Incorrect Search

Which character position does find() return, and how can that position expose a wrong search or condition?

Tracing the integer returned by find() can reveal what went wrong. In the generated line "From alice@example.com", the at sign starts at position 10, so find("@") returns 10. If the search is changed to find("#"), the return value becomes -1 because that substring is not present. Checking the searched substring and the returned index helps distinguish a missing match from an incorrect expectation about its position.

Mistakes in File Searches

Common mistake
Treating -1 as if it were a valid matching position.
What it looks like
A condition keeps a line when find() returns -1.
Why it fails
-1 means the substring was not found.
Do this instead
Skip when the result equals -1, and keep the line when the result is not -1.
Common mistake
Forgetting that positions are zero-indexed.
What it looks like
Expecting the first character to have position 1.
Why it fails
The first character is at position 0.
Do this instead
Count from 0 when tracing the returned index.
Common mistake
Searching the raw line before removing trailing whitespace.
What it looks like
```
Calling find() before line.rstrip().
```
Why it fails
A file line includes its ending newline, and trailing whitespace can make a search fail unexpectedly or behave strangely.
Do this instead
Call rstrip() before searching each line.
Common mistake
Assuming every nonzero result means no match.
What it looks like
Rejecting a result such as 5 or 10.
Why it fails
Any returned index identifies where the substring starts.
Do this instead
Only -1 means that the substring was not found.

Practice the Filter

MEDIUM

For each generated line, predict the result of find("@") and decide whether the line is kept by the condition find("@") == -1 followed by continue. Then explain why rstrip() belongs before the search.

Hints

Use zero-based positions.
A result of -1 means the line is skipped by this pattern.
A result other than -1 means the line passes through to the print or collection step.

Your attempt (not saved)

Worked example

One line at a time

Trace the generated line "To alice@example.com" when the searched substring is "@".

1
Clean
Call rstrip() so trailing whitespace, including a line-ending newline when present, is removed.
2
Search
The at sign begins at position 8 in the generated line, so find("@") returns 8.
3
Filter
Because 8 is not -1, the skip condition is false and the line is retained.

Result

The line passes the substring filter.

Key Takeaways

1
find() returns the starting index of a found substring.
2
The first character has index 0, and -1 means that the substring was not found.
3
Call rstrip() before searching a line read from a file.
4
A loop, find(), a conditional, and continue can filter file lines by substring presence.
5
When debugging, inspect both the searched substring and the integer returned by find().

At a glance

find() reports where a substring begins, using zero-indexed positions.
A return value of -1 means there is no match.
rstrip() should be called before searching each file line.
Combining find() with a loop and conditional lets you extract matching lines.
Tracing the returned index is a practical way to debug a file-search condition.

Related concepts

More concepts starting with L

Try it: Reading and Writing Text Files

A Search Result with Two Meanings

Mapping a Match to an Index

Preparing Each File Line

Filtering Lines with a Conditional

Walking Through the File

Tracing an Incorrect Search

Mistakes in File Searches

Practice the Filter

Key Takeaways

At a glance

Related Topics

Related concepts