Concepts / Opening and Reading Files

Opening and Reading Files

The file search pattern reads through a file line by line, applies a filter condition using string methods, and processes only matching lines.

  • Programming
Interactive lab

Try it: Reading and Writing Text Files

How open() returns a file object with a position pointer, how read(), read(n), readline(), readlines() and for line in fh advance it (and return '' at the end), how 'w' and 'a' modes change the file, and why a closed file (after close() or a with block) can no longer be read.

How it works

  1. open(name, 'r') starts the pointer at 0; 'w' empties the file first; 'a' starts at the end.
  2. read() returns everything from the pointer to the end; read(n) at most n characters; readline() up to and including the next \n; each moves the pointer past what it returned.
  3. At the end of the file every read returns '' (an empty string), which is how loops know to stop; for line in fh reads line by line the same way.
  4. write() text waits in a buffer and reaches the file when it is closed (or the program ends); reading a 'w'/'a' file or writing an 'r' file raises io.UnsupportedOperation.
  5. close(), or leaving a with block, closes the file; any further read raises ValueError: I/O operation on closed file.

Default run (10 steps): Ready: notes.txt holds 4 lines (51 characters). Nothing is open yet. … Finished. notes.txt now holds "From: stephen\nSubject: meeting\nFrom: louis\nThanks!\n".

Simplified: The lab's own small Python interpreter runs a fixed, hand-written script (you choose the variant and the values, never the code); nothing is executed as real Python. Every script was run in CPython 3.12 with the same inputs and the lab reproduces its output and error messages. One virtual text file with \n line endings (as on Linux/macOS), positions counted in characters (ASCII only), and a write buffer that is only flushed by close() or the end of the program.

Educational simulation

Loading the simulation…

Why Targeted Searches Matter

A text file may contain hundreds, thousands, or even millions of lines, while your task may require only a small subset. For example, a log file may contain many kinds of entries, but you may need only the errors. A contact list may contain many names, but you may be looking for one particular name. In these situations, processing every line in the same way is unnecessary. A targeted file search reads the file, checks each line against a condition, and processes only the lines that match.

Key point

The central pattern is read, filter, process. The program reads a line, filters it with a search condition, and processes it only when the condition is true.

Read lineone line from the fileFilterstring conditionProcess linematching lines onlyNext linenon-matching line skippedline arrivesmatchno match
How does each line move from being read, through a search condition, to being processed only when it matches?

Reading One Line at a Time

The read step moves through the file line by line. The search does not need to treat the whole file as one large piece of data. Instead, each line reaches the condition in turn. After the program makes a decision about the current line, it continues to the next line.

Line 1current lineLine 2next lineLine 3next lineRemaining linescontinue until file endsafter decisionafter decisionrepeat
How does the program progress through the file one line at a time rather than loading and processing everything at once?

Filtering with String Methods

The filter is the decision point in the pattern. A condition examines the current line and determines whether it belongs in the result. Python string operations can test several kinds of matches. The in operator checks whether specific text appears anywhere in a line. The startswith method checks whether a line begins with a chosen prefix. The find method searches for a substring. These checks can be placed inside an if statement in the file-reading loop.

python

In this generated example, the loop reads one line at a time. The condition checks whether the text ERROR appears in the current line. Only a line that passes this condition reaches print. The strip method removes surrounding whitespace from the displayed line, which is useful because lines read from a text file may include surrounding whitespace such as a line ending.

Current linetext from fileSearch conditioncontains target text?Process linecondition is trueSkip linecondition is falseNext linecontinue searchtesttruefalsethen continuecontinue
What happens next when a line meets the condition, and what happens when it does not?

A Complete Search Trace

Worked example

Finding Error Entries

Search a log file and process only lines containing the text ERROR.

  1. 1
    Read

    The file-reading loop obtains one line from the log file.

  2. 2
    Filter

    The condition checks whether ERROR occurs in that line. This uses the in string operation.

  3. 3
    Process a match

    If the condition is true, the line is sent to the processing action, such as displaying it.

  4. 4
    Skip a non-match

    If the condition is false, no processing action is applied to that line, and the search continues with the next line.

  5. 5
    Repeat

    The same read-and-filter decision is made for each line in the file.

Result

Only log lines containing ERROR are processed, while other lines are skipped.

Every lineread and processMatching linespass conditionProcessing actionapplied to all linesProcessing actionapplied to matchesalways processprocess after match
What is the difference between applying processing to every line and applying it only after the search condition matches?

Both approaches read through the file, but the targeted pattern limits the processing step. This matters when the file is large and the useful result is small. The program still examines each line to decide whether it matches, but it avoids applying the later processing action to lines that fail the condition. It also avoids loading the entire file into memory at once.

Mistakes That Change Search Results

  • Common mistake

    Forgetting to strip whitespace before processing a matching line

    What it looks like

    A displayed result may include surrounding whitespace or a line ending.

    Why it fails

    The search can still identify the line, but the processed output may not have the clean form you expect.

    Do this instead

    Strip surrounding whitespace when the processing step requires a cleaned line.

  • Common mistake

    Using a case-sensitive comparison without checking the intended capitalization

    What it looks like

    A condition looking for ERROR may not match a line containing error.

    Why it fails

    The condition may evaluate as false even though the line represents the kind of entry you intended to find.

    Do this instead

    Handle capitalization deliberately when choosing the search condition.

  • Common mistake

    Processing every line instead of placing the action after the condition

    What it looks like

    The file is read, but the processing action runs regardless of whether the search text appears.

    Why it fails

    The program loses the main benefit of a targeted search and performs unnecessary work.

    Do this instead

    Put the processing action inside the matching branch so that only lines passing the filter move forward.

  • Common mistake

    Not closing the file after reading

    What it looks like

    A file is opened without a structure that closes it when the reading work is finished.

    Why it fails

    The source emphasizes closing files after reading as an important file-handling practice.

    Do this instead

    Use a file-handling approach that ensures the file is closed after the search.

Practice the Pattern

EASY

Design a Python file search for a contact list that processes only lines containing the name Mira. State what the read step does, write the filter condition in plain language, and describe what happens to a line that does not contain Mira.

Hints
  • Use the three labels read, filter, and process.
  • The filter should test whether the target text appears in the current line.
  • A non-matching line should be skipped rather than processed.
MEDIUM

Compare two designs for a large log file: one processes every line, and one processes only lines that satisfy a string-based condition. Explain why the second design is more efficient for a search that needs only a small subset of entries.

Hints
  • Both designs may read through the file.
  • Focus on which lines receive the later processing action.
  • Also consider whether the entire file needs to be loaded into memory.

The Search Pattern in Practice

Use this pattern when a text file is large or when only selected entries matter. Begin by reading the file line by line. Choose a string-based condition that expresses the search: in for text appearing anywhere, startswith for a required beginning, or find for locating a substring. Put the processing action behind the condition. This creates a single-pass search that avoids unnecessary processing and avoids loading unnecessary data into memory.

  1. 1
    Read each line from the file.
  2. 2
    Filter the current line with a condition based on string methods.
  3. 3
    Process only the lines for which the condition is true.
  4. 4
    Skip non-matching lines and continue to the next line.
  5. 5
    Strip whitespace, handle case sensitivity carefully, and close the file after reading.

Key Takeaways

  • File searching follows a read, filter, process sequence.
  • String operations such as in, startswith, and find can form the filter condition.
  • A matching line moves to processing, while a non-matching line is skipped.
  • The pattern is efficient because it makes one pass and avoids unnecessary processing and memory use.
  • Whitespace, capitalization, and file closing require deliberate attention.

Related Topics

Related concepts

More concepts starting with O