Concepts / Processing Structured Text Files

Processing Structured Text Files

A line read from a text file usually contains an invisible newline character at its end.

  • Programming

A Line Has More Than You See

A line read from a text file usually contains an invisible newline character at its end. That character is part of the string even though it is not visible on the screen. If you pass the line directly to print(), the existing newline and print()'s automatic newline work together, creating an empty line between displayed lines.

What do you think happens?

Suppose line contains the text From: sender followed by a newline character. What happens when print(line) runs?

  • The text appears with no newline

  • The text appears with normal single spacing

  • The text is followed by an extra blank line

Reveal answer
Answer:

The text is followed by an extra blank line.

The newline already stored in the file line moves to the next line, and print() adds another newline after printing the string.

file lineFrom: sender + newlineclean lineFrom: senderprinted textFrom: sender + extranewlinesingle-spaced textFrom: sender + printnewlineprint()print()
What changes when a file line is printed before and after its trailing newline is removed?

Prefix Checks with startswith()

Definition

The startswith() method checks whether a string begins with a specified prefix. It returns True when the beginning matches and False otherwise. It does not search for the prefix anywhere else in the string.

python
Notes on the code
  • The first string begins with From:, so the check returns True.
  • The second string contains similar text later, but it does not begin with From:, so the check returns False.
Output
True
False

This distinction matters when a file uses prefixes to identify different kinds of lines. For example, a structured text file might use From:, Subject:, or Date: at the beginning of lines. A conditional using startswith() can select only the kind of line you need instead of matching the same text when it appears in the middle or at the end.

Removing Trailing Characters

Definition

The rstrip() method removes trailing whitespace and newline characters from a string. It operates at the end of the string, so the meaningful text at the beginning remains unchanged. It can remove multiple trailing whitespace characters, including spaces, tabs, or newlines.

python
Notes on the code
  • The original line ends with spaces and a newline.
  • rstrip() removes those trailing characters.
  • The text Subject: Report remains.
raw lineSubject: Report + spaces +newlineclean lineSubject: Reportrstrip()
Which characters does rstrip() remove, and which part of the line remains?

The Filtering Pipeline

The useful workflow combines both methods. For each line, first remove trailing whitespace with rstrip(). Then check the cleaned line with startswith(). If the Boolean result is True, display the line. If it is False, skip it. Calling rstrip() first creates a clean value for both the test and the output, even though a newline at the end would not change a prefix check at the beginning.

file lineone line from the fileclean linerstrip() appliedprefix resultstartswith("From:")display lineonly when result is Trueskipped linewhen result is FalsereadcleanTrueFalse
How does each file line move through cleaning, prefix checking, and display?
Code walkthrough
for line in file:
    clean_line = line.rstrip()
    if clean_line.startswith("From:"):
        print(clean_line)
  1. 1
    The for loop processes each line from the file.
  2. 2
    rstrip() removes trailing whitespace and the newline character.
  3. 3
    startswith("From:") checks only the beginning of the cleaned line.
  4. 4
    print() displays the line only when the prefix check returns True.
Output
From: sender@example.com
From: archive@example.com

In this generated example, lines beginning with Subject: or other text are not displayed. The selected From: lines appear with single spacing because rstrip() removed their stored newline before print() added its own.

Structured Text in Practice

Structured text files often give different meanings to lines based on their prefixes. Email files can use prefixes such as From:, Subject:, To:, and Date:. Log files can use ERROR:, WARNING:, or INFO: to mark severity levels. The same processing pattern works in both cases: clean each line, test its prefix, and display or process only the lines that match.

Worked example

Selecting Warning Lines

Process lines from a log-like file and display only lines that begin with WARNING:.

  1. 1
    Clean

    Call rstrip() on the line so the trailing newline and other trailing whitespace do not remain in the displayed result.

  2. 2
    Check

    Call startswith("WARNING:") on the cleaned line. The result is True only when WARNING: is at the beginning.

  3. 3
    Display

    Use print() inside the conditional so only matching lines are displayed.

Result

The workflow extracts WARNING: lines and displays them without the extra blank lines caused by printing the original file lines.

Mistakes to Avoid

  • Common mistake

    Treating startswith() as a general search method

    What it looks like
    "Message received from: sender".startswith("From:")
    Why it fails

    The target text is not at the beginning of the string, so startswith() returns False.

    Do this instead

    Use startswith() when the prefix position matters. It checks only the beginning of the string.

  • Common mistake

    Printing the original file line directly

    What it looks like
    print(line)
    Why it fails

    The file line already contains a newline, and print() adds another newline.

    Do this instead
    Call line.rstrip() before printing.
  • Common mistake

    Using rstrip() after the line has already been printed

    What it looks like
    print(line)
clean_line = line.rstrip()
    Why it fails

    The extra spacing has already occurred by the time the cleaned value is created.

    Do this instead

    Clean the line first, then pass the cleaned value to startswith() and print().

  • Common mistake

    Checking one value but displaying a different uncleaned value

    What it looks like
    clean_line = line.rstrip()
if clean_line.startswith("From:"):
    print(line)
    Why it fails

    The condition uses the cleaned line, but the output still contains the original trailing newline.

    Do this instead

    Print clean_line so the value tested and displayed has the same cleaned form.

Practice the Pattern

EASY

Write a loop that processes each line in file and displays only lines beginning with Subject:. Remove trailing whitespace before checking and printing the line.

Hints
  • Create a cleaned value by calling rstrip() on each line.
  • Use startswith("Subject:") as the conditional test.
  • Pass the cleaned value to print().
python
Notes on the code
  • Each file line is cleaned before it is tested.
  • Only lines beginning with Subject: reach print().
  • The cleaned value is printed, preventing double spacing.

Key Takeaways

  1. 1
    startswith() checks only whether a string begins with a specified prefix and returns True or False.
  2. 2
    A line read from a file includes a trailing newline, while print() adds another newline, which can create double spacing.
  3. 3
    rstrip() removes trailing whitespace and newline characters from the end of a string.
  4. 4
    A reliable filtering pattern is to call rstrip(), test the result with startswith(), and print the cleaned line when the test is True.
  5. 5
    Prefix-based filtering is useful for structured text such as email-style files and log files.

At a glance

  • Use startswith() to recognize lines by their beginning, not by text appearing anywhere.
  • Remove a file line's trailing newline with rstrip() before displaying it.
  • Clean each line first, then check its prefix and print only matching lines.
  • This pattern supports practical filtering of structured email and log text files.

Related Topics

More concepts starting with P