Concepts / File handling basics

File handling basics

For file handling, this protocol means the file is acquired before processing begins and closed after processing ends.

  • Programming
Interactive lab

Try it: Context Managers and the with Statement

What with really does: it calls __enter__ before the block and __exit__ after it — even when an exception leaves the block — so a file opened with with is always closed before the exception continues; try/finally gives the same guarantee by hand, a bare close() is skipped by the exception, and a context manager whose __exit__ returns True suppresses the exception.

How it works

  1. with EXPR as f: evaluates EXPR (e.g. open(...)), calls its __enter__() and binds the returned value to f (a file's __enter__ returns the file itself).
  2. The block runs. If a statement raises (here int() on text that is not a number), the rest of the block is skipped.
  3. Whether the block finished or an exception is leaving it, Python calls __exit__(exc_type, exc, tb) — for a file this closes it. This happens before any outer except sees the exception.
  4. If __exit__ returns a true value the exception is suppressed and execution continues after the block; a file's __exit__ returns None, so the exception propagates to the nearest matching except (or stops the program).
  5. try: … finally: f.close() gives the same cleanup by hand (start with f = None and check it); open() … f.close() without either leaves the file open when an exception jumps past close().

Default run (10 steps): The program opens data.txt, which contains '4x\n'. Watch when the file is closed — and what happens if int() raises ValueError. … The program finished. The file was closed.

Simplified: Fixed program templates reading one line of data.txt (text you type, up to 12 ASCII characters) and converting it with int(). A small interpreter written for these statements runs them — nothing is executed — and it reproduces CPython 3.12's sys.settrace line/call/exception events, printed output, final error line and file state for every combination. File objects' own __enter__/__exit__ are C code, so they appear as protocol steps rather than traced lines.

Educational simulation

Loading the simulation…

The Risk of an Unreleased File

Opening a file claims a resource from the operating system. That resource remains claimed until the file is closed. If the program crashes before reaching a close operation, the file can remain open, wasting system resources and potentially preventing other programs from accessing it. File handling therefore has a simple lifecycle: acquire the resource, use it, and release it.

Python provides the with statement to manage this lifecycle automatically. Instead of separately remembering to close a file after every possible path through the program, you place the file operation inside a with block. Python then handles the cleanup when the block ends.

Acquire fileresource obtainedUse filewith block runsLeave blocknormal exit or exceptionClose fileautomatic cleanupthenexecution continuesalways
What happens to a file resource from acquisition through use to automatic cleanup when execution leaves the with block?

Tracing the File State

What do you think happens?

A file is opened inside a with block, and an exception occurs while the block is processing the file. What happens to the file?

  • The file remains open because the block ended abnormally.

  • The file is closed automatically before the exception continues.

  • The file is closed only if the program catches the exception.

Reveal answer
Answer:

The file is closed automatically before the exception continues.

The with protocol calls __exit__ after the block, including when an exception is raised. The cleanup occurs before the exception reaches an exception handler or causes the program to stop.

Unopenedno resource claimedOpenresource in useClosedresource releasedresource acquisitionblock ends or exception occurs
How does a file change between unopened, open, and closed states as the with statement runs?
Worked example

Processing a File Safely

A program needs to open a file and iterate through its lines.

  1. 1
    Acquire

    The with statement performs the file resource acquisition, typically using open for a file.

  2. 2
    Bind

    The acquired resource is associated with the variable named after as.

  3. 3
    Process

    The indented block iterates through the file's lines while the file is open.

  4. 4
    Release

    When execution leaves the block, the file closes automatically. No explicit close call is needed.

Result

The file is used during the block and cleaned up automatically afterward.

The with Protocol

The with statement works through a protocol: a set of rules describing how the statement interacts with the resource object. The protocol calls __enter__ before the with block begins and __exit__ after the block finishes. This is what connects resource acquisition and cleanup to the block's control flow.

Definition

__enter__ is the protocol step called before the code inside the with block. __exit__ is the protocol step called after the block, including when an exception is raised inside it.

with statementresource supplied__enter__before blockBlock bodyfile processing__exit__after blockResource releasedcleanup completecallsthennormal or exceptional exitcleanup
How do __enter__ and __exit__ get called before and after the body of a with statement?

For file handling, this protocol means the file is acquired before processing begins and closed after processing ends. If processing raises an exception, __exit__ still runs before the exception reaches an exception handler or stops the program. The cleanup guarantee is therefore tied to leaving the block, not only to successful completion.

Writing the Resource Pattern

The structure of a file-handling with statement is: write with, then the resource acquisition, typically open for files; write as; provide a variable name; add a colon; and place the file-processing code in the indented block. The block is the region in which the resource is used.

  • with introduces the managed resource statement.
  • The resource acquisition, typically open for files, obtains the file resource.
  • as associates the acquired resource with a variable name.
  • The colon begins the block whose resource use is managed.
  • The indented block contains the file-processing operations.
  • Leaving the block triggers automatic cleanup.

with Compared with try...finally

Manual resource management uses a try...finally pattern so that cleanup code runs whether the protected operations finish normally or raise an exception. For a file, that approach requires the programmer to arrange an explicit close operation in the finally part.

The with statement provides the same important cleanup guarantee without requiring the repeated manual pattern. It encodes the acquisition and cleanup behavior into the language construct, making the result shorter, clearer, and less error-prone. A programmer cannot accidentally omit the finally block or its close operation when using the managed pattern.

with statementcleanup encodedtry...finallycleanup writtenBlock endsautomatic cleanuptry endsfinally runsException raised__exit__ still runsException raisedfinally still runsnormal control flowerror control flownormal control flowerror control flow
How does control flow differ between a with statement and an explicit try...finally block when normal execution or an error occurs?
with statementManual try...finally
Acquisition and cleanup are expressed through one language construct.The programmer writes the cleanup structure explicitly.
The resource is cleaned up when the block ends or an exception occurs.The finally part is used to guarantee cleanup after normal or exceptional control flow.
Shorter, clearer, and less error-prone for this recurring pattern.More verbose and easier to get wrong if cleanup code is omitted.

Both approaches provide a cleanup guarantee, but with directly encodes the resource-management pattern.

Mistakes Beginners Make

  • Common mistake

    Assuming cleanup happens only after successful processing.

    What it looks like

    A file-processing operation raises an exception inside the with block.

    Why it fails

    The with protocol calls __exit__ even when an exception is raised.

    Do this instead

    Treat the with block as safe for both normal completion and exceptional completion.

  • Common mistake

    Replacing the managed pattern with a manually remembered close operation.

    What it looks like

    The program opens a file and plans to close it later, but an earlier operation raises an exception.

    Why it fails

    Control may leave the intended path before the close operation is reached, leaving the resource open.

    Do this instead

    Use a with statement so cleanup is connected to leaving the block.

  • Common mistake

    Putting file processing outside the with block.

    What it looks like

    The file is acquired in the managed statement, but later processing is planned after the block.

    Why it fails

    The with statement closes the file when the block ends, so later operations are outside the resource's managed use period.

    Do this instead

    Place the operations that use the file inside the indented with block.

Practice the Lifecycle

MEDIUM

Describe the order of events for a with statement that opens a file, processes two lines, and then encounters a ValueError while still inside the block. Name the protocol steps and state when the file is closed.

Hints
  • Identify the protocol method called before the block.
  • The exception occurs during the block, not before resource acquisition.
  • Identify what runs after the block when an exception occurs.
Worked example

Checking Your Trace

Trace the managed file lifecycle through normal entry, file processing, an exception, and cleanup.

  1. 1
    Entry

    __enter__ is called before the file-processing block.

  2. 2
    Processing

    The block processes the file's lines.

  3. 3
    Exception

    A ValueError occurs during processing, so normal completion of the block does not continue.

  4. 4
    Cleanup

    __exit__ runs before the exception reaches an exception handler or stops the program, and the file closes.

Result

An exception changes how execution leaves the block, but it does not remove the with statement's cleanup step.

Key Takeaways

  1. 1
    Opening a file claims an operating-system resource that must be released.
  2. 2
    The with statement combines resource acquisition, use, and automatic cleanup.
  3. 3
    __enter__ runs before the with block, while __exit__ runs after it.
  4. 4
    A file opened with with is closed when the block ends, including when an exception occurs.
  5. 5
    For recurring resource-management tasks, with is shorter, clearer, and less error-prone than manual try...finally code.

At a glance

  • The with statement manages the lifecycle of a file resource.
  • __enter__ is called before the block and __exit__ is called after it.
  • Cleanup occurs when the block ends normally or when an exception leaves it.
  • The with pattern avoids the repeated boilerplate and risks of manual try...finally resource management.

Related Topics

Related concepts

More concepts starting with F