Concepts / Python File Structure

Python File Structure

A package is a folder containing Python modules and a special __init__.py file that marks it as a package.

  • Programming

From Files to Organization

As a Python project grows, related code needs an organized structure. Python packages provide that structure: a package is a folder containing Python modules and a special __init__.py file that marks the folder as a package. Packages can also contain subpackages, allowing a project to be arranged across multiple levels.

Key point

The organizational pattern is: variables in functions, functions in modules, and modules in packages.

The Containment Hierarchy

A package is not just a collection of unrelated files. It provides a level of containment above individual modules. A module belongs inside a package, and a subpackage is itself a package inside another package. This creates a hierarchy that can group related modules together and then group those groups within a larger project structure.

project_packagereportssubpackagemonthly.pymoduleuserssubpackageannual.pymoduleprofiles.pymodulecontainscontainscontainscontainscontains
What contains what when a Python package is organized into subpackages and modules?

In this generated arrangement, project_package is the outer package. reports and users are subpackages within it. The files monthly.py, annual.py, and profiles.py represent modules grouped according to the subpackage that contains them. The diagram illustrates the source rule that subpackages are packages within packages and can provide multi-level organization.

Package Parts on Disk

The basic physical arrangement has three roles. The package folder provides the container, __init__.py marks that folder as a package, and Python module files hold the modules inside it. When a package contains subpackages, each folder in that package hierarchy must contain its own __init__.py to be recognized as a package.

analyticspackage folder__init__.pymarks the folder as apackagesales.pyPython modulecustomers.pyPython modulecontainscontainscontains
How do the package folder, __init__.py file, and Python module files fit together on disk?
Worked example

A Small Package Arrangement

Arrange two related modules inside a package named analytics.

  1. 1
    Create the folder

    Use analytics as the package folder.

  2. 2
    Add the package marker

    Place __init__.py inside analytics so the folder is marked as a package.

  3. 3
    Add modules

    Place sales.py and customers.py inside analytics as related Python modules.

Result

analytics contains __init__.py, sales.py, and customers.py. The arrangement demonstrates a package containing a special __init__.py file and Python modules.

__init__.py as the Marker

The special filename __init__.py has a structural role: it marks the folder containing it as a package. This is different from an ordinary Python module file. A module is a Python module within the organization; a package is a folder that contains modules and the special marker file.

toolsfolderformat.pymodule filetoolspackage folder__init__.pypackage markerformat.pymodule filecontainscontainscontains
What changes in Python's interpretation of a folder when it contains an __init__.py file?

Module or Package

reports.pysingle module filereportspackage folder__init__.pypackage markerdetail.pymodulecontainscontains
What is the structural difference between a single Python module and a package containing multiple modules?
StructureWhat it containsOrganizational role
ModuleA Python moduleA unit inside the file structure
PackagePython modules and __init__.pyA folder that organizes modules

The comparison is structural: a package is a folder that contains modules and the special __init__.py file.

The distinction is about level and containment. A module is one part of the organization, while a package is a containing folder that groups modules. A package may also contain subpackages, so it can represent a larger level of organization than a single module.

Building Subpackages

To organize a larger project, place related modules in a subpackage and place that subpackage inside an outer package. The outer package and every subpackage must contain __init__.py. This preserves the package hierarchy at every folder level.

apppackage__init__.pypackage marker__init__.pypackage markerdatasubpackagerecords.pymodulereportssubpackage__init__.pypackage markersummary.pymodulecontainscontainscontainscontainscontainscontainscontains
How should folders and files be arranged to create a package with related modules grouped into subpackages?
  1. 1
    Choose an outer package folder, such as app.
  2. 2
    Place __init__.py inside the outer folder.
  3. 3
    Create subpackage folders for related groups, such as data and reports.
  4. 4
    Place __init__.py inside every subpackage folder.
  5. 5
    Place related Python modules inside the appropriate subpackage.

In the generated arrangement, records.py belongs to the data subpackage, while summary.py belongs to the reports subpackage. Both subpackages belong to app. The repeated __init__.py files are what mark app, data, and reports as package folders throughout the hierarchy.

Mistakes in Package Layouts

  • Common mistake

    Treating a package as though it were only one module file.

    What it looks like

    Thinking that a package is equivalent to reports.py.

    Why it fails

    A package is a folder containing Python modules and __init__.py.

    Do this instead

    Look for the containing folder, its __init__.py file, and the modules inside it.

  • Common mistake

    Adding __init__.py only to the outer folder.

    What it looks like

    Creating app/__init__.py but leaving app/data without __init__.py.

    Why it fails

    Every folder in the package hierarchy must contain __init__.py to be recognized as a package.

    Do this instead

    Place __init__.py in the outer package and in each subpackage folder.

  • Common mistake

    Putting related modules at unrelated levels.

    What it looks like

    Placing one data-related module in the outer package and another in a separate subpackage without an organizing reason.

    Why it fails

    Packages and subpackages are intended to organize modules into a hierarchy.

    Do this instead

    Group related modules within an appropriate package or subpackage.

Practice the Structure

MEDIUM

Design a package structure for a project that has two groups of related modules: customer-related modules and report-related modules. Include one outer package, one subpackage for each group, and the required __init__.py file in every package folder.

Hints
  • Start with one outer package folder.
  • Create customer and report subpackage folders inside it.
  • Add __init__.py to the outer folder and to both subpackage folders.
  • Place each related module inside its matching subpackage.
Key point

Check your answer at three levels: the outer package, each subpackage, and the modules contained by those folders. Every package folder needs __init__.py.

Structure Checklist

  1. 1
    A package is a folder containing Python modules and a special __init__.py file.
  2. 2
    The __init__.py file marks its containing folder as a package.
  3. 3
    Packages organize modules into a hierarchy, and subpackages are packages within packages.
  4. 4
    Every folder in a package hierarchy must contain __init__.py to be recognized as a package.
  5. 5
    A useful organization pattern is variables in functions, functions in modules, and modules in packages.

Key Takeaways

  • A module is a Python module within a file structure; a package is a folder that contains modules and __init__.py.
  • __init__.py marks its folder as a package.
  • Packages can contain subpackages, creating multiple levels of organization.
  • Every folder in the package hierarchy needs __init__.py.
  • The hierarchy follows the pattern of variables in functions, functions in modules, and modules in packages.

Related Topics

Related concepts

More concepts starting with P