Packages and Imports#

This document explains how the codebase is structured in terms of Python packages and modules, and what the import rules are. Read this before writing any code that imports from torx.

What is a Python package?#

A Python package is a directory with an __init__.py file. When Python imports a package, it executes the __init__.py. This file is where the package defines its public API by explicitly re-exporting names from its internal module files.

torx/
    __init__.py      ← executed when you write `import torx` or `from torx import ...`
    grid/
        __init__.py  ← executed when you write `from torx.grid import ...`
        grid_2d_m.py
        grid_3d_m.py

The __init__.py typically looks like this:

# torx/grid/__init__.py
from .grid_2d_m import Grid2D
from .grid_3d_m import Grid3D

This means Grid2D and Grid3D are part of the public API of torx.grid. A developer using the package only needs to know about torx.grid, not about grid_2d_m.py or grid_3d_m.py.

Note

The order of the imports in the init file matters.

What is a sub-package?#

A sub-package is a package nested inside another package. It has its own __init__.py and its own public API.

torx/                        ← root package
    __init__.py
    equilibrium/             ← sub-package of torx
        __init__.py
        numerical_m.py
        io/                  ← sub-package of torx.equilibrium
            __init__.py
            netcdf_io_m.py

Sub-packages are imported using their full dotted path:

from torx.equilibrium import NumericalEquilibrium
from torx.equilibrium.io import read_netcdf_equilibrium

What is a module?#

A module is a single .py file containing implementation code. In this codebase, module files are named with a _m suffix, for example polygon_2d_m.py. This convention makes it immediately clear that a file is an internal implementation detail and not a package.

Warning

We enforce the name convention _m (except __init__.py files) in the tests.

Modules are never imported directly from outside the package. They are an implementation detail. Their public names are re-exported through the parent package’s __init__.py.

# yes: import from the package
from torx.geometry import Polygon2D

# no: import from the module file directly
from torx.geometry.polygon_2d_m import Polygon2D

The benefit is that the internal file structure can change (files split, renamed, moved) without breaking any code that uses the package.

What about folders without __init__.py?#

Not every folder inside a package is a sub-package. Some folders exist purely for organization, grouping related module files together. These folders do not have an __init__.py and are not importable as a package.

torx/
    analysis/
        __init__.py              ← analysis is a sub-package
        integrations_m.py
        smoothing/               ← no __init__.py, just a folder for organization
            smoothing_1d_m.py
            smoothing_2d_m.py

Here torx.analysis.smoothing is not importable. The names from smoothing_1d_m.py and smoothing_2d_m.py are imported by the module files in torx/analysis/ using relative imports, and then re-exported through torx/analysis/__init__.py as part of the torx.analysis public API.

As a developer you do not need to know that smoothing/ exists. You just import from torx.analysis.

Import Rules#

There are two reasons we have established import rules:

  1. Avoid circular imports: Python initializes modules sequentially. If module A imports from module B, and module B imports from module A, Python cannot finish initializing either. This is a circular import and will raise an ImportError at startup. In a large codebase with many inter-dependencies this can easily happen by accident. The import rules prevent this by enforcing a clear direction of dependencies: relative imports within a sub-package avoid triggering the sub-package __init__.py, and absolute imports across sub-packages make the dependency direction explicit.

  2. Forces a clean API interface: By requiring that all external code imports from the package or sub-package level only, we are forced to think carefully about what is part of the public API and what is an implementation detail. Every name that a user can import must be explicitly re-exported through an __init__.py. This means:

    • Internal files can be renamed, split or reorganized without breaking any external code.

    • The __init__.py files serve as a clear contract of what the package provides.

    • Users never need to know about the internal module structure, they only need to know the package name.

Without these rules, a large codebase tends to accumulate implicit dependencies between internal modules, making it increasingly difficult to refactor, test or understand the code.

Rule 1: from outside, never import from a module file directly#

From anywhere outside the package (notebooks, scripts, tests, other packages), always import from the package or sub-package, not from a module file:

# yes
from torx.geometry import Polygon2D
from torx.equilibrium import NumericalEquilibrium
from torx.equilibrium.io import read_eqdsk_file

# no: _m suffix means it is a module file, never import from it directly
from torx.geometry.polygon_2d_m import Polygon2D
from torx.equilibrium.numerical_m import NumericalEquilibrium
from torx.equilibrium.io.netcdf_io_m import read_eqdsk_file

Note

The only exception to this rule are tests where for coverage reasons we also test internal functionality.

Rule 2: inside the package, use relative imports within the same sub-package#

When writing code inside a module file that needs to import from another module in the same sub-package, use a relative import:

# inside torx/geometry/polygon_2d_m.py

# yes: relative import from the same sub-package
from .utils_m import some_helper
from .intersection_m import intersect_lines

Why? Because a relative import does not trigger the sub-package __init__.py. If you used an absolute import like from torx.geometry.utils_m import ..., Python would execute torx/geometry/__init__.py, which itself imports from utils_m.py, causing a circular import.

Rule 3: inside the package, use absolute imports across sub-packages#

When a module file needs something from a different sub-package or from the root package, use an absolute import:

# inside torx/geometry/polygon_2d_m.py

# yes: absolute import from a different sub-package
from torx.grid import Grid2D

# no: relative import across sub-package boundaries is fragile and confusing
from ..grid import Grid2D

Summary#

Location

Importing from

Use

Outside torx/

anywhere in torx

absolute, package level only

Inside torx/geometry/

same sub-package

relative

Inside torx/geometry/

different sub-package

absolute

Enforcement#

The rules above are checked automatically by infra/pre_commit/check_imports.py. Run it before submitting code:

# check internal import style within the package
python infra/pre_commit/check_imports.py torx/

# check a directory for invalid imports from the package
python infra/pre_commit/check_imports.py torx/ --directory notebooks/

The checker reports the following violations:

Violation

Meaning

[DIRECT MODULE IMPORT]

Importing directly from a _m module file

[SAME (SUB-)PACKAGE]

Should use a relative import

[DIFFERENT (SUB-)PACKAGE]

Should use an absolute import

[NOT EXPORTED]

Name is not exported from the sub-package __init__.py

[EXTERNAL]

Import from a _m module file from outside the package

If the checker reports a [NOT EXPORTED] violation, it means the name you are trying to import exists in a module file but has not been added to the __init__.py of the sub-package. The fix is to add it to __init__.py:

# torx/geometry/__init__.py
from .new_module_m import NewClass   # add this line