# Packages and Imports

This document explains how the codebase is structured in terms of Python
packages and modules, and what the import rules are. Read this before
writing any code that imports from `torx`.

---

## What is a Python package?

A Python package is a directory with an `__init__.py` file. When Python
imports a package, it executes the `__init__.py`. This file is where the
package defines its **public API** by explicitly re-exporting names from
its internal module files.

```
torx/
    __init__.py      ← executed when you write `import torx` or `from torx import ...`
    grid/
        __init__.py  ← executed when you write `from torx.grid import ...`
        grid_2d_m.py
        grid_3d_m.py
```

The `__init__.py` typically looks like this:

```python
# torx/grid/__init__.py
from .grid_2d_m import Grid2D
from .grid_3d_m import Grid3D
```

This means `Grid2D` and `Grid3D` are part of the public API of `torx.grid`.
A developer using the package only needs to know about `torx.grid`, not about
`grid_2d_m.py` or `grid_3d_m.py`.

```{note}
The order of the imports in the __init__ file matters.
```

---

## What is a sub-package?

A sub-package is a package nested inside another package. It has its own
`__init__.py` and its own public API.

```
torx/                        ← root package
    __init__.py
    equilibrium/             ← sub-package of torx
        __init__.py
        numerical_m.py
        io/                  ← sub-package of torx.equilibrium
            __init__.py
            netcdf_io_m.py
```

Sub-packages are imported using their full dotted path:

```python
from torx.equilibrium import NumericalEquilibrium
from torx.equilibrium.io import read_netcdf_equilibrium
```

---

## What is a module?

A module is a single `.py` file containing implementation code. In this
codebase, module files are named with a `_m` suffix, for example
`polygon_2d_m.py`. This convention makes it immediately clear that a file
is an internal implementation detail and not a package.

```{warning}
We enforce the name convention `_m` (except `__init__.py` files) in the tests.
```

Modules are **never imported directly** from outside the package. They are
an implementation detail. Their public names are re-exported through the
parent package's `__init__.py`.

```python
# yes: import from the package
from torx.geometry import Polygon2D

# no: import from the module file directly
from torx.geometry.polygon_2d_m import Polygon2D
```

The benefit is that the internal file structure can change (files split,
renamed, moved) without breaking any code that uses the package.

---

## What about folders without `__init__.py`?

Not every folder inside a package is a sub-package. Some folders exist
purely for **organization**, grouping related module files together. These
folders do not have an `__init__.py` and are not importable as a package.

```
torx/
    analysis/
        __init__.py              ← analysis is a sub-package
        integrations_m.py
        smoothing/               ← no __init__.py, just a folder for organization
            smoothing_1d_m.py
            smoothing_2d_m.py
```

Here `torx.analysis.smoothing` is not importable. The names from
`smoothing_1d_m.py` and `smoothing_2d_m.py` are imported by the module files
in `torx/analysis/` using relative imports, and then re-exported through
`torx/analysis/__init__.py` as part of the `torx.analysis` public API.

As a developer you do not need to know that `smoothing/` exists. You just
import from `torx.analysis`.

---

## Import Rules

There are two reasons we have established import rules:

1. **Avoid circular imports**: Python initializes modules sequentially. If module
   A imports from module B, and module B imports from module A, Python cannot
   finish initializing either. This is a circular import and will raise an
   `ImportError` at startup. In a large codebase with many inter-dependencies
   this can easily happen by accident. The import rules prevent this by
   enforcing a clear direction of dependencies: relative imports within a
   sub-package avoid triggering the sub-package `__init__.py`, and absolute
   imports across sub-packages make the dependency direction explicit.

2. **Forces a clean API interface**: By requiring that all external code imports
   from the package or sub-package level only, we are forced to think carefully
   about what is part of the public API and what is an implementation detail.
   Every name that a user can import must be explicitly re-exported through an
   `__init__.py`. This means:
   - Internal files can be renamed, split or reorganized without breaking
     any external code.
   - The `__init__.py` files serve as a clear contract of what the package
     provides.
   - Users never need to know about the internal module structure, they only
     need to know the package name.

Without these rules, a large codebase tends to accumulate implicit dependencies
between internal modules, making it increasingly difficult to refactor, test or
understand the code.

### Rule 1: from outside, never import from a module file directly

From anywhere outside the package (notebooks, scripts, tests, other
packages), always import from the package or sub-package, not from a
module file:

```python
# yes
from torx.geometry import Polygon2D
from torx.equilibrium import NumericalEquilibrium
from torx.equilibrium.io import read_eqdsk_file

# no: _m suffix means it is a module file, never import from it directly
from torx.geometry.polygon_2d_m import Polygon2D
from torx.equilibrium.numerical_m import NumericalEquilibrium
from torx.equilibrium.io.netcdf_io_m import read_eqdsk_file
```

```{note}
The only exception to this rule are tests where for coverage reasons we
also test internal functionality.
```

### Rule 2: inside the package, use relative imports within the same sub-package

When writing code inside a module file that needs to import from another
module in the **same sub-package**, use a relative import:

```python
# inside torx/geometry/polygon_2d_m.py

# yes: relative import from the same sub-package
from .utils_m import some_helper
from .intersection_m import intersect_lines
```

Why? Because a relative import does not trigger the sub-package `__init__.py`.
If you used an absolute import like `from torx.geometry.utils_m import ...`,
Python would execute `torx/geometry/__init__.py`, which itself imports from
`utils_m.py`, causing a **circular import**.

### Rule 3: inside the package, use absolute imports across sub-packages

When a module file needs something from a **different sub-package** or from
the **root package**, use an absolute import:

```python
# inside torx/geometry/polygon_2d_m.py

# yes: absolute import from a different sub-package
from torx.grid import Grid2D

# no: relative import across sub-package boundaries is fragile and confusing
from ..grid import Grid2D
```

### Summary

| Location | Importing from | Use |
|---|---|---|
| Outside `torx/` | anywhere in `torx` | absolute, package level only |
| Inside `torx/geometry/` | same sub-package | relative |
| Inside `torx/geometry/` | different sub-package | absolute |

---

## Enforcement

The rules above are checked automatically by
`infra/pre_commit/check_imports.py`. Run it before submitting code:

```bash
# check internal import style within the package
python infra/pre_commit/check_imports.py torx/

# check a directory for invalid imports from the package
python infra/pre_commit/check_imports.py torx/ --directory notebooks/
```

The checker reports the following violations:

| Violation | Meaning |
|---|---|
| `[DIRECT MODULE IMPORT]` | Importing directly from a `_m` module file |
| `[SAME (SUB-)PACKAGE]` | Should use a relative import |
| `[DIFFERENT (SUB-)PACKAGE]` | Should use an absolute import |
| `[NOT EXPORTED]` | Name is not exported from the sub-package `__init__.py` |
| `[EXTERNAL]` | Import from a `_m` module file from outside the package |

If the checker reports a `[NOT EXPORTED]` violation, it means the name you
are trying to import exists in a module file but has not been added to the
`__init__.py` of the sub-package. The fix is to add it to `__init__.py`:

```python
# torx/geometry/__init__.py
from .new_module_m import NewClass   # add this line
```