# Jupyter Notebooks

Jupyter notebooks are a great way to perform post-processing. They let you interactively explore your data and quickly iterate by writing small code blocks.

## Launch locally

TorX comes with a copy of Jupyter installed. If you're running TorX on your local machine (i.e. laptop), you can launch a Jupyter session by running

```bash
source <path to torx home>/venv
jupyter lab
```

or alternatively

```bash
jupyter notebook
```

This should open a web-browser (generally `localhost:8888/`) where you can create and open notebooks.

To check which kernels are available, run

```bash
jupyter kernelspec list
```

## Use port forwarding

If you're running on a remote machine (i.e. the TOK clusters), you can use port forwarding. In one terminal window run (where `XXXX` is some 4-digit number)

```bash
source <path to torx home>/venv
jupyter lab --no-browser --port=XXXX
```

and then in another terminal run (where `YYYY` is another four-digit number)

```bash
localuser@localhost: ssh -N -f -L localhost:YYYY:localhost:XXXX remoteuser@remotehost
```

then open `localhost:YYYY/` in a web-browser on your local machine. Note that this doesn't work for all machines.

## Use RVS

If you want to do your analysis on a MPCDF machine, you can use the [remote visualization service](https://rvs.mpcdf.mpg.de/rv/). You'll first need to click `Initialise Remote Visualization` for the machine that you want to run on (only needs to be done once), then SSH into the machine and install TorX (see the [installation guide](#installation)) to get a TorX kernel on the machine. Next, launch a RVS session using the web-interface, and once you've opened a notebook select `torx (env)` in the list of kernels, or `torx (<name>)` if you installed the environment with `--name <name>`.

RVS launches JupyterLab from the `Anaconda/3` module and never activates a TorX environment, so the environment a notebook runs in is decided solely by the kernel you select.
- The kernel records the full path of its interpreter, so the environment must live somewhere the compute node can read, such as your home directory rather than node-local scratch.
- If you use FLARE, install with `-fr`. The install script then records `LD_LIBRARY_PATH` in the kernel, because RVS cannot module-load libraries that set it from inside a Jupyter session.
- The MPCDF Jupyter does not have `jupytext` installed, so notebooks are not paired automatically. Editing a notebook in a RVS session changes only the `.ipynb`. To update the committed `.py`, run `nbsync` (see [sync notebooks](#nbsync)) in a terminal with the TorX environment activated.

(switching-kernels)=
## Choose which environment a notebook uses

Each TorX environment installs its own Jupyter kernel, named after the environment: an environment installed with `--name v2` gives a kernel displayed as `torx (v2)`. This is what lets you keep several environments side by side and run one notebook against one environment and another notebook against another. Run `jupyter kernelspec list` to see which kernels you have.

A notebook records only the *name* of its kernel. The path to the Python interpreter lives in the kernel itself, not in the notebook. Note that:

- Moving or renaming an environment directory breaks its kernel, because the recorded interpreter path no longer exists. Reinstall the environment or manually update the path in the notebook metadata to fix it.
- If you install a new environment under a different name, an existing notebook still asks for the old kernel. If that old environment is still present, the notebook keeps running against it, and nothing tells you that you are not using the new one. If the old environment has been deleted, the kernel fails to start instead.

Either way, you have to point the notebook at the new kernel yourself. There are two ways to do this.

### From within Jupyter

Open the notebook, choose `Kernel` -> `Change Kernel...`, select `torx (v2)`, and save. This keeps the cell outputs, so it is the right choice for a notebook whose results you want to hold on to. It works for any notebook, including ones you created outside the TorX repository.

### From the command line

`nbsync` writes the kernel of the currently active environment whenever it generates an `.ipynb`, so converting a notebook to `.py` and back re-points it:

```bash
source <path to v2 env>/bin/activate
nbsync -t py -p mynotebook.ipynb -o
nbsync -t nb -p mynotebook.py -o
```

The `-p` flag takes any path, so the notebook does not need to live in the repository or be registered in `notebooks/notebooks_m.py`.

```{warning}
The `.py` representation does not store cell outputs, so this round trip clears them. Use `Change Kernel` in Jupyter instead if you want to keep your results.
```

Notebooks tracked in the repository need no manual step, since they are generated from `.py` files that deliberately carry no kernel. Regenerate them with the environment you want activated:

```bash
nbsync -t nb -k <key> -o
```

This is also the fix for a notebook written before the kernel was named after its environment, which may still refer to a kernel called `torx` that does not exist on a fresh installation.

### Remove old kernels

Installing a new environment never removes the kernel of an old one, so kernels of environments you have deleted linger in the list. Remove them yourself:

```bash
jupyter kernelspec list
jupyter kernelspec uninstall torx-v1
```