What is TorX#
TorX is a Python post-processing framework for codes built on parallax. It contains several distinct components which are be described here.
The torx environment#
TL;DR: The torx environment gives you a version of Python with several libraries (like numpy or xarray) that help for post-processing. It is installed by following the installation instructions.
This is a virtual environment which contains several different packages that help post-process our data. You can think of this as a local copy of the Python executable which which has lots of useful extra packages installed. These packages (plus a short description of what they do) are listed in infra/requirements.in.
If you follow the installation instructions, you’ll get an extra subfolder in your torx top-level directory, called env by default, which contains the environment. If you ls env/bin you’ll see lots of executables, including python. If you have your TorX environment activated (using source venv from the TorX home folder), you’ll have access to all of those packages. Activating the environment modifies your $PATH such that when you ask for an executable like python or dask, it uses the version in env/bin. You can run which python to check this.
A nice thing about virtual environments is that you can also install more packages. You can either add packages to infra/requirements.in if you want a package to be permanently available, or you can also activate the torx environment and run pip install <package name>. If it all goes terribly wrong, you can rm -rf env to completely remove the environment and start again.
Note that you can rename your TorX environment and even install a new one with a different name by providing the --name or -n argument to the install script. For example ./install.sh -n my_env names the environment my_env instead of env.
An install prefix, --prefix_env can also be provided here which installs the TorX environment to a different folder. For example, ./install.sh -n my_env --prefix_env /envs/folder/ installs an environment named my_env to the /envs/folder folder.
The torx library (torx/torx/)#
The torx library is all of the code contained in the torx subfolder (i.e. torx/torx). The library contains many self-contained functions and tools which can be mixed and matched to suit your needs. For example, there are equilibria (which are compatible with PARALLAX equilibria), tools for reading PARALLAX meshes, a field-line tracer, a Fourier analysis tool, and more. These tools use the packages in the torx environment.
Like the third-party packages, the torx library is ‘installed’ in the torx environment, and so you can use it just like any other package (i.e. import torx or from torx import Quantity).
To see the tools from the library in action, take a look at the example TorX notebooks.
We’ll discuss in the rest of the documentation the functionality included in the library. Feel free to write alternatives to the library functionality or to extend it. If you write something that you think is useful, please put it into the library (and add tests) so others can use it.
The torx Jupyter kernel#
One line executed during the installation is ./env/bin/python -m ipykernel install --user --name torx-env --display-name "torx (env)". This lets you use the torx environment in a Jupyter notebook, instead of the system-default python.
The kernel is named after the environment, so installing with --name v1 instead gives a kernel torx-v1, shown in Jupyter as torx (v1). Every environment therefore gets its own entry in the kernel list, and a notebook is run against an environment by selecting its kernel. The kernel is installed for the user rather than into the environment, so it is offered by any Jupyter session, including one started by the remote visualization service, which never activates a TorX environment.
The kernel that a notebook uses is stored in the .ipynb file and not in its .py representation, so switching a notebook between environments does not change anything that is committed. Notebooks created before this change may still refer to a kernel called torx; regenerate them with nbsync -t nb -k <key> -o.
Jupyter is a nice web-based interface which lets you interactively execute code blocks. You can launch a session using the rvs.mpcdf.mpg.de service, or by activating the TorX environment and then running jupyter lab. If you want to run on a remote server, you can use port-forwarding:
ssh -N -f -L localhost:<port>:localhost:<port> user@server
ssh user@server
source <path to torx env>/bin/activate
jupyter lab --port <port> --no-browser
and then opening localhost:<port> in your local web browser.
Example notebooks (torx/notebooks/)#
This folder contains example notebooks showing how you can perform analyses with the torx library and environment. Since .ipynb notebook files are difficult to review with git, we use Jupytext to convert them to .py python files. This also lets us run the notebooks in the pipeline, to make sure that they run through without errors. See Jupyter and Jupytext for more details.