This tutorial covers releasing a Python distribution locally using GateWay as an example. It focuses on how to package a simple project in Python. It will show you how to add the necessary files and structure to create a package, how to build a package, and how to install it on any platform.
1. Preparatory activities
Please note that these steps are not mandatory to proceed successfully, however they are considered good practice.
1.1. PIP upgrade
Before any Python package activity, it is always a good move to upgrade your main Python package manager, such as PIP. This can be done from any location, however, sometimes the user is unable to do so due to file system permissions. In this case, it can be done for a local user using the --user option on the command line.
This tutorial is based on using GateWay, therefore a Python module shall be loaded first.
1.1.1. With necessary permissions
$ module load itm-python
$ python3 -m pip install --upgrade pip
1.1.2. Without necessary permissions
This solution is not recommended, but may be needed under certain circumstances, such as when the user does not have permissions.
$ module load itm-python
$ python3 -m pip install --upgrade pip --user
In that case it may be needed to add a local site-packages directory to $PATH:
$ PATH=$HOME/.local/bin:$PATH
1.2. Virtual Environment
At its core, the main purpose of Python virtual environments is to create an isolated environment for Python projects. This means that each project can have its own dependencies, regardless of what dependencies every other project has. The great advantage is that there is no limit to the number of environments that can be created as it is just a directory with a few scripts that handle system package isolation.
As shown in the previous section, there are at least two different locations where Python packages can be installed on the system. This can pose risks such as name shadowing or even suppression of certain packages from the kernel to the system. Most of the time, nothing like this happens, but even so, using a Python virtual environment is also considered good practice. This tutorial assumes usage of at least Python 3.6, therefore it should already have the venv module from the standard library installed.
1.2.1. Creating a Virtual Python Environment
# Python 3
$ python3 -m venv env-name
And that's just it. It is important to use the venv module as others are deprecated. The name of the environment can be anything as it is just a directory name, however it is good practice to use names like env or venv or expand with these names.
1.2.2. Activate The Virtual Environment
To activate the environment just created, navigate to the location where it was created, if not already there, and execute the following command:
$ source env-name/bin/activate
(env-name): $
As a result of activation, the currently used shell should explicitly show the name of the environment being activated on the command line.