Python/Conda

User-space python installation.

A lot of libraries for machine learning (scikit-learn), numerical computing (numpy, scipy), and data visualization (matplotlib, pandas, seaborn) are available in the form of python packages.

We keep the default 'base' environment slim, created system-wide conda environments and installed the following frameworks, shown below with (env_name):: packages format

• torch :: latest stable GPU-enabled torch, torchaudio, torchvision, and torchtext

• tf-keras-jax :: latest stable GPU-enabled tensorflow 2, keras, and jax

You can activate these global environments with

// conda activate torch # or tf-keras-jax

While pip let's you easily install new packages, it doesn't fully account for their dependencies which may result in malfunctionings or unreplicable results. Therefore, we use conda;a package and environment manager to create and maintain multiple development environments at the same time.

Some packages may not exist in conda; as a last resort, you can then use pip from within the conda environment:

pip install --user <PACKAGE_NAME>

If absolutely needed, you can create a new environment called <ENV_NAME>at your user space, by

conda create --name <ENV_NAME>

You can then activate the environment by running:

conda activate <ENV_NAME>

You will know that the environment is active because its name will appear between parentheses at the beginning of the command prompt. Once an environment is active, you can access all the python packages installed within. You can also use the command above to switch between existing environments.

By default, Conda environments are stored in the user’s home directory ($HOME/.conda/envs/).

In order to install the custom package <PACKAGE_NAME> within the environment, make sure the environment is active then type:

conda install <PACKAGE_NAME>

You can search for python packages on this repository, which also contains detailed instruction on how to install most packages.

Other commands

• Update conda:

  conda update conda

• Update a package in the environment:

  conda update <PACKAGE_NAME>

• Remove a package from the environment:

  conda remove <PACKAGE_NAME>

• List environments:

  conda info --envs

• List packages inside current environment:

  conda list

• Delete environment:

  conda env remove --name <ENV_NAME>

The following commands can be used to share environments.

• Export environment into .yml file:

  conda env export > environment.yml

• Create environment from .yml file:

  conda env create -f environment.yml

For more commands, refer to this page.