Jupyter notebook inside a virtual environment

Follow these steps to run a jupyter notebook inside a python virtual environment where you can install all the packages you need for your project.  Using virtual environments will allow you to keep projects completely separate with different packages, different python versions, etc.

Initial setup

The initial creation of the virtual environment and python kernel should be done outside of jupyter.  This is due to the fact that jupyter was built with a specific version of python.  If you use jupyter to create the environment it is more complicated (although not impossible) to use a different version of python.

Note: The current jupyter lab installation will work with kernels built on python version 3.12 and higher.  Older versions of python are available on the HPC cluster, but they will fail to launch kernels in jupyter.  Please open a ticket with support if you need to use jupyter with an older version of python.

Create the virtual environment and install the python kernel

  1. This can be done either over SSH or through Open OnDemand.  Log into whichever is your preference
    • SSH: <your utcid>@login01.mocshpc.utc.edu
    • Open OnDemand - https://ondemand.mocshpc.utc.edu/ - After logging in, in the top menu click on "Clusters" -> "mocshpc Shell Access"
  2. In the terminal window, enter the following commands. I'm using "jupyter-tensorflow" as the project name, but you can use whatever you want. Just make sure to use that name everywhere you see my example name. 
    # Change to the directory where you want to store your environment
cd ~/HPC

# Load the python module you want to use.  To see what versions are available, you can use "module avail python"
module load python/3.12.12

# Create a virtual environment
python -m venv jupyter-tensorflow 

# Change into the new environment's directory
cd jupyter-tensorflow

# Activate the new virtual environment. Your CLI prompt will change after this - e.g. (jupyter-tensorflow)
source bin/activate

# Install a new kernel in your user account for this new environment
pip install ipykernel
python -m ipykernel install --user --name=jupyter-tensorflow --display-name="Jupyter TensorFlow Test"

# Install any other packages you need for your project
pip install <other things you need, like tensorflow>
  3. Once everything is installed, close the terminal window or log out of SSH.

Start a new project using your virtual environment

  1. Login to Open OnDemand - https://ondemand.mocshpc.utc.edu/
  2. Select "JupyterLab" from the "Interactive Apps" menu
  3. Fill out the form selecting the node type and resources you need for your code to run
  4. After filling out the form, click "Launch"
  5. After the session has started, click "Connect to Jupyter"
  6. In the new window you will see the Jupyter launcher. Click on either the "Notebook" or "Console" buttons labeled with your evnrinment name.
  7. Do your work in the new window that opens up

If you want to resume work on a project that uses the virtual environment

  1. Just open the file you were working on the last time and it will load up in the virtual environment

If you ever decide to delete the virtual environment

  1. Login to Open OnDemand - https://ondemand.mocshpc.utc.edu/
  2. Select "JupyterLab" from the "Interactive Apps" menu
  3. Fill out the form selecting the node type and resources you need for your code to run
  4. After filling out the form, click "Launch"
  5. After the session has started, click "Connect to Jupyter"
  6. In the new window you will see the Jupyter launcher. Click on the "Terminal" button under the "Other"section.  You can also open a new terminal by selecting the "File" menu -> "New" -> "Terminal"
  7. In the terminal window that opens, enter the following commands. I'm using "jupyter-tensorflow" as the project name, but you can use whatever you want. Just make sure to use that name everywhere you see my example name. 
    # Remove the ipykernel
jupyter kernelspec remove jupyter-tensorflow

# Remove the virtual environment.  Be aware that if you have any code stored in this folder that it will get deleted also.  You should back up your work first if there is anything you want to save.
rm -r -f jupyter-tensorflow
Print Article