The following page documents the required steps for integrating new lectures or laboratories into the RWTHjupyter cluster.

We recommend that each course planning to use Jupyter creates a new profile in the cluster. After login, users will be greeted with a profile-selection page from which they can choose their desired environment. Alternatively, you can add a permalink which allows you to generate a link that directly opens a particular profile and notebook. These links are an adequate method to open Jupyter notebooks from your Moodle course.

A profile needs to be prepared and installed on the cluster before it can be used.

It consists of:

• A name identifying purpose of the profile (or the lecture title).
• A Git repository containing Jupyter notebooks which should be imported into the workspace by default.
• A Dockerfile sepcifying the runtime environment and optional third-party Python packages.

We provide a list of generic profiles for the most common programming languages but offer the possibility to add new profiles for lectures including customizations like custom third-party dependencies, Jupyter kernels, etc.

A key point during the preparation of a new profile is the separation of the runtime environment from the notebook content:

• Separation of runtime environment and Jupyter notebooks
• Warning for Git repositories
• Perform separation

Separation of runtime environment and Jupyter notebooks 

We use nbgitpuller to regularly synchronize Jupyter notebooks from a Git repository with the user's workspace. This allows the instructors to gradually release the notebooks alongside the timeline of the lecture. Each user can maintain a private clone of this Git repository and is able to make permanent changes to the notebooks in that copy.

In contrast, the runtime environment is supposed to be more or less static over the duration of the course and should not require regular updates. All users of a profile share the same runtime environment. Changes outside the home directory are not persistent and will be lost after logout.

Warning for Git repositories

Students will get read-only access to the Git repository you provide. In case you provide us with a token to synchronize private/internal repositories, the students will also see this token.

This means: Only share read-only tokens to repositories which don't contain sensitive information (also in other branches) as students will be able to access this data.

Perform separation

In general, the following steps are required for adding a new profile to the cluster:

1. Set up a local test environment to prepare your Jupyter environment
2. Choose a Jupyter kernel
3. Create a new Git repository for Jupyter notebooks and runtime environment
4. Prepare Jupyter notebooks
5. Collect Python/Third-Party requirements
6. Adapt Dockerfile
7. Submit a request for inclusion in the profile list
8. Wait until the request is reviewed and the new profile is included in the list

Steps 4-6 are optional and only required if the course requires a non-standard Jupyter kernel or special Python dependencies.

1. Set up a local test environment for preparing your Jupyter environment

Please follow the instructions from the official Jupyter website: https://jupyter.org/install

For Windows users, we recommend using the Anaconda Python distribution. Linux and macOS users can use Anaconda as well or simply rely on pip installed via the system's package manager.

2. Choose a Jupyter Kernel

Jupyter supports a variety of different programming languages via different kernels. The original and most commonly used kernel is the one using the IPython interpreter, although alternatives also exist. An updated list of kernels can be found at the Jupyter Wiki.

In principle, almost all of the existing kernels can be used on jupyter.rwth-aachen.de by creating custom runtime environments. Please consult the documentation for the installation of other kernels and add these instructions to the Dockerfile (step 6).

However, in most cases the standard IPython kernel is used and no further steps are necessary.

3. Create a new Git repository for Jupyter notebooks and runtime environment

We recommend to use the RWTH Software Engineering Services to manage, track, distribute Jupyter notebooks and the definition of the required runtime environment.

This allows for a continuous delivery of updates to Jupyter as well as welcomes contributions by students to the Jupyter notebooks.

You can either use an existing Git repository or create a new one by forking our example profile.

Please use the following link to create a new fork of the example profile: create fork of example profile.

After the fork has been completed, you can clone the fresh repository to your local machine:

git clone git@git.rwth-aachen.de:/stvogel/my-new-course.git
cd my-new-course

4. Prepare Jupyter notebooks

A new profile can be filled with a collection of notebooks. These notebooks will be synchronized with the Git repository every time the user logs in.

As such, the collection of notebooks for a course can be expanded during the course of the semester.

To start, please launch Jupyter on your local machine:

5. Collect Python requirements

Depending on the contents of the Jupyter notebook, additional third-party Python packages might be required. Usually these external dependencies are collected in a requirements.txt or environment.yml file:

Please have a look at the following links for further information:

• Creating Conda environments
• PIP Requirements Files

6. Adapt Dockerfile

Some profiles (e.g. when using special Jupyter Kernels) may require additional modifications to the runtime environment beyond the installation of new Python requirements.

To accommodate these the user can modify the included Dockerfile to run arbitrary commands during the preparation of the environment. Note that these commands are only executed during the Docker build phase.

7. Publish Jupyter notebooks and Dockerfile in Git repository

After completing steps 2.- 4., the resulting changes need to be committed to the Git repository and published on a Git hoster (e.g. RWTH GitLab or GitHub):

git add .
git commit -m "first version of new profile"
git push

8. Open request for creation of new RWTHjupyter profile

Please open a new request for the inclusion of your profile into the RWTHjupyter cluster using the following link: Create new profile

The link above will open a form to submit a profile request via the RWTH GitLab system. Please follow the provided template and give us a few days to review your request before we make it available in the cluster.