The following page documents the required steps for integrating new lectures or laboratories into the RWTHjupyter cluster.

We recommend that each course which plans to use Jupyter, creates a new profile in the cluster. After login, users will be greeted with a profile selection page from which they can choose their desired environment. We also will add a perma-link feature which allows you to generate a link which directly opens a particular profile & notebook. These links are an adequate form to open Jupyter Notebooks from your Moodle course.

A profile needs to be prepared and installed to the cluster before it can be used.

It consists of:

• A name identifying purpose of the profile (or the lecture title)
• A Git repository containing Jupyter notebooks which should be imported into the workspace by default
• A Dockerfile sepcifying the runtime environment and optional third-party Python packages

We provide a list of generic profiles for the most common programming languages but offer the possibility to add new profiles for lectures including customizations like custom third-party dependencies, Jupyter kernels, etc.

Separation of runtime environment and Jupyter notebooks

A key point during the preparation of a new profile is the separation of the runtime enviroment from the Notebook content.

We use nbgitpuller to reguarily synchronize Jupyter notebooks from a Git repositry with the users workspace. This allows the instructors to gradually release the Notebooks alongside the timeline of the lecture. Each user can maintain a private clone of this Git repository and is able to make permanent changes to the Notebooks in that copy.

In contrast, the runtime enviroment is supposed to me more or less static over the duration of the course and should not require regular updates. All users of a profile share the same runtime environment. Changes outside the home directory are not presistent and will be lost after logout.

Warning for Git repos

Students will get full read-only access to the Git repository you provide. In case you provide us with a token to synchronize private/internal repositories, the students will also see this token.

This means: Only share read-only tokens to repositories which dont contain sensitive information (also in other branches) as students will be able to access this data.

Procedure

In general, the following steps are required for adding a new profile to the cluster:

1. Setup a local test environment to prepare your Jupyter Environment
2. Choose a Jupyter Kernel
3. Create a new Git repository for Jupyter Notebooks and runtime environment
4. Preparartion of Jupyter Notebooks
5. Collect Python/Third-Party requirements
6. Adapt Dockerfile
7. Submit request for inclusion in profile list
8. Wait until request is reviewed and the new profile is included in the list

Hereby, steps 4.-6. are optional and only required if the course requires a non-standard Jupyter kernel or special Python dependencies.

1. Setup a local test environment for preparing your Jupyter Environment

Please follow the instructions from the official Jupyter website: https://jupyter.org/install

For Windows users, we recommend to use the Anaconda Python distribution. Linux and macOS users can use Anaconda as well or simply rely on pip installed via the systems package manager.

2. Choose a Jupyter Kernel

Jupyter supports a variety of different programming languages via different kernels. The original and most commonly used kernel using the IPython interpreter. However also others exist. An updated lists of kernels can be found at the Jupyter wiki.

In principle, almost all of the existing kernels can be used on jupyter.rwth-aachen.de by creating custom runtime environments. Please consult the documentation for the installation of other kernels and add these instructions to the Dockerfile (step 6).

However, in most cases the standard IPython kernel is used and no further steps are necessary.

3. Create a new Git repository for Jupyter Notebooks and runtime environment

We recommend to use the RWTH Software Engineering Services to manage, track, distribute Jupyter Notebooks and the definition of the required runtime environment.

This allows for a continous delivery of updates to Jupyter as well as welcomes contributions by Students to the Jupyter notebooks.

You can either use an existing Git repository, which might be already existing for a lecture or create a new one by forking our example profile.

Please use the following link to create a new fork of the example profile: create fork of example profile.

After the fork has been completed, you can clone the fresh repository to your local machine:

4. Prepare Jupyter Notebooks

A new profile can be filled with a collection of notebooks. These notebooks will be synchronized with the Git repository everytime the user logs in.

As such, the collection of Notebooks for a course can be expandend during the course of the semester.

To start, please launch Jupyter on your local machine:

5. Collect Python requirements

Depending on the contents of the Jupyter Notebook, additional third-party Python packages might be required. Usually these external dependencies are collected in a requirements.txt or environment.yml file:

Please have a look at the following links for further information:

• Creating Conda environments
• PIP Requirements Files

6. Adapt Dockerfile

Some profiles (e.g. when using special Jupyter Kernels) may require additional modifications to the runtime environment beyond the installation of new Python requirements.

To accomodate these the user can modify the included Dockerfile to run arbitrary commands during the preperation of the environment. Note that these commands are only executed during the Docker build phase.

7. Publish Jupyter Notebooks and Dockerfile in Git repository

After completing steps 2.- 4., the resulting changes need to be commited to the Git repository and published on a Git hoster (e.g. RWTH GitLab or GitHub):

8. Open request for creation of new RWTHjupyter profile

Please open a new request for the inclusion of your profile into the RWTHjupyter cluster using the following link: Create new profile

The link above will open a form to submit a profile request via the RWTH GitLab system. Please follow the provided template and give us a few days to review your request before we make it available in the cluster.