Upload Files to the Cluster
CLAIX provides storage space to each user and there are many ways to transfer files to and from the HPC system.
Large transfers are best conducted through the dedicated copy-login-nodes.
Prerequisites: Please ensure that you have completed all the prerequisite steps to access CLAIX.
Table of Contents
SFTP
From your terminal you can use the OpenSSH tool sftp to transfer files.
Connect to CLAIX with sftp:
sftp <user-id>@copy23-1.hpc.itc.rwth-aachen.de
Replace <user-id> with your credentials (e.g. ab123456). After entering your password and MFA code, you enter an interactive sftp shell. Here, there some basic commands available for navigation and file management. Mind that if you want to switch to $WORK or $HPCWORK, you will need to use the full path:
sftp> cd /hpcwork/<user-id>
To upload a file to CLAIX use the following command. Mind that sftp> is the prompt, that indicates that the shell is ready to accept commands.
sftp> put <local_file>
To download a file from CLAIX to your computer, use:
sftp> get <remote_file>
After you have transferred all files, you can exit the shell using:
sftp> quit
Rsync
If you are using a Linux or Mac system, you can use the more advanced rsync:
The default operations of rsync is as follows
rsync OPTIONS SOURCE DESTINATION
To upload a local file to your $HOME directory on CLAIX, use the following command on your local terminal:
rsync -aP -e ssh <your-file> <user-id>@copy23-1.hpc.itc.rwth-aachen.de:/home/<uder-id>/
To download a file to your current working directory, use:
rsync -aP -e ssh <user-id>@copy23-1.hpc.itc.rwth-aachen.de:/home/<user-id>/<your-file> .
Here, -a tells rsync to preserve the metadata of your file, e.g. the creation date, -P gives a shows the progress and -e ssh tells rsync to use the ssh protocoll for file transfer, which is more secure than the default rsh protocoll.
If you would like to transfer files from a Windows system to CLAIX, you can use WinSCP.
When starting WinSCP you will see the following window, where you can enter the hostname and your credentials.
After entering your password and MFA code, you can drag and drop files from your computer to CLAIX and vice versa.
Mind that when you want to access your $WORK and $HPCWORK directories, you need specify these explicitly. For this click on the button that is highlighted with a red circle in the image below.
In the new window you then need to enter the path to your $WORK or $HPCWORK directory:
If you use the FastX web interface to interact with CLAIX, you can use the provided file manager. Mind that you should only use this for small file transfers.
For more information see the official user documentation.
Web Interface with HPC JupyterHub
It is also possible to upload files through the HPC JupyterHub interface. Please use this method only for small file transfers.
