You are located in service: RWTH High Performance Computing (Linux)

Upload Files to the Cluster

Upload Files to the Cluster

CLAIX provides storage space to each of its users and there are many ways to transfer files to and from CLAIX.

Large transfers are best conducted through the dedicated copy-login-nodes.

Mind that transferring files needs the same prerequisites as to login to the cluster.

Table of Contents

  1. Command Line Tools
  2. WinSCP
  3. FastX File Manager
  4. HPC JupyterHub
  5. See also

Command Line Tools

SFTP

From your terminal you can use the OpenSSH tool sftp to transfer files.

Connect to CLAIX with sftp:

sftp <user-id>@copy23-1.hpc.itc.rwth-aachen.de

Replace <user-id> with your credentials (e.g. ab123456). After entering your password and MFA code, you enter an interactive sftp shell. Here, there some basic commands available for navigation and file management. Mind that if you want to switch to $WORK or $HPCWORK, you will need to use the full path:

sftp> cd /hpcwork/<user-id>

To upload a file to CLAIX use the following command. Mind that sftp> is the prompt, that indicates that the shell is ready to accept commands.

sftp> put <local_file>

To download a file from CLAIX to your computer, use:

sftp> get <remote_file>

After you have transferred all files, you can exit the shell using:

sftp> quit

Rsync

If you are using a Linux or Mac system, you can use the more advanced rsync:

The default operations of rsync is as follows

rsync OPTIONS SOURCE DESTINATION

To upload a local file to your $HOME directory on CLAIX, use the following command on your local terminal:

rsync -aP -e ssh <your-file> <user-id>@copy23-1.hpc.itc.rwth-aachen.de:/home/<uder-id>/

To download a file to your current working directory, use:

rsync -aP -e ssh <user-id>@copy23-1.hpc.itc.rwth-aachen.de:/home/<user-id>/<your-file> .

Here, -a tells rsync to preserve the metadata of your file, e.g. the creation date, -P gives a shows the progress and -e ssh tells rsync to use the ssh protocoll for file transfer, which is more secure than the default rsh protocoll.

WinSCP

If you would like to transfer files from a Windows system to CLAIX, you can use WinSCP.

When starting WinSCP you will see the following window, where you can enter the hostname and your credentials.

WinSCP Log In

After entering your password and MFA code, you can drag and drop files from your computer to CLAIX and vice versa.

Mind that when you want to access your $WORK and $HPCWORK directories, you need specify these explicitly. For this click on the button that is highlighted with a red circle in the image below.

WinSCP window after login. Red circle shows the button that needs to be pressed in order to specify a directory.

In the new window you then need to enter the path to your $WORK or $HPCWORK directory:

WinSCP window for opening a directory

 

Using the FastX File Manager

If you use the FastX web interface to interact with CLAIX, you can use the provided file manager. Mind that you should only use this for small file transfers.

For more information see the official user documentation.

 

Web Interface with HPC JupyterHub

It is also possible to upload files through the HPC JupyterHub interface. Please use this method only for small file transfers.


See also

last changed on 07/09/2024

How did this content help you?

Creative Commons Lizenzvertrag
This work is licensed under a Creative Commons Attribution - Share Alike 3.0 Germany License