File Systems Overview
For each CLAIX user and computing time project space is allocated on the CLAIX storage systems to facilitate their research and computational tasks. The cluster offers various file systems, each with its unique advantages and limitations. The appropriate file system for any workflow depends on the task at hand.
You are primarily responsible for backing up your data. The HPC cluster is not intended as a long-term data storage solution. It is crucial to regularly backup data to avoid any potential loss.
The three permanent file systems are as follows:
- $HOME: Upon logging in, users are directed to their personal home-directory within $HOME. With an integrated backup solution, this file system is particularly well-suited for the storage of important results that are challenging to reproduce, as well as for the development of code projects. However, frequent and massive file transfers, creation, and deletion should be avoided.
- $WORK: This file system is on the same storage system as $HOME, but has no backup solution.
- $HPCWORK: This file system is on a different storage system than $HOME and $WORK and offers better I/O performance. In addition, it provides an HDD and a SSD storage pool. By default, all files are stored on the HDD pool. For specific tasks that require very fast I/O, you may use the limited SSD storage.
To switch between the filesystems, use cd $HOME
, cd $WORK
and cd $HPCWORK
.
Some additional key points include:
- To check your file system quota, please use r_quota.
- $HOME and $WORK offer snapshots, which allows you to access previous states of your files.
- File system quotas for individual user are not extendable. However, for project storage it can be upon request.
- The computing nodes of CLAIX-2023 also offer local SSD storage, which can be used with BeeOND.
In the following table you can see a summary of all and additional details of the available file systems discussed before.
File System | Path | Persistence | Snapshots | Backup | Quota (space) | Quota (#files) | Use Cases |
---|---|---|---|---|---|---|---|
$HOME | /home/<username> | permanent | $HOME_SNAPSHOT | yes | 250 GB | 1 million | Source code, configuration files, important results |
$WORK | /work/<username> | permanent | $WORK_SNAPSHOT | no | 250 GB | 1 million | I/O intensive computing jobs |
$HPCWORK | /hpcwork/<username> | permanent | - | no | 1000 GB | 1 million | I/O intensive computing jobs large files |
$BEEOND | stored in $BEEOND | temporary | - | no | HPC Segment: ML | - | I/O intensive computing jobs, many small working files, any kind of scratch data |