Available Filesystems (RWTH High Performance Computing Linux)
The following table gives an overview of all file systems that each user has at their disposal:
|Source code, configuration files|
|Output files, working data|
IO intensive jobs, large files
Parallel high performance file system designed for high throughput when working with few large files.
|$TMP||Local (Ext4/XFS)||n.a.||yes||-||no||-||limited by the size of local disk||-|
local scratch data
|Local on each node. The space is shared by all processes(/jobs/users for non-exclusive case) of a batch job on the node. Limited space available - see size of local disks in Batch Systems (SLURM) and note that some 40GB per node is used by the operating system.|
|$BEEOND||BeeGFS On Demand https://www.beegfs.io/wiki/BeeOND||n.a.||yes||-||no||-|
limited by the sum of sizes of local disks, e.g.:
|-||IO intensive jobs, large amount of small files, any kind of scrath data||Local for the batch job, shared for all processes on each node belonging to the job. Limited space available - see size of local disks in Batch Systems (SLURM) and note that some 40GB per node is used by the operating system.|
Each user file system is only mounted when a process actually accesses it. Therefore, you might not see a specific user directory in a listing of
/hpcwork (which can be confusing, especially if you are using a graphical file manager). This does not mean that a user directory does not exist, but that you might have to type the path to a user directory explicitly to actually get there. Any action that causes an access to your directory will mount it and thereby make it visible. This can be done by changing into the directory in a terminal or typing the full path of your directory in the address bar of your file manager.
We currently remove outdated files in
$TMP only. The
$BEEOND directories belonging to a batch job are removed immediately after the job ends. The lifetime policies regarding
$HPCWORK are subject to change however, i.e. we may decide to clean up outdated files in these areas, too, if space starts to get short.
We make a backup of
$HOME on a daily basis. For all other areas no backup is being made.
The following files and directories are currently excluded from the tape backup:
- Complete sub-directories:
- File patterns:
Snapshots reflect the state of a file system at previous points in time. By changing to a snapshot directory, e.g.
$WORK_SNAPSHOT, you can access previous versions of your files. The files within the snapshots are read-only, they can not be altered or deleted. Please note that the snapshots creation policy is subject to change. If space gets short, we may decide to create less snapshots, delete existing snapshots or omit them completely.
Please Note: Snapshots are not an alternative to a tape backup. If a file system gets damaged, all snapshots are lost, too. This means that you should not store files in
$HPCWORK or even
$TMP that you cannot reproduce with little effort.