Accounting with Core Hours

Definition of a Core Hour

In the HPC world, consumption of computing resources is measured in core hours (core-h). As the name suggests, one core-h typically represents the usage of one CPU core for one hour. However, it really represents the extend to which any given computing ressource was used. This is especially important when nodes are shared among users.

Accounting on Exclusive Nodes

Exclusive use of any given node is billed the same for any partition within a specific cluster. On Claix-2023 the exclusive use of one node costs 96 core-h.

Accounting on Shared Nodes

The cluster supports shared node usage, which introduces complexity to the billing process. The reason is that the limiting resources extend beyond CPU cores to include memory (RAM) and possibly GPUs. When you share a node with other users, you are billed based on the resource you use the most.

Billing for fractional node usage is calculated in three ways:

CPU usage: c_c = #cores • time
Memory usage: c_m = x • M • 96 • time
GPU usage (if applicable): c_g = #gpu • 24 • time

In these formulas, #cores is the number of cores used, M is the total available memory on the node, x is the fraction of M used, #gpu is the number of gpus occupied.
For the billing the greatest of the three (c_c, c_m, c_g) will be used.

Accounting Examples

Here are some examples of what 1000 core-h equates to on Claix-2023:

• 10.4 hours exclusive use of any given node.
• Usage of a single GPU for 42 hours, provided no more than 24 CPU cores and 64 GB of memory is used.
• Utilizing 128 GB of memory on a node with 256 GB and no more than 48 cores for 20.8 hours.

Computing Time Use and Monitoring

Sliding Window Quota

Computing time quotas are allocated on a monthly basis. However, recognizing that it may be challenging to utilize the quota precisely each month, a sliding window quota is implemented. This approach allows you to carry over unused quota from the previous month and to draw in advance from the next month's quota. Naturally, the order in which allocated computing time is consumed is as follows: first previous month, then current month, lastly next month.

For instance, if your monthly quota is 2000 core-h and you haven't used any computing time in the previous and current month, you can access up to 6000 core-h.

Checking the Computing Time Quota

To review the computing time usage of the last six months, use the command r_wlm_usage. Adding the parameter -p <project-ID> will show a report of the consumption within the provided project. This report includes the usage by all users involved in the project.

To check the remaining sliding window quota, total quota, and additional information about the computing time project use the additional parameter -q. You will see an output similar to the example below.

Account:                               example1234
Type:                                      example
Start of Accounting Period:             08.05.2023
End of Accounting Period:               07.05.2024
State of project:                           active
--------------------------------------------------
Quota monthly (core-h):                      10000
Remaining  core-h of prev. month:             -100
Consumed   core-h current month:              1000
Consumed   core-h last 4 weeks:               8000
Consumable core-h (%):                          89
Consumable core-h:                            8900
--------------------------------------------------
Total quota (core-h):                       120000
Total consumed core-h so far:                60000
--------------------------------------------------
Default partition:                           c23ms
Allowed partitions:              c23ml,c23ms,c23mm
Max. allowed wallclocktime:             24.0 hours
Max. allowed cores per job:                     96

To view additional parameters and report settings of r_wlm_usage, use the parameter -h or -man.

Computing Time Quota Limitations

Exceeding your sliding window quota leads to notable changes in job processing. In such cases, further job submissions will be put on a low-priority Slurm queue, resulting in significantly longer wait times for job starts.

If a project's consumption exceeds six times its monthly core-h quota within a four-week period, new job submissions will be temporarily suspended until the usage drops below this threshold. Using more than double the total project core-h quota will permanently disablefurther job submissions the project's period. These restrictions similarly apply to personal compute time quotas.

Please be aware that in times of high demand, we may adjust these limits to ensure fair distribution of ressources.