Compute time quota
What is a core-hour?
Usage of RWTH compute cluster's resources is accounted in core-hours. One core-hour equals one CPU core being used for the duration of one hour of execution time. The latter is always measured as the elapsed wall clock time from the job start to the job finish and not as the actual CPU time. Also note that since jobs in the JARA queue use compute nodes exclusively, usage is always equal to the total number of CPU cores on the allocated nodes times the execution time, regardless of the actual number of node slots allocated to the job.
Accounting is implemented as a three months wide sliding window. Each month a certain amount of compute time is granted, hereby referred as monthly allowance. Unused quota from the previous month, up to the monthly allowance, is transferred automatically to the current month but not further. Because of the limit on the amount of quota transferred and because the transfer does not occur past the next month, it is not possible to "save" compute time and accumulate it for later usage. It is also possible to borrow compute time from the next month's allowance. Usage of compute time transferred from the previous month or borrowed from the following month is counted towards the current month's usage. Transfer and borrow only occur if the respective month is within the accounting period.
The core-hours quota available for use in the current month is computed as follows:
- The monthly allowance for the previous, the current, and the next month are added together.
- The consumed core-hours for the previous and for the current month are added together.
- The difference between both values is the amount of core-hours available in the current month.
Once the quota has been fully consumed, all new and pending jobs will only get dispatched if there are no pending jobs from other projects with unused CPU quota, i.e. in a low-priority mode. Jobs that run in low-priority mode are still counted towards the core-hour usage for the current month. Further job submission is prevented as soon as one of the following conditions hold:
The monthly quota was exceeded by a factor of 6.
- The total project quota was exceeded by a factor of 2.
Note that according to this model usage in the current month of either transferred or borrowed time has a negative impact on the next month's allowance. For example (the current month is italicised):
|Consumed core-hours||0||70000||up to 80000|
In this scenario 50000 unsed core-hours from January were transferred into the next month for a total of 100000 core-hours available in February, of which 70000 core-hours were used. This leaves the project with only 80000 core-hours In March (= 3 x 50000 - 70000) since the 20000 core-hours consumed from what was transferred from January were accounted as February usage.
|Consumed core-hours||0||120000||up to 30000|
In this scenario 50000 unused core-hours from January were transferred to and consumed in February. Also, 20000 core-hours were borrowed from March. In March the project could only use up to 30000 core-hours (= 3 x 50000 - 120000) with all of them borrowed from April. The capacity to use the monthly allowance in its entirety will be restored again in April.
This accounting model is designed to encourage you to spread your usage as evenly as possible throughout the compute period.
Monitoring quota utilization
You can query the status of your core-hours quota using the r_wlm_usage [-p <project-num>] -q command:
|$ r_wlm_usage -p <project-num> -q|
Start of Accounting Period: 01.01.2012
End of Accounting Period: 30.06.2012
State of project: active Quota
monthly (core-h): 1000
Remaining core-h of prev. month: 200
Consumed core-h act. month: 0
Consumable core-h (%): 120
Consumable core-h: 2200
Without the -p option the status of your own open-use quota is shown. In the example above 1000 hours per month are available. In the previous month only 800 hours have been used leaving a total of 1200 core-hours (120%) for the current month. Borrowing all of next month’s quota up to 2200 cores-hours can be used. The displayed percentage value ranges from 200% (no core-hours were used during the previous and the current month) down to -101% (the combined usage for the current and the previous month is more than the three months' allowance), with negative values indicating that the combined usage for the current and previous months is more than twice the monthly quota and therefore compute time from the next month is being borrowed. Once the percentage value drops below -100%, the project enters low-priority mode.
Note that the r_wlm_usage command shows consumable core-hours values less than -100% as -101%.
Please examine this Excel spreadsheet to better understand the formula which is used to calculate the quota at the beginning of each month.