The Nvidia CUDA Toolkit provides support for CUDA.

Depending on the version, you might have to load additional modules until you can load CUDA:

ml CUDA

Available CUDA versions can be listed with ml avail CUDA. If you need a specific version, load it explicitly, e.g. ml CUDA/12.6.0.

Table of Contents

1. NVIDIA CUDA
2. CUDA + MPI
3. CUDA Samples
4. PGI CUDA Fortran

NVIDIA CUDA

Loading the module will prepend to your $PATH and $LD_LIBRARY_PATH variables the correct locations and furthermore will provide the variable $CUDA_ROOT, which contains the root directory of the loaded toolkit installation. The currently loaded version can also be obtained by nvcc --version. For documentation on CUDA, please visit https://docs.nvidia.com/cuda

You can compile and link e.g. your CUDA program called pi.cu by:

nvcc pi.cu

If you want to set a certain computate capability, you can use the corresponding compiler flag. For example, to set compute capability 9.0 for the H100 GPUs available in CLAIX do:

nvcc -arch=sm_90 pi.cu
 

CUDA + MPI

Combining MPI and CUDA is a possibility to scale over several nodes. For instance, you can run one MPI process per machine while each process uses one (or two if available) GPUs. Another possible scenario for dual-GPU machines is to specify that there should be two MPI processes per node and each of them uses one GPU.

To use CUDA with MPI, our recommendation is to compile your CUDA code with nvcc and then link it with $MPICC or $MPICXX by explicitly specifying the CUDA libraries. For example:

nvcc -arch=sm_90 -m64 -c foo.cu -o foo.o
$MPICXX -c bar.cpp -o bar.o
$MPICXX foo.o bar.o -o foobar.exe -L$CUDA_ROOT/lib64 -lcudart

Run your program with $MPIEXEC.
 

CUDA Samples

The NVIDIA GPU Computing SDK provides a lot of examples in CUDA C. They can be used to verify the correct setup of the GPU (see examples deviceQuery and bandwithTest), as a starting point for your own application and to give you the idea of how to implement certain algorithm on a GPU. You can find the how-to of the SDK here.
 

CUDA Forward Compatibility

If an application is linked against a newer CUDA version than what the currently installed driver supports, you can use the CUDAcompat module to provide forward compatibility: 

ml CUDAcompat

This makes it possible to run applications built with a newer CUDA toolkit than the one provided by our current system driver. For further details, please refer to NVIDIA's documentation at https://docs.nvidia.com/deploy/cuda-compatibility/forward-compatibility.html