Ansys Suite (ansys, cfx, fluent, icem)
1. Ansys
1.1 How to Access the Software
Load the ANSYS module
$ module load TECHNICS ansys # or load specific version $ module load TECHNICS ansys/22.1 |
1.2 Example Batch Script
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=ANSYS_JOB ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=ansys-job-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Request number of CPUs #SBATCH --ntasks=8 ### Request all CPUs on one node #SBATCH --nodes=1 ### Change to the work directory cd $HOME/jobdirectory # Comment out the module switch command and use the following line instead if you want to use Ansys < 20.0: # module unload intelmpi module switch intelmpi intelmpi/2021 module load TECHNICS ansys ### Execute your application ### For different ansys versions, change the executable name accordingly, ### e.g. ansys195 for Ansys 19.5 ### -b : Run the job in ANSYS batch mode. ### -i : ANSYS input file. ### -o : ANSYS output file: ### Notice: Remove '-mpi ibmmpi' for Ansys20+ and make sure intelmpi is loaded ansys201 -b -o ansys.out -i ansys.in -mpi ibmmpi |
Please make sure you specify #SBATCH --ntasks=<#> in your batch script, otherwise $SLURM_NTASKS will not be set and be unavailable during execution! This will produce an error!
1.3 Licensing
You may use Ansys using either one of two different licenses: The Ansys Academic Teaching license and the Ansys Academic Research license. The former one may be used for free by students (bachelor/master) for educational purposes and will automatically be available when you load the software. The Academic Research license may be purchased by RWTH institutes for research purposes. Users have to be granted access to use one of these licenses by the owning institute. Please contact your local IT administrator for this.
Please be aware that the teaching license is restricted in its features. For further details, please take a look at the feature set. Most importantly, the teaching licenses are restricted to 4 cores, since they do not provide the aa_r_hpc feature.
Per default, only the license paths to the research licenses (need to be paid) are set. If you wish to use the teaching licenses, please set the following environment variables AFTER loading the ansys module:
| export ANSYSLI_SERVERS=50198@license3.rz.rwth-aachen.de:50198@license5.rz.rwth-aachen.de export ANSYSLMD_LICENSE_FILE=50199@license.itc.rwth-aachen.de |
One needs to reset the license preferences for each version one wishes to use. Please use
/rwthfs/rz/SW/ANSYS/shared_files/licensing/lic_admin/anslic_admin
and then
Set License Preferences for User -> Choose Version -> Reset to Default
1.3.1 Changing License Preferences in Ansys
You may change the license preferences within the Ansys workbench according to your needs. This is especially important when choosing a Teaching license over an available Research license since the desired license needs to have a higher priority. To do so, start the workbench and navigate to Tools -> License Preferences... in the context menu:
Within the upcoming menu, use the buttons "Move up" and "Move down" to reorder the priority. The priority decreases from top to bottom. Proceed on all necessary tabs within the preference window.
1.4 Frequently Asked Questions
$ module load TECHNICS ansys $ runwb2 |
2. CFX
2.1 How to Access the Software
module load TECHNICS module load ansys |
2.2 Example Batch Scripts
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=CFX_SERIAL ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=cfx-serial-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Change to the work directory, if not here cd $HOME/jobdirectory ### load modules and execute module load TECHNICS module load ansys # start non-interactive batch job cfx5solve -def job.def |
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=CFX_MPI ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=cfx-mpi-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Request number of hosts #SBATCH --nodes=1 ### Request number of CPUs #SBATCH --ntasks=8 ### Change to the work directory cd $HOME/jobdirectory ### load modules and execute module load TECHNICS module load ansys ### start non-interactive batch job cfx5solve -def job.def -par-dist "$R_WLM_CFXHOSTLIST" -start-method "Intel MPI Distributed Parallel" |
2.3 Additional Information
Depending on the requirements of your simulation, the default single precision of CFX's solver might not yield proper results. The solver and its components can be set to double precision by adding the -double argument to cfx5solve.
3. Fluent
3.1 How to Access the Software
module load TECHNICS module load ansys |
3.2 Example Batch Scripts
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=FLUENT_SERIAL ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=fluent-serial-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Change to the work directory cd $HOME/jobdirectory ### load modules and execute module load TECHNICS module load ansys # start non-interactive batch job fluent -g -platform=intel -i journalfile |
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=FLUENT_OMP ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=fluent-omp-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need in MB #SBATCH --mem-per-cpu=1024 ### Request number of CPUs=Threads #SBATCH --ntasks=8 ### Change to the work directory cd $HOME/jobdirectory ### Load modules module load TECHNICS module load ansys # start non-interactive batch job fluent 3d -g -pshmem -t$SLURM_NTASKS -platform=intel -i journalfile |
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=FLUENT_MPI ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output fluent-MPI-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Request number of CPUs #SBATCH --ntasks=8 ### Change to the work directory cd $HOME/jobdirectory ### Load modules module purge module load TECHNICS module load ansys # Set environment variables export FLUENT_AFFINITY=0 export SLURM_ENABLED=1 ### start non-interactive batch job fluent 3ddp -g $R_WLM_FLUENT_PARAMETERS -i journalfile |
3.3 Interactive usage
module load TECHNICS ansys runwb2 |
In the Ansys Workbench:
- Select the "Component Toolbox"
- Add the "Fluent" Toolbox
- Run "Setup"
- Select the "Processing Option" "Parallel"
(+ "Show more options")
(Tab "Parallel Settings") - Choose "infiniband" as "Interconnects"
- Choose "openmpi" as "MPI types"
4. ICEM
4.1 How to Access the Software
module load TECHNICS module load icem |
4.2 Example Batch Script
#!/usr/local_rwth/bin/zsh ### Job name #SBATCH --job-name=ICEM_SERIAL ### File / path where STDOUT will be written, the %J is the job id #SBATCH --output=icem-serial-log.%J ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 ### Request memory you need for your job in MB #SBATCH --mem-per-cpu=3900 ### Export an environment var - do you need that? export A_ENV_VAR=10 ### Change to the work directory cd $HOME/workdirectory ### load modules and execute module load TECHNICS module load ansys # start non-interactive batch job icemcfd -batch -script icem.script # You need a file containing information for net generation, e. g. # an icem.skript containing the following line # # ic_run_tetra "Job.tin" "tetra_mesh.uns" |
5. Known limitations
5.1 MPI vendors
Ansys is delivered with support for different MPI vendors: IBM-MPI (previously known as HP-MPI), Open MPI, Intel MPI, each in varying versions, sometimes more than one, but mostly older versions. As we have an Intel OmniPath fabric in our clusters we focus on Intel MPI. We do not support IBMMPI and OpenMPI (but do not rule out the possibility that these could work on some circumstandes)
As some older versions of Intel MPI used in older versions of Ansys do need additinal tuning to run (with good performance) on OPA fabric, we moved these older versions of Ansys to DEPRECATED area even if we know how-to tweak these; please use actual versions of Ansys
In terms of performance, a single-node job is (almost) always faster than the same amount of ranks spread across multiple nodes due to the fact that a fabric is always slower than intra-node communication. Try to stay on a single node whenever possible.
