OpenFOAM is a free, open source CFD software package. OpenFOAM has an extensive range of features to solve anything from complex fluid flows involving chemical reactions, turbulence and heat transfer, to solid dynamics and electromagnetics.
Table of Contents
- How to access the software
- User libraries and extensions
- Example batch scripts
- Further information / Known issues
Depending on the version, you might have to load additional modules until you can load OpenFOAM:
module load GCC/10.3.0 module load OpenMPI/4.1.1 module load OpenFOAM/9
Available OpenFOAM versions can be listed with
module spider OpenFOAM. Specifying a version will list the needed modules:
module spider OpenFOAM/9
There are many extensions and libraries for OpenFOAM like cfMesh and any user is able to write their own applications. These are to be installed into the user's
$HOME directory, typically into
$WM_PROJECT_USER_DIR. Typical installation of such an extension consists of:
- downloading the tarball, extract it into some directory,
- load the OpenFOAM module you like to use
Allwmakeor somewhat alike.
Note that in order to use the extension you will need the same OpenFOAM module to be loaded; changing the version of OpenFOAM means you need to recompile the extension/library/application.
#!/usr/bin/zsh ### Job name #SBATCH --job-name=FOAM_SERIAL ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 #### Request the memory you need for your job. You can specify this ### in either MB (1024M) or GB (4G). #SBATCH --mem-per-cpu=3900M ### load modules module load GCC/10.3.0 module load OpenMPI/4.1.1 module load OpenFOAM/9 ### Create a project directory, Copy the tutorial examples mkdir -p $FOAM_RUN cp -Lr "$FOAM_TUTORIALS" "$FOAM_RUN" cd "$FOAM_RUN/tutorials/incompressible/icoFoam/cavity/cavity" # start non-interactive batch job blockMesh icoFoam # (OFF) after the batch job has finished #paraFoam
MPI Parallel Job:
#!/usr/bin/zsh ### Job name #SBATCH --job-name=FOAM_SERIAL ### Request the time you need for execution. The full format is D-HH:MM:SS ### You must at least specify minutes or days and hours and may add or ### leave out any other parameters #SBATCH --time=80 #### Request the memory you need for your job. You can specify this ### in either MB (1024M) or GB (4G). #SBATCH --mem-per-cpu=3900M ### Request 12 processes, all on a single node #SBATCH --nodes=1 #SBATCH --ntasks=12 ### Load the required module files module load GCC/10.3.0 module load OpenMPI/4.1.1 module load OpenFOAM/9 ### start the OpenFOAM binary in parallel, cf. $MPIEXEC $FLAGS_MPI_BATCH dieselFoam -parallel
Known issue on parallel start: As described in Running applications in parallel you have to start you application in parallel by using
$MPIEXECand add the
-parallelflag. However, if you get this error (known for multiphaseInterFoam):
--> FOAM FATAL ERROR in Foam::findEtcFiles() : could not find mandatory file 'controlDict'
try to use
foamExecwrapper as workaround (see forum entry):
$MPIEXEC $FLAGS_MPI_BATCH foamExec multiphaseInterFoam -parallel
In case this still does not work define a batch job which will definitely run on a single node.
SIGFPE, coredump, exit 136 and friends
Your calculation is aborted with messages like this:
#0 Foam::error::printStack(Foam::Ostream&) at ??:? #1 Foam::sigFpe::sigHandler(int) at ??:? #2 ? in "/lib64/libc.so.6" #3 void Foam::fv::rotorDiskSource::calculate<Foam::geometricOneField>(Foam::geometricOneField const&, Foam::Field<Foam::Vector<double> > const&, Foam::Field<double> const&, Foam::Field<Foam::Vector<double> >&, bool, bool) const at ??:? #4 Foam::fv::rotorDiskSource::addSup(Foam::fvMatrix<Foam::Vector<double> >&, int) at ??:? #5 ? at ??:? #6 ? at ??:? #7 __libc_start_main in "/lib64/libc.so.6" #8 ? at ??:?
(in Slurm you get an exit code 136; you also may find a file named "core.login18-1.hpc.itc.rwth-aachen.de.271599.11" in your directory). What does that mean?
- The application did run into a Floating point exception which could be a division by 0 or an over/underflow, see
- The root of issue could be
- an error in the application (less likely but possible in OpenFOAM-provided binaries, more likely in self-written/-installed extensions, or
- some numerical issue (round-off error), or
- physically meaningless data set, or a data set with error(s), or
- something else (Learning never stops).
- Possible ways to go:
- check your data set
debug your application (if #3 line in stack dump points to your appilcation), see OpenFOAM wiki on HowTo debugging
try out a different version of OpenFOAM (newer is usually better, sometimes an old good one is good for you)
try out the same version compiled by different compiler, e.g. by GCC:
prior to loading the OpenFOAM module in a different shell/batch job!
- set/change FOAM_SIGFPE, FOAM_SETNAN envvars, see OpenFOAM API
- WARNING: YOU MAY GET PHYSICALLY MEANINGLESS RESULTS.
- WARNING: YOU MAY GET PHYSICALLY MEANINGLESS RESULTS.
paraFoam ParaView issuesOpenFOAM is huge. It also brings a boxed version of ParaView, which itself is not that easy to be build. Not all versions of OpenFOAM contain a working version of ParaView. However we offer a standalone installation of ParaView.
Jülich - Aachen OpenFOAM User Group
In 2018 an inofficial Jülich - Aachen OpenFOAM User Group was founded. If you are interested in this user group, please subscribe to the mailing list.