Sie befinden sich im Service: RWTH Compute Cluster Linux (HPC)

openfoam

openfoam

Detailinformation

1. General Information

OpenFOAM is a free, open source CFD software package released by multiple competitive groups:

In the HPC Cluster, OpenFOAM is usually build in multiple versions (using Intel and GCC compilers, Intel and Open MPI). The version is chosen based on the loaded compiler and MPI modules (by default Intel compiler + Open MPI). Check the available versions (typically incompatible to each other!) by

$ module load TECHNICS
$ module avail openfoam
 

2. How to Access the Software

$ module load TECHNICS
$ module load openfoam
 

3. User Libraries and Extensions

There are many extensions and libraries for OpenFOAM like cfMesh, as well as any user is able to write own applications. These are to be installed into the user's $HOME directory, typically into $WM_PROJECT_USER_DIR. Typical installation of such an extension consists of

  • downloading the tarbal, extract it into some directory, 'cd' into it
  • load the OpenFOAM module you like to use
  • call 'Allwclean' and 'Allwmake' or somewhat alike. 

Read the manual!

Note that in order to use the extension you will need the same OpenFOAM module to be loaded; changing the version of OpenFOAM used lead to need to recompile the extension/library/application.

 

4. Example Batch Scripts

Serial Job

#!/usr/local_rwth/bin/zsh
  
### Job name
#SBATCH --job-name=FOAM_SERIAL
  
### Request the time you need for execution. The full format is D-HH:MM:SS
### You must at least specify minutes or days and hours and may add or
### leave out any other parameters
#SBATCH --time=80
  
#### Request the memory you need for your job. You can specify this
### in either MB (1024M) or GB (4G).
#SBATCH --mem-per-cpu=1850M
  
### load modules
module load TECHNICS
module load openfoam
  
### Create a project directory, Copy the tutorial examples
mkdir -p $FOAM_RUN
cp -Lr $FOAM_TUTORIALS $FOAM_RUN
cd $FOAM_RUN/tutorials/incompressible/icoFoam/cavity
  
  
# start non-interactive batch job
blockMesh
icoFoam
  
# (OFF) after the batch job has finished
#paraFoam

MPI Parallel Job

#!/usr/local_rwth/bin/zsh
  
### Job name
#SBATCH --job-name=FOAM_SERIAL
  
### Request the time you need for execution. The full format is D-HH:MM:SS
### You must at least specify minutes or days and hours and may add or
### leave out any other parameters
#SBATCH --time=80
  
#### Request the memory you need for your job. You can specify this
### in either MB (1024M) or GB (4G).
#SBATCH --mem-per-cpu=1850M
 
### Request 12 processes, all on a single node
#SBATCH --nodes=1
#SBATCH --ntasks=12
 
### Load the required module files
module load TECHNICS
module load openfoam
  
### start the OpenFOAM binary in parallel, cf.
$MPIEXEC $FLAGS_MPI_BATCH dieselFoam -parallel
 

5. Further Information / Known issues

  • Note: Different from most modules, the 'openfoam' module is not unloadable.
    You also cannot switch to other version of OpenFOAM in the same shell; use different shells / batch jobs for different versions of OpenFOAM.
  • Known issue on parallel start: As described in http://www.openfoam.org/docs/user/running-applications-parallel.php  you have to start you application in parallel by using $MPIEXEC and add the '-parallel' flag. However, if you  get this error (known for multiphaseInterFoam):

    --> FOAM FATAL ERROR in Foam::findEtcFiles() : could not find mandatory file
        'controlDict'

    ... try to use 'foamExec' wrapper as workaround (see https://www.cfd-online.com/Forums/openfoam-solving/69561-foam-fatal-error-foam-findetcfile-could-not-find-mandatory-file.html#post322759):

    $MPIEXEC $FLAGS_MPI_BATCH foamExec multiphaseInterFoam -parallel

    In case this still does not work define a batch job which will definitely run on a single node.

  • Known issues on openfoam/3.0.1 (see http://bugs.openfoam.org/view.php?id=2067)
    • 'decomposePar' and 'snappyHexMesh' - fixed by rebuild with the very default optimisation flags.
    • 'fireFoam' fails with message Floating point exception(core dumped) on 'tutorials/combustion/fireFoam/les/flameSpreadWaterSuppressionPanel' - currently no workaround; however future versions should be fixed, already (see above bug report)
  • Known issues on openfoam/4.*
    • by loading the module the following message is printed:

      /usr/local_rwth/sw/openfoam/4.1/intel_16.0.2.181-openmpi_1.10.4/OpenFOAM-4.1/etc/config.sh/aliases:73: bad option: -t

      This message is results from incompatibility of ZSH (our default shell) and BASH. There are no limitations in using OpenFOAM known connected to this message; if in doubt try to use the BASH shell.

  • SIGFPE, coredump, exit 136 and friends

    Your calculation is aborted with messages like this:

    #0  Foam::error::printStack(Foam::Ostream&) at ??:?
    #1  Foam::sigFpe::sigHandler(int) at ??:?
    #2  ? in "/lib64/libc.so.6"
    #3  void Foam::fv::rotorDiskSource::calculate<Foam::geometricOneField>(Foam::geometricOneField const&, Foam::Field<Foam::Vector<double> > const&, Foam::Field<double> const&, Foam::Field<Foam::Vector<double> >&, bool, bool) const at ??:?
    #4  Foam::fv::rotorDiskSource::addSup(Foam::fvMatrix<Foam::Vector<double> >&, int) at ??:?
    #5  ? at ??:?
    #6  ? at ??:?
    #7  __libc_start_main in "/lib64/libc.so.6"
    #8  ? at ??:?

    (in LSF you get an exit code 136; you alco may find a file named "core.login18-1.hpc.itc.rwth-aachen.de.271599.11" in your directory). What does that mean?

  • paraFoam ParaView issues

    OpenFOAM is huge. It also brings a boxed version of ParaView, which itself is not that easy to be build. Not all versions of OpenFOAM contain a working version of ParaView. However we offer a standalone installation of ParaView.
    • Known issues in SLURM/CLAIX18 environment:
    • openfoam/5.0: paraFoam dies with " FATAL ERROR: ParaView reader module libraries do not exist". No workaround known; try to use alternative methods to visualize your data sets after loading ParaView in a separete session, see https://www.openfoam.com/documentation/user-guide/paraview.php
    • openfoam/6: no boxed ParaView; need to load one from modules.  After that 'paraFoam -builtin' dies with '.... paraFoam: line 63: 168444 Segmentation fault      (core dumped) paraview $_opt "$@"'
      • workaround:

        export ParaView_GL=system
    • openfoam/esi-v2006: no boxed ParaView; need to load version paraview/5.6.0 from modules.

  • swak4foam

    The swak4Foam is a library many users of OpenFOAM also want to use. This is a very dusty software; in fact we have seem many issues with it. As the last supporter version of OpenFOAM is v2.4 (!!!) we give no warranty it works on our cluster. However we know that using the below commands you are able to compile  swak4foam for OpenFOAM version 6:

    $ module switch intel gcc
    $ module load TECHNICS
    $ module load openfoam
    $ hg clone http://hg.code.sf.net/p/openfoam-extend/swak4Foam
    $ cd swak4Foam
    $ time  ./Allwmake 2>&1 | tee 20190613.02.gcc.txt
    $ ls -la $FOAM_USER_APPBIN $FOAM_USER_LIBBIN

    Note that the GCC-compiled version of OpenFOAM has been used (due to change to GCC form Intel compiler *PRIOR*  loading OpenFOAM).

    The Intel compiler cannot compile swak4foam.

     

Zusatzinformation

Jülich - Aachen OpenFOAM User Group

In 2018 an inofficial Jülich - Aachen OpenFOAM User Group was founded. If you are interested in this user group, please subscribe yourself to the wiki or the mailing list.

zuletzt geändert am 29.01.2021

Wie hat Ihnen dieser Inhalt geholfen?