Increasing the Core Count of Parallel Steps

The following section will walk through how to run the parallel steps of the OpenFOAM motorbike tutorial workflow on more cores by modifying Fuzzfile openfoam-motorbike-mpi.fz used in the previous section which ran the parallel steps on 6 cores on a single node. In this section, we will modify the Fuzzfile to run the parallel steps using 8 cores and 2 nodes for a total of 16 cores.

First, in the job named prepare-motorbike-case, you will need to update the input file motorBike/system/decomposeParDict to decompose the model into 16 subdomains and update the coefficients in the file to 3 integers which are factors of 16 (ex. 2, 2, and 4). You can update the contents of this file using the sed command. Below is an example job command for the job prepare-motorbike-case which copies the motorbike case file into the working directory, updates parameter numberOfSubdomains in motorBike/system/decomposeParDict to 16, and updates the coefficients parameter to (2 2 4) in the same file.

command: ["/bin/bash", "-c", "cp -r $WM_PROJECT_DIR/tutorials/incompressible/simpleFoam/motorBike .; \
            cp motorBike/system/decomposeParDict.6 motorBike/system/decomposeParDict; \
            sed -i 's/numberOfSubdomains\\ 6/numberOfSubdomains\\ 16/g' motorBike/system/decomposeParDict; \
            sed -i 's/\\(3 2 1\\)/2 2 4/g' motorBike/system/decomposeParDict; \
            cat motorBike/system/decomposeParDict;"]

Next, you can update the resource requests of parallel jobs snappy-hex-mesh,toposet, patch-summary, potential-foam, check-mesh, and simple-foam to use 16 cores (2 nodes, 8 cores per node). First, request 2 node by updating nodes in the multinode block to 2. Each job mentioned should have a multinode block as follows:

multinode:
  nodes: 2
  implementation: openmpi

Finally, update the resource block of the jobs to use 8 cores. Each job mentioned should have a resource block as follows:

resource:
  cpu:
    cores: 8
    affinity: NUMA
  memory:
    size: 4GiB

If you are using the Fuzzfile in the previous section, all lines which contain the comment # Update nodes such that nodes * cores = numSubDomains if you are editing file decomposeParDict to run on more cores will need to be updated.