Integrating Open MPI With CLion on Apple M1

A short tutorial on how to integrate Open MPI 4.1.2 with CLion 2021.3.2 running on Apple M1

Photo by Carles Rabada on Unsplash

TL;DR

1. Setup compilation databases
2. Replace the default “command” with the output of mpicc -showme
3. Setup custom build targets by setting the “Program” as the location of clang, the “Arguments” as the arguments from mpicc -showme, the location of the file that will be compiled, and the location of the executable and the “Working directory” as the location of clang.
4. Setup custom run/debug configuration by setting the “Executable” as mpirun and the “Program Arguments” as the number of processors and the location of the executable (the output of the compiler)

Going parallel

Suppose that someday you get bored with sequential programming and decide to try something more parallel.

You don’t have any idea about libraries or frameworks for parallel programming but you have a nice experience with C programming. Doing some googling you reach pretty quickly at MPI (Message Passing Interface).

Long story short, MPI it’s a standard for how to connect things in parallel architectures.

There are two main implementations of this standard: Open MPI and MPICH. Because I just wanted to play, I randomly chose Open MPI .

Since I had already installed CLion 2021.3.2 on my MacBook M1 (macOS Monterey 12.0), I decided to integrate Open MPI with CLion. I have installed Open MPI 4.1.2. using brew following the instructions from here.

The Wrong Way

At first sight, nothing could go wrong. Googling for some tutorials on how to bring together Open MPI and CLion I found two nice resources.

Both tutorials indicate that in order to use Open MPI with CLion all you have to do is to include find_package(MPI REQUIRED) inside the CMakeLists.txt file. The problem:

My first thought was to specify the path where Open MPI was installed so I use find_package(MPI REQUIRED PATHS /opt/homebrew/Cellar/open-mpi/4.1.2). Unfortunately, this didn’t work either:

Doing some extra research I found that the problem is related to the version of CMake.

Newer versions of CMake like 3.11 have this problem while older versions like 3.5 did not. You can read more about this here.

The Solution

Photo by AbsolutVision on Unsplash

First of all, here is my code sample which I want to compile and run inside CLion:

It is a hello world application for Open MPI given in this very nice tutorial. To compile the code we use mpicc:

mpicc main.c -o main.o

To run the executable we use mpirun. The argument -np 4 indicates the number of processors on which the above code will run in parallel:

mpirun -np 4 main.o

Both commands (mpicc and mpirun) became available when we install the Open MPI using brew.

All I wanted to do in CLion was to link the build button with the mpicc command and the run button with the mpirun command. The solution was to use two cool features of CLion: compilation databases and custom build targets.

Once you enable the compilation databases you will get a JSON file named compiled_commands.json. In this JSON you will find an element called command. That is the command that will be executed when compiling main.c. The first idea was to replace the default command by the mpicc:

The problem is that CLion doesn’t recognize mpicc as a compiler by default so it will display the following error:

Cannot determine compiler type by executable file: ‘/opt/homebrew/bin/mpicc

Doing another extra research I found that mpicc is actually a wrapper over the system compiler. In my case, mpicc will use the clang compiler. To find what is behind mpicc you can execute the following command:

mpicc -showme

Here is my output:

clang - I/opt/homebrew/Cellar/open-mpi/4.1.2/include -L/opt/homebrew/Cellar/open-mpi/4.1.2/lib - L/opt/homebrew/libevent/lib -lmpi

When specifying the “command” from the compiled_commands.json file you need to replace mpicc with the output of mpicc -showme. Here is my “command”:

To compile main.c use the Recompile ‘main.c’ button from the Build section:

Although the compilation succeeds, an executable is not produced because CLion automatically adds the flag -fsyntax-only to the compilation command specified in the JSON file.

In order to generate an executable, you need to create a custom build target. Following the instruction from CLion we get to the following form that we have to fill in:

The “Program” describes the program that will be run when we press the build button. In our case we want to run the clang compiler so the “Program” must be:

/usr/bin/clang

The “Arguments” refers to the arguments passed to the “Program”. In our case, the “Arguments” are composed of the ones of the clang compiler shown by mpicc -showme command, the file that will be compiled, and the output file. For me, the “Arguments” are:

The working directory is the location of the “Program” so in our case:

/usr/bin

With these settings, when we press the build button an executable named main.o will be generated. To run this executable we must set up a custom run/debug configuration. What we basically need to do is to fill the following form:

The “Executable” refers to the program which will be run when we press the run button. In our case, this is the mpirun:

/opt/homebrew/bin/mpirun

The “Program arguments” refers to the arguments of the “Executable”. In our case, we have two arguments: the number of processors and the location of the executable (the output of the mpicc):

-np 4 /Users/mihailplesa/Documents/Doctorat/Experiments/MPI2/main.o

Now, when we press the run button we can see the output of the sample code from above:

/opt/homebrew/bin/mpirun -np 4 /Users/mihailplesa/Documents/Doctorat/Experiments/MPI2/main.o
Hello world! I'm process 2 out of 4 processes
Hello world! I'm process 0 out of 4 processes
Hello world! I'm process 3 out of 4 processes
Hello world! I'm process 1 out of 4 processes
Process finished with exit code 0

Final Remarks

Although this is the solution to the problem of integrating Open MPI with CLion on Apple M1, I have a feeling that this isn’t the most elegant solution. If you know other ways please let me know in the comments.

Photo by Kenny Timmer on Unsplash

Resources

• https://www.jetbrains.com/help/clion/compilation-database.html
• https://www.jetbrains.com/help/clion/custom-build-targets.html
• https://www.open-mpi.org
• https://www.mpich.org
• https://en.wikipedia.org/wiki/Message_Passing_Interface
• https://formulae.brew.sh/formula/open-mpi
• https://www.onooks.com/how-to-import-openmp-and-mpi-to-a-large-clion-cmake-project/
• https://intellij-support.jetbrains.com/hc/en-us/community/posts/115000001110-Configure-CMakeList-properly-to-work-with-OpenMPI-c-
• https://cmake.org/cmake/help/latest/command/find_package.html
• https://stackoverflow.com/questions/49816206/cmake-find-package-specify-path
• https://github.com/Intel-HLS/GenomicsDB/issues/187
• http://condor.cc.ku.edu/~grobe/docs/intro-MPI-C.shtml