Slurm提交OpenMPI作業

OpenMPI（open Message Passing Interface），OpenMPI是MPI的一種實現，是信息傳遞接口庫項目。

1、安裝OpenMPI

# wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.4.tar.bz2
# tar jxvf openmpi-4.0.4.tar.bz2 # cd openmpi-4.0.4/ # ./configure --prefix=/usr/local/openmpi # make # make install

 

2、添加環境變量

# export PATH="/usr/local/openmpi/bin:$PATH" # export LD_LIBRARY_PATH="/usr/local/openmpi/lib/:$LD_LIBRARY_PATH"

寫環境變量文件

# vim /etc/profile
export PATH="/usr/local/openmpi/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/openmpi/lib/:$LD_LIBRARY_PATH"

 

或者使用Moudle管理環境變量

# cat /etc/modulefiles/mpi/openmpi-4.04 
#%Module 1.0
##
## OpenMPI module for use with 'environment-modules' package:
module-whatis "sets up access to openmpi-4.04"
#prereq openmpi-4.04
conflict mpi
setenv SOMEVERION 4.04
append-path PATH /usr/local/openmpi/bin
append-path LD_LIBRARY_PATH /usr/local/openmpi/lib

 

3、測試mpirun

# cd openmpi-4.0.4/examples # mpicc hello_c.c -o hello # mpirun --allow-run-as-root -np 2 hello
Hello, world, I am 0 of 2, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 1 of 2, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)

多節點運行
多節點盡量使用NFS共享存儲，不然所有節點都需要安裝OpenMPI；

# for i in `seq 1 3` ; do echo "node$i" >> hostlist ; done # /usr/local/openmpi/bin/mpirun --allow-run-as-root -np 6 --hostfile hostlist /openmpi-4.0.4/examples/hello
Hello, world, I am 0 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 1 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 4 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 5 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 2 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)
Hello, world, I am 3 of 6, (Open MPI v4.0.4, package: Open MPI root@m1 Distribution, ident: 4.0.4, repo rev: v4.0.4, Jun 10, 2020, 102)

 

4、Slurm提交mpi任務

Slurm部署：https://www.cnblogs.com/liu-shaobo/p/13285839.html

# cat hello.sh
內容如下：
#!/bin/bash
#SBATCH --output=/tmp/job.%j.out
#SBATCH --error=/tmp/job.%j.err
#SBATCH --nodes=3                    ##使用節點數量
#SBATCH --ntasks-per-node=2          ##每個節點的進程數
mpirun --allow-run-as-root -np $SLURM_NPROCS ./openmpi-4.0.4/examples/hello

提交mpi任務

# sbatch hello.sh