Hybrid Slurm Job

Work in progress ....

Short introduction: Sample Page is a page that shows how to layout a Wiki-Page. In the introduction you should describe what this is and what it is used for.

ProPE Logo

Basic usage

$ cd ..

tut a, b, c

$ ls -l

tut d, e und f

Tips and Tricks

Get the source code of this sample Page and copy it into new pages to start with a resonable structure.

Common Pitfalls

blabla ... Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.

Links and more Information

For some information how to do some things like LaTeX, Code Highlight or pictures, check the Wiki Syntax

This hybrid MPI+OpenMP job will start the parallel program "hello.exe" with 4 MPI processes and 3 OpenMP threads each on 2 compute nodes.

#!/bin/zsh

### Job name
#SBATCH --job-name=HelloHybrid

### 2 compute nodes
#SBATCH --nodes=2

### 4 MPI ranks
#SBATCH --ntasks=4

### 2 MPI ranks per node
#SBATCH --ntasks-per-node=2

### 3 tasks per MPI rank
#SBATCH --cpus-per-task=3

### the number of OpenMP threads 
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

### Change to working directory
cd /home/usr/workingdirectory

### Run your parallel application
srun hello.exe

You can use this hybrid toy Fortran90 program to test the above job script

program hello
   use mpi
   use omp_lib

   integer rank, size, ierror, tag, status(MPI_STATUS_SIZE),threadid
   character*(MPI_MAX_PROCESSOR_NAME) name
   
   call MPI_INIT(ierror)
   call MPI_COMM_SIZE(MPI_COMM_WORLD, size, ierror)
   call MPI_COMM_RANK(MPI_COMM_WORLD, rank, ierror)
   call MPI_GET_PROCESSOR_NAME(name,len,ierror)
   
!$omp parallel private(threadid)
   threadid=omp_get_thread_num()
   print*, 'node: ', trim(name), '  rank:', rank, ', thread_id:', threadid
!$omp end parallel
   
   call MPI_FINALIZE(ierror)
   
end program

When sorting the program output it may look like

 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           0 , thread_id:           0
 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           0 , thread_id:           1
 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           0 , thread_id:           2
 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           1 , thread_id:           0
 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           1 , thread_id:           1
 node: ncm1018.hpc.itc.rwth-aachen.de  rank:           1 , thread_id:           2
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           2 , thread_id:           0
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           2 , thread_id:           1
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           2 , thread_id:           2
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           3 , thread_id:           0
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           3 , thread_id:           1
 node: ncm1019.hpc.itc.rwth-aachen.de  rank:           3 , thread_id:           2