From Research Computing Center Wiki
Jump to: navigation, search

Compiling OpenMP code on Sapelo2

The compilers installed on Sapelo2 support shared memory applications using OpenMP. Here are the compiler options to include when using OpenMP:

Compiler Commands Option
PGI pgcc, pgCC, pgf90, etc. -mp
Intel icc, ifort -openmp
GNU gcc, gfortran, g++, gcc44, etc. -fopenmp

For more information about the compilers on Sapelo2, please see Code Compilation on Sapelo2.

Defining number of OpenMP threads

The number of OpenMP threads can be specified using the environment variable OMP_NUM_THREADS. For example, to define 4 threads:

For bash/sh:


For csh/tcsh:


Setting thread-core binding

In general, an OpenMP code will run a lot more efficiently when threads are pinned to a given core on a compute node. Please set the environment variable OMP_PROC_BIND=true to specify threads' affinity policy (thread-core bind).

For bash/sh:

export OMP_PROC_BIND=true

For csh/tcsh:

setenv OMP_PROC_BIND  true

Sample script to run a shared memory job using OpenMP

Sample job submission script ( to run a program that uses 4 OpenMP threads:

#PBS -S /bin/bash
#PBS -q batch
#PBS -N testjob
#PBS -l nodes=1:ppn=4
#PBS -l walltime=48:00:00
#PBS -l mem=30gb
#PBS -M 
#PBS -m abe


export OMP_PROC_BIND=true

echo "Job ID: $PBS_JOBID"
echo "Queue:  $PBS_QUEUE"
echo "Cores:  $PBS_NP"
echo "Nodes:  $(cat $PBS_NODEFILE | sort -u | tr '\n' ' ')"

time ./a.out > outputfile

Please note that the number of cores requested by this job (ppn=4) is the same as the number of OpenMP threads that the application will use (export OMP_NUM_THREADS=4).

Sample job submission command on Sapelo2

To submit a shared memory job that uses 4 OpenMP threads:


For more information about running jobs on Sapelo2, please see Running Jobs on Sapelo2.