Sample Scripts

Sample batch job submission scripts on zcluster

To run a serial job

#!/bin/bash
cd working_directory
time ./myprog < myin > myout

Note that the 'time' command included in the sample script above is optional (this command measures the amount of time it takes to run an executable). Entering a standard input file (myin) and standard output file (myout) is only necessary if your executable requires input from standard input and if it outputs data to standard output.

You can specify the working_directory as in the example below:

#!/bin/bash
cd ${HOME}/projectA
time ./myprog < myin > myout

If the job is submitted from within the working directory, you can use the following sample script:

#!/bin/bash
cd `pwd`
time ./myprog < myin > myout

To run R

Sample script to run an R program called program.R:

#!/bin/bash
cd `pwd`
time /usr/local/R/2.15.2/bin/R CMD BATCH program.R

Parallel Jobs using MPI libraries

Several MPI libraries are installed on zcluster. Here are sample scripts to use some version of MPI llibraries. For other versions, you need to change the path to mpirun appropriately. For a list of all MPI libraries installed on zcluster, please see Code Compilation on zcluster.

In the examples below, the executable name will be called myprog. The second line in the scripts below, namely cd working_directory or cd ${HOME}/subdirectory can be replaced by cd `pwd` if the job will be submitted from within the working directory.

NOTE: MPICH jobs executed with mpirun have to use the -machinefile option as shown in the example above, otherwise your mpi job will not use the processors assigned to it by the queueing system. In contract, when using MPICH2, OpenMPI and MVAPICH2 it is not necessary to use the -machinefile option.

To run a parallel MPI job using MPICH/PGI (e.g. script name submpi.sh):

#!/bin/bash
cd working_directory
echo "Got $NSLOTS processors."
echo "Machines:"
cat $TMPDIR/machines
/usr/local/pgi/linux86-64/2012/mpi/mpich/bin/mpirun -np $NSLOTS -machinefile $TMPDIR/machines ./myprog

Note that lines 3, 4, and 5 in this script are optional.

To run a parallel MPI job using the default MPICH2 (PGI) compilers. For example, if you compiled the code with mpicc or mpif90, etc, without full path (e.g. script name submpich2.sh)

#!/bin/bash
cd working_directory
export LD_LIBRARY_PATH=/usr/local/mpich2/1.4.1p1/pgi123/lib:${LD_LIBRARY_PATH}
mpirun -np $NSLOTS ./myprog

To run a parallel MPI job using MPICH2 and e.g. GNU 4.4.4 compilers (e.g. script name submpich2.sh)

#!/bin/bash
cd working_directory
export LD_LIBRARY_PATH=/usr/local/mpich2/1.4.1p1/gcc_4.4.4/lib:${LD_LIBRARY_PATH}
/usr/local/mpich2/1.4.1p1/gcc_4.4.4/bin/mpirun -np $NSLOTS ./myprog

Note that with MPICH2 it is not necessary to include the -machinefile option when submitting the job to a batch queue. When using other MPICH2 compilations, such as for PGI compilers, users need to adjust the path to the libraries and to mpirun appropriately in the script.

To run a parallel MPI job using OpenMPI 1.4.4 and e.g. GNU 4.1.2 compilers (e.g. script name subompi.sh)

#!/bin/bash
cd working_directory
export LD_LIBRARY_PATH=/usr/local/openmpi/1.4.4/gcc412/lib:${LD_LIBRARY_PATH}
/usr/local/openmpi/1.4.4/gcc412/bin/mpirun ––mca btl "tcp,self" -np $NSLOTS ./myprog

To run a parallel MPI job using OpenMPI 1.6.2 and e.g. GNU 4.7.1 compilers (e.g. script name subompi.sh)

#!/bin/bash
cd working_directory
export LD_LIBRARY_PATH=/usr/local/gcc/4.7.1/lib64:/usr/local/openmpi/1.6.2/gcc471/lib:${LD_LIBRARY_PATH}
/usr/local/openmpi/1.6.2/gcc471/bin/mpirun ––mca btl "tcp,self" -np $NSLOTS ./myprog

Note that with OpenMPI you can use the mpirun command and there is no need to include the -machinefile option. When using other OpenMPI compilations, such as the one for PGI compilers, users need to adjust the path to the libraries and to mpirun appropriately in the script. To use OpenMPI over Infiniband, remove the mpirun option

––mca btl "tcp,self"

from the script.

To run a parallel MPI job using MVAPICH2/GNU over Infiniband (e.g. script name submvapich2.sh)

#!/bin/bash
cd working_directory
export LD_LIBRARY_PATH=/usr/local/mvapich2/1.8/gcc444/lib:${LD_LIBRARY_PATH}
/usr/local/mvapich2/1.8/gcc444/bin/mpirun -np $NSLOTS ./myprog