Running Jobs on the teaching cluster - Revision history

Shtsai at 02:49, 5 April 2026

2026-04-05T02:49:42Z

← Older revision		Revision as of 22:49, 4 April 2026
Line 17:		Line 17:
	The table below summarizes the partitions (queues) defined and the compute nodes that they target:		The table below summarizes the partitions (queues) defined and the compute nodes that they target:
	{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"		{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"
			\|+ Partitions defined on the teaching cluster
	\|-		\|-
	! scope="col" \| ~~Queue~~ Name		! scope="col" \| Partition Name
	! scope="col" \| Node Type		! scope="col" \| Node Type
	! scope="col" \| Node Number		! scope="col" \| Node Number
Line 26:		Line 27:

	\|-		\|-
	\| batch \|\| Intel\|\| 8 \|\| 32-core, 128GB RAM, AMD Naples \|\| For regular jobs.		! scope="row" \| batch
			\| Intel\|\| 8 \|\| 32-core, 128GB RAM, AMD Naples \|\| For regular jobs.
	\|-		\|-
	\| highmem \|\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.		! scope="row" \| highmem
			\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.
	\|-		\|-
	\| gpu \|\| GPU\|\| 1 \|\| 32-core, 190GB RAM, Intel Skylake, 1 P100 GPU \|\| For GPU-enabled jobs.		! scope="row" \| gpu
			\| GPU\|\| 1 \|\| 32-core, 190GB RAM, Intel Skylake, 1 P100 GPU \|\| For GPU-enabled jobs.
	\|-		\|-
	\| interactive \|\| Intel \|\| 2 \|\| 32-core, 128GB RAM, Intel Xeon \|\| For interactive jobs.		! scope="row" \| interactive
			\| Intel \|\| 2 \|\| 32-core, 128GB RAM, Intel Xeon \|\| For interactive jobs.
	\|-		\|-
	\| franklin_gpu \|\| GPU\|\| 2 \|\| 96-core, 1TB RAM, Intel Sapphire Rapids, 4 A30 GPU \|\| For Franklin College only.		! scope="row" \| franklin_gpu
			\| GPU\|\| 2 \|\| 96-core, 1TB RAM, Intel Sapphire Rapids, 4 A30 GPU \|\| For Franklin College only.
	\|-		\|-

Line 164:		Line 170:

	{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"		{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"
			\|+ Environment variables exported by batch jobs
	\|-		\|-
	! scope="col" \| Variable		! scope="col" \| Variable
Line 170:		Line 177:

	\|-		\|-
	\| SLURM_ARRAY_JOB_ID \|\| Job id of the array job		! scope="row" \| SLURM_ARRAY_JOB_ID
			\| Job id of the array job
	\|-		\|-
	\| SLURM_ARRAY_TASK_ID \|\| Array task index (element ID)		! scope="row" \| SLURM_ARRAY_TASK_ID
			\| Array task index (element ID)
	\|-		\|-
	\| SLURM_CPUS_ON_NODE \|\| Number of CPUS on the allocated node.		! scope="row" \| SLURM_CPUS_ON_NODE
			\| Number of CPUS on the allocated node.
	\|-		\|-
	\| SLURM_CPUS_PER_TASK \|\| Number of cpus requested per task. Only set if the --cpus-per-task option is specified.		! scope="row" \| SLURM_CPUS_PER_TASK
			\| Number of cpus requested per task. Only set if the --cpus-per-task option is specified.
	\|-		\|-
	\| SLURM_JOB_ID \|\| Full job ID (for array job: jobID_taskID)		! scope="row" \| SLURM_JOB_ID
			\| Full job ID (for array job: jobID_taskID)
	\|-		\|-
	\| SLURM_JOB_NAME \|\| User specified jobname		! scope="row" \| SLURM_JOB_NAME
			\| User specified jobname
	\|-		\|-
	\| SLURM_JOB_CPUS_PER_NODE \|\| Count of processors available to the job on this node.		! scope="row" \| SLURM_JOB_CPUS_PER_NODE
			\| Count of processors available to the job on this node.
	\|-		\|-
	\| SLURM_JOB_NAME \|\| Name of the job.		! scope="row" \| SLURM_JOB_NAME
			\| Name of the job.
	\|-		\|-
	\| SLURM_JOB_NODELIST \|\| List of nodes allocated to the job.		! scope="row" \| SLURM_JOB_NODELIST
			\| List of nodes allocated to the job.
	\|-		\|-
	\| SLURM_JOB_NUM_NODES \|\|Total number of nodes in the job's resource allocation.		! scope="row" \| SLURM_JOB_NUM_NODES
			\|Total number of nodes in the job's resource allocation.
	\|-		\|-
	\| SLURM_JOB_PARTITION \|\| Name of the partition (i.e. queue) in which the job is running.		! scope="row" \| SLURM_JOB_PARTITION
			\| Name of the partition (i.e. queue) in which the job is running.
	\|-		\|-
	\| SLURM_NTASKS \|\| Same as -n, --ntasks		! scope="row" \| SLURM_NTASKS
			\| Same as -n, --ntasks
	\|-		\|-
	\| SLURM_NTASKS_PER_NODE \|\| Number of tasks requested per node. Only set if the --ntasks-per-node option is specified.		! scope="row" \| SLURM_NTASKS_PER_NODE
			\| Number of tasks requested per node. Only set if the --ntasks-per-node option is specified.
	\|-		\|-
	\| SLURM_SUBMIT_DIR \|\| The directory from which sbatch was invoked.		! scope="row" \| SLURM_SUBMIT_DIR
			\| The directory from which sbatch was invoked.
	\|-		\|-
	\| SLURM_TASKS_PER_NODE \|\| Number of tasks to be initiated on each node.		! scope="row" \| SLURM_TASKS_PER_NODE
			\| Number of tasks to be initiated on each node.
	\|-		\|-
	\|}		\|}

Moses at 18:11, 9 January 2026

2026-01-09T18:11:10Z

← Older revision		Revision as of 14:11, 9 January 2026
Line 256:		Line 256:

	module load OpenMPI/5.0.3-GCC-13.3.0		module load OpenMPI/5.0.3-GCC-13.3.0
	~~mpirun ./mympi.exe~~

			srun ./mympi.exe
	</pre>		</pre>

Line 285:		Line 285:

	time ./a.out		time ./a.out

	</pre>		</pre>

Line 313:		Line 312:

	velvetg [options]		velvetg [options]

	</pre>		</pre>

Line 342:		Line 340:
	export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK		export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

	~~mpirun~~ ./myhybridprog.exe		srun ./myhybridprog.exe

	</pre>		</pre>

Line 366:		Line 363:

	time ./a.out < input_$SLURM_ARRAY_TASK_ID		time ./a.out < input_$SLURM_ARRAY_TASK_ID

	</pre>		</pre>

Shtsai at 17:50, 9 January 2026

2026-01-09T17:50:39Z

← Older revision		Revision as of 13:50, 9 January 2026
Line 11:		Line 11:
	[[#top\|Back to Top]]		[[#top\|Back to Top]]

	===~~Queues~~ defined on the teaching cluster===		===Partitions (queues) defined on the teaching cluster===

	There are different ~~queues~~ defined on the teaching cluster. The SLURM queueing system refers to queues as partition. Users are required to specify, in the job submission script or as job submission command line arguments, the ~~queue~~ and the resources needed by the job in order for it to be assigned to compute node(s) that have enough available resources (such as number of cores, amount of memory, GPU cards, etc).		There are different partitions defined on the teaching cluster. The SLURM queueing system refers to queues as partition. Users are required to specify, in the job submission script or as job submission command line arguments, the partition and the resources needed by the job in order for it to be assigned to compute node(s) that have enough available resources (such as number of cores, amount of memory, GPU cards, etc).

	The table below summarizes the partitions (queues) defined and the compute nodes that they target:		The table below summarizes the partitions (queues) defined and the compute nodes that they target:

Shtsai: /* Queues defined on the teaching cluster */

2026-01-09T17:44:28Z

Queues defined on the teaching cluster

← Older revision		Revision as of 13:44, 9 January 2026
Line 30:		Line 30:
	\| highmem \|\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.		\| highmem \|\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.
	\|-		\|-
	\| gpu \|\| GPU\|\| 1 \|\| 32-core, 190GB RAM, Intel ~~Sapphire Rapids~~, 1 P100 GPU \|\| For GPU-enabled jobs.		\| gpu \|\| GPU\|\| 1 \|\| 32-core, 190GB RAM, Intel Skylake, 1 P100 GPU \|\| For GPU-enabled jobs.
	\|-		\|-
	\| interactive \|\| Intel \|\| 2 \|\| 32-core, 128GB RAM, Intel Xeon \|\| For interactive jobs.		\| interactive \|\| Intel \|\| 2 \|\| 32-core, 128GB RAM, Intel Xeon \|\| For interactive jobs.
			\|-
			\| franklin_gpu \|\| GPU\|\| 2 \|\| 96-core, 1TB RAM, Intel Sapphire Rapids, 4 A30 GPU \|\| For Franklin College only.
	\|-		\|-

Moses: /* GPU/CUDA */

2026-01-09T15:44:27Z

GPU/CUDA

← Older revision		Revision as of 11:44, 9 January 2026
Line 388:		Line 388:
	cd $SLURM_SUBMIT_DIR		cd $SLURM_SUBMIT_DIR

	ml Amber/22.0-foss-~~2021b~~-AmberTools-22.3-CUDA-11.4.1		ml Amber/24.3-foss-2022a-AmberTools-24.10-CUDA-12.1.1

	srun $AMBERHOME/bin/pmemd.cuda -O -i ./prod.in -o prod.out -p ./dimerFBP_GOL.prmtop -c ./restart.rst -r prod.rst -x prod.mdcrd		srun $AMBERHOME/bin/pmemd.cuda -O -i ./prod.in -o prod.out -p ./dimerFBP_GOL.prmtop -c ./restart.rst -r prod.rst -x prod.mdcrd
Line 420:		Line 420:

	<pre class="gcommand">		<pre class="gcommand">
	PARTITION AVAIL TIMELIMIT NODES STATE NODELIST		PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
	~~highmem~~ up 7-00:00:00 2 idle rb1-[1-2]		batch up 7-00:00:00 8 idle rb1-[3-10]
	~~interactive~~ up 7-00:00:00 2 idle ~~rb1-[11~~-~~12]~~		gpu up 7-00:00:00 1 idle c4-23
	~~fsr4601~~ up 1:00 8 idle rb1-[3-10]		highmem up 7-00:00:00 2 idle rb1-[1-2]
	~~fsr8602~~ up 10:00 8 idle rb1-[3-10]		interactive up 7-00:00:00 2 idle rb1-[11-12]
	~~batch~~ up 2-00:00:00 ~~3 mix rb1-3,rb1~~-[6-8]		franklin_gpu up 7-00:00:00 2 idle b8-[6-7]
	~~batch~~ up ~~2-00:00~~:00 ~~1 alloc~~ rb1-4		fsr4601 up 1:00 8 idle rb1-[3-10]
	~~batch~~ up ~~2-00~~:00~~:00 36~~ idle rb1-[~~5,9~~-10]		fsr8602 up 10:00 8 idle rb1-[3-10]
	</pre>		</pre>
	where some common values of STATE are:		where some common values of STATE are:
Line 453:		Line 453:

	<pre class="gcomment">		<pre class="gcomment">
	~~[shtsai~~@~~teach1 ~]~~$ interact -h		zhuofei@teach-sub1 test$ interact -h

	Usage: interact [OPTIONS]		Usage: interact [OPTIONS]

Moses: /* Header lines */

2026-01-09T15:40:51Z

Header lines

← Older revision		Revision as of 11:40, 9 January 2026
Line 84:		Line 84:
	Set a limit on the total run time. Acceptable formats include "minutes", "minutes:seconds", "hours:minutes:seconds", "days-hours", "days-hours:minutes" and "days-hours:minutes:seconds"		Set a limit on the total run time. Acceptable formats include "minutes", "minutes:seconds", "hours:minutes:seconds", "days-hours", "days-hours:minutes" and "days-hours:minutes:seconds"

	* --mem=MB		* --mem=num'''GB''' or num'''MB'''
	Maximum memory per node the job will need in MegaBytes		Maximum memory per node the job will need in Gigabytes (GB) or MegaBytes (MB)

	* --mem-per-cpu=MB		* --mem-per-cpu=num'''GB''' or num'''MB'''
	Memory required per allocated CPU in MegaBytes		Memory required per allocated CPU in Gigabytes (GB) or MegaBytes (MB)

	* -N, --nodes=num		* -N, --nodes=num

Moses: /* Job submission Scripts */

2026-01-09T15:35:29Z

Job submission Scripts

← Older revision		Revision as of 11:35, 9 January 2026
Line 49:		Line 49:
	===Job submission Scripts===		===Job submission Scripts===

	Users are required to specify '''the number of cores''', '''the amount of memory''', '''the partition name''', and '''the maximum wallclock time''' needed by the job.		Users are required to specify '''1. the number of cores''', '''2. the amount of memory''', '''3. the partition name''', and '''4. the maximum wallclock time''' needed by the job.

	====Header lines====		====Header lines====

Moses: /* Job submission Scripts */

2026-01-09T15:34:52Z

Job submission Scripts

← Older revision		Revision as of 11:34, 9 January 2026
Line 49:		Line 49:
	===Job submission Scripts===		===Job submission Scripts===

	Users are required to specify the number of cores, the amount of memory, the ~~queue~~ name, and the maximum wallclock time needed by the job.		Users are required to specify '''the number of cores''', '''the amount of memory''', '''the partition name''', and '''the maximum wallclock time''' needed by the job.

	====Header lines====		====Header lines====

Moses: /* Queues defined on the teaching cluster */

2026-01-09T15:33:52Z

Queues defined on the teaching cluster

← Older revision		Revision as of 11:33, 9 January 2026
Line 26:		Line 26:

	\|-		\|-
	\| batch \|\| Intel\|\| 8 \|\| 32-core, 128GB RAM, AMD Naples \|\| ~~Regular nodes~~.		\| batch \|\| Intel\|\| 8 \|\| 32-core, 128GB RAM, AMD Naples \|\| For regular jobs.
	\|-		\|-
	\| highmem \|\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.		\| highmem \|\| Intel \|\| 2 \|\| 64-core, 1TB RAM, Intel Xeon \|\| For high memory jobs.

Moses at 15:27, 9 January 2026

2026-01-09T15:27:53Z

Show changes