Shtsai: Created page with "Category:Sapelo2 = Job submission partitions on Sapelo2 = == Overview == This page describes the Slurm partitions available on the Sapelo2 cluster, including job limits and the resources available in each partition. In Slurm, queues are called ''partitions''. When you submit a job, you must request both: * the partition to use, and * the resources your job needs, such as CPU cores, memory, or GPU devices. Slurm will reject a job submission if no nodes match the r..."

2026-03-31T13:34:22Z

Created page with "Category:Sapelo2 = Job submission partitions on Sapelo2 = == Overview == This page describes the Slurm partitions available on the Sapelo2 cluster, including job limits and the resources available in each partition. In Slurm, queues are called ''partitions''. When you submit a job, you must request both: * the partition to use, and * the resources your job needs, such as CPU cores, memory, or GPU devices. Slurm will reject a job submission if no nodes match the r..."

New page

[[Category:Sapelo2]]

= Job submission partitions on Sapelo2 =

== Overview ==
This page describes the Slurm partitions available on the Sapelo2 cluster, including job limits and the resources available in each partition.

In Slurm, queues are called ''partitions''. When you submit a job, you must request both:
* the partition to use, and
* the resources your job needs, such as CPU cores, memory, or GPU devices.

Slurm will reject a job submission if no nodes match the resources you request. For background on Slurm, see [[Migrating from Torque to Slurm]].

== How to use this page ==
Use the first table to choose a partition based on job type and time limit.

Use the second table to confirm that your requested resources fit within the hardware available in that partition.

== Partition limits ==
{| class="wikitable unsortable"
|+ Sapelo2 partitions, time limits, and per-user job limits
|-
! scope="col" | Partition name
! scope="col" | Time limit
! scope="col" | Maximum running jobs per user
! scope="col" | Maximum submitted jobs per user
! scope="col" | Intended use and notes
|-
! scope="row" | <code>batch</code>
| 7 days
| 250
| 10,000
| Standard partition for regular compute jobs on general-purpose nodes.
|-
! scope="row" | <code>batch_30d</code>
| 30 days
| 1
| 2
| Standard partition for long-running jobs on regular nodes. A user may have one running job and one pending job, or two pending jobs and no running job. A third submission to this partition will be rejected.
|-
! scope="row" | <code>highmem_p</code>
| 7 days
| 6
| 100
| High-memory partition for jobs that require more memory than standard nodes provide.
|-
! scope="row" | <code>highmem_30d_p</code>
| 30 days
| 1
| 2
| High-memory partition for long-running jobs. A user may have one running job and one pending job, or two pending jobs and no running job. A third submission to this partition will be rejected.
|-
! scope="row" | <code>hugemem_p</code>
| 7 days
| 4
| 4
| Huge-memory partition for jobs needing up to 3 TB of memory.
|-
! scope="row" | <code>hugemem_30d_p</code>
| 30 days
| 4
| 4
| Huge-memory partition for long-running jobs needing up to 3 TB of memory.
|-
! scope="row" | <code>gpu_p</code>
| 7 days
| 6
| 20
| GPU-enabled partition for jobs that require one or more GPUs.
|-
! scope="row" | <code>gpu_30d_p</code>
| 30 days
| 2
| 2
| GPU-enabled partition for long-running jobs. A user may have one running job and one pending job, or two pending jobs and no running job. A third submission to this partition will be rejected.
|-
! scope="row" | <code>inter_p</code>
| 2 days
| 3
| 20
| Interactive partition for interactive jobs on regular nodes.
|-
! scope="row" | <code>''name''_p</code>
| Variable
| Variable
| Variable
| Partition for a specific group's buy-in nodes. Replace <code>''name''</code> with the group-specific partition prefix.
|}

== Resource limits by partition ==
Before submitting a job, make sure your requested memory, CPU cores, and GPU count fit within the limits of the partition you choose.

In the table below, the phrase '''partition maximum''' identifies the largest per-node resource values available within that partition. This replaces color-only emphasis so that the information is available to all users.

{| class="wikitable unsortable"
|+ Node resources available in each Sapelo2 partition
|-
! scope="col" | Partition
! scope="col" | Number of nodes
! scope="col" | Memory per node (GB)
! scope="col" | CPU cores per node
! scope="col" | Processor type
! scope="col" | GPU configuration
! scope="col" | Notes
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 16
| 740
| 128
| AMD EPYC Genoa (4th gen)
| None
| Partition maximum for memory and cores is available on this node type.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 120
| 500
| 128
| AMD EPYC Milan (3rd gen)
| None
| Partition maximum for cores is also available on this node type.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 4
| 250
| 64
| AMD EPYC Milan (3rd gen)
| None
| Standard-capacity general-purpose nodes.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 2
| 120
| 64
| AMD EPYC Milan (3rd gen)
| None
| Standard-capacity general-purpose nodes.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 123
| 120
| 64
| AMD EPYC Rome (2nd gen)
| None
| Standard-capacity general-purpose nodes.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 25
| 120
| 32
| AMD EPYC Naples (1st gen)
| None
| Lower-core-count general-purpose nodes.
|-
! scope="row" | <code>batch</code>, <code>batch_30d</code>
| 40
| 180
| 32
| Intel Xeon Skylake
| None
| Lower-core-count general-purpose nodes.
|-
! scope="row" | <code>highmem_p</code>, <code>highmem_30d_p</code>
| 10
| 500
| 32
| AMD EPYC Naples (1st gen)
| None
| High-memory nodes.
|-
! scope="row" | <code>highmem_p</code>, <code>highmem_30d_p</code>
| 2
| 990
| 128
| AMD EPYC Milan (3rd gen)
| None
| Partition maximum for memory and cores is available on this node type.
|-
! scope="row" | <code>highmem_p</code>, <code>highmem_30d_p</code>
| 12
| 990
| 32
| AMD EPYC Milan (3rd gen)
| None
| High-memory nodes with fewer available cores per node.
|-
! scope="row" | <code>hugemem_p</code>, <code>hugemem_30d_p</code>
| 3
| 3000
| 48
| AMD EPYC Genoa (4th gen)
| None
| Partition maximum for memory and cores is available on this node type.
|-
! scope="row" | <code>hugemem_p</code>, <code>hugemem_30d_p</code>
| 2
| 2000
| 32
| AMD EPYC Rome (2nd gen)
| None
| Huge-memory nodes with lower maximums than the partition peak.
|-
! scope="row" | <code>gpu_p</code>, <code>gpu_30d_p</code>
| 2
| 180
| 32
| Intel Xeon Skylake
| 1 NVIDIA P100
| Older GPU nodes.
|-
! scope="row" | <code>gpu_p</code>, <code>gpu_30d_p</code>
| 2
| 120
| 64
| AMD EPYC Rome (2nd gen)
| 1 NVIDIA V100S
| Single-GPU nodes with 64 cores.
|-
! scope="row" | <code>gpu_p</code>, <code>gpu_30d_p</code>
| 14
| 1000
| 64
| AMD EPYC Milan (3rd gen)
| 4 NVIDIA A100
| Partition maximum for memory is available on this node type.
|-
! scope="row" | <code>gpu_p</code>, <code>gpu_30d_p</code>
| 12
| 1000
| 64
| Intel Xeon Sapphire Rapids
| 4 NVIDIA H100
| Partition maximum for memory is available on this node type.
|-
! scope="row" | <code>gpu_p</code>, <code>gpu_30d_p</code>
| 12
| 740
| 128
| AMD EPYC Genoa (4th gen)
| 4 NVIDIA L4
| Partition maximum for cores is available on this node type.
|-
! scope="row" | <code>''name''_p</code>
| Variable
| Variable
| Variable
| Variable
| Variable
| Resource limits depend on the group's buy-in nodes.
|}

== Choosing a partition ==
A general rule of thumb is:

* Use <code>batch</code> for most non-GPU jobs.
* Use <code>batch_30d</code> only when your job genuinely needs a longer wall time.
* Use <code>highmem_p</code> or <code>highmem_30d_p</code> when your memory requirements exceed what standard nodes provide.
* Use <code>hugemem_p</code> or <code>hugemem_30d_p</code> for jobs that need very large memory allocations, including jobs approaching 3 TB of memory.
* Use <code>gpu_p</code> or <code>gpu_30d_p</code> for GPU jobs.
* Use <code>inter_p</code> for interactive work.
* Use <code>''name''_p</code> only if your group has access to a buy-in partition with that name.

== Example Slurm directives ==
The examples below show common ways to request a partition.

=== Regular compute job ===
<syntaxhighlight lang="bash">
#SBATCH --partition=batch
#SBATCH --time=2-00:00:00
#SBATCH --cpus-per-task=16
#SBATCH --mem=64G
</syntaxhighlight>

=== GPU job ===
<syntaxhighlight lang="bash">
#SBATCH --partition=gpu_p
#SBATCH --time=1-00:00:00
#SBATCH --gres=gpu:1
#SBATCH --cpus-per-task=8
#SBATCH --mem=64G
</syntaxhighlight>

=== High-memory job ===
<syntaxhighlight lang="bash">
#SBATCH --partition=highmem_p
#SBATCH --time=12:00:00
#SBATCH --cpus-per-task=16
#SBATCH --mem=700G
</syntaxhighlight>

== Terms used on this page ==
; Partition
: A Slurm queue that determines which nodes your job may run on.
; Time limit
: The maximum wall-clock runtime allowed for a job in that partition.
; Running jobs
: Jobs currently executing for a user in that partition.
; Submitted jobs
: Total jobs a user may have in the partition, including running and pending jobs.
; Buy-in nodes
: Nodes purchased by a specific group and made available through a group-specific partition.

== Related documentation ==
* [[Migrating from Torque to Slurm]]

Job Submission partitions on Sapelo2 access - Revision history