Available Toolchains and Toolchain Compatibility: Difference between revisions

From Research Computing Center Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(45 intermediate revisions by 3 users not shown)
Line 3: Line 3:
We use '''EasyBuild''' to build most of the software modules on the Sapelo2 cluster. Easybuild is a software build and installation framework that allows us to manage scientific software on High Performance Computing (HPC) systems in an efficient way. EasyBuild employs so-called '''compiler toolchains''' or, simply '''toolchains''' for short, in handling the build and installation processes. A typical toolchain consists of one or more compilers, usually put together with some libraries for specific functionality, e.g., for using an MPI stack for distributed computing, or which provide optimized routines for commonly used math operations, e.g., the well-known BLAS/LAPACK APIs for linear algebra routines. For detailed info about EasyBuild and its toolchains, please refer to [https://easybuild.readthedocs.io/en/latest/ EasyBuild] and [https://easybuild.readthedocs.io/en/latest/Concepts_and_Terminology.html#toolchains toolchains]
We use '''EasyBuild''' to build most of the software modules on the Sapelo2 cluster. Easybuild is a software build and installation framework that allows us to manage scientific software on High Performance Computing (HPC) systems in an efficient way. EasyBuild employs so-called '''compiler toolchains''' or, simply '''toolchains''' for short, in handling the build and installation processes. A typical toolchain consists of one or more compilers, usually put together with some libraries for specific functionality, e.g., for using an MPI stack for distributed computing, or which provide optimized routines for commonly used math operations, e.g., the well-known BLAS/LAPACK APIs for linear algebra routines. For detailed info about EasyBuild and its toolchains, please refer to [https://easybuild.readthedocs.io/en/latest/ EasyBuild] and [https://easybuild.readthedocs.io/en/latest/Concepts_and_Terminology.html#toolchains toolchains]


==Available foss/fosscuda toolchains on Sapelo2==
==Available foss and intel toolchains on Sapelo2==
 
*Common compiler toolchains:
 
{| class="wikitable sortable" width="100%" border="1" cellspacing="0" cellpadding="2" align="left"
{| class="wikitable sortable" width="100%" border="1" cellspacing="0" cellpadding="2" align="left"
|-
|-
Line 17: Line 14:
|-
|-
<!--LIST BEGIN-->|-
<!--LIST BEGIN-->|-
|foss
| rowspan="6" |foss
|2020b
|2023b
| GCC/10.2.0
|GCC/13.2.0
|OpenMPI/4.0.5
|OpenMPI/4.1.6
| OpenBLAS/0.3.12, ScaLAPACK/2.1.0
|FlexiBLAS/3.3.1, ScaLAPACK/2.2.0
|FFTW/3.3.8
|FFTW/3.3.10, FFTW.MPI/3.3.10
|-
|2023a
|GCC/12.3.0
|OpenMPI/4.1.5
|FlexiBLAS/3.3.1, ScaLAPACK/2.2.0
|FFTW/3.3.10, FFTW.MPI/3.3.10
|-
|2022b
|GCC/12.2.0
|OpenMPI/4.1.4
|FlexiBLAS/3.2.1, ScaLAPACK/2.2.0
|FFTW/3.3.10, FFTW.MPI/3.3.10
|-
|-
|foss||2019b||GCC/8.3.0||OpenMPI/3.1.4||OpenBLAS/0.3.7, ScaLAPACK/2.0.2||FFTW/3.3.8
|2022a
|GCC/11.3.0
|OpenMPI/4.1.4
|FlexiBLAS/3.2.0, ScaLAPACK/2.2.0
|FFTW/3.3.10, FFTW.MPI/3.3.10
|-
|-
|gompi
|2021b
| 2020b
| GCC/11.2.0
| GCC/10.2.0
|OpenMPI/4.1.1
|OpenMPI/4.0.5
| FlexiBLAS/3.0.4, ScaLAPACK/2.1.0
|
|FFTW/3.3.10
|
|-
|-
|gompi ||2019b|| GCC/8.3.0||OpenMPI/3.1.4|| ||
|2019b||GCC/8.3.0||OpenMPI/3.1.4||OpenBLAS/0.3.7, ScaLAPACK/2.0.2||FFTW/3.3.8
|-
|-
|GCC||6.4.0-2.28, 8.3.0, 9.2.0, 10.2.0|| || || ||
| rowspan="6" |gompi
|2023b
|GCC/13.2.0
|OpenMPI/4.1.6
| rowspan="6" |
| rowspan="6" |
|-
|-
|GCCcore||6.4.0, 8.3.0, 9.2.0, 10.2.0|| || || ||
|2023a
|GCC/12.3.0
|OpenMPI/4.1.5
|-
|2022b
|GCC/12.2.0
|OpenMPI/4.1.4
|-
|2022a
|GCC/11.3.0
|OpenMPI/4.1.4
|-
| 2021b
| GCC/11.2.0
|OpenMPI/4.1.1
|-
|2019b|| GCC/8.3.0||OpenMPI/3.1.4
|-
| rowspan="3" | gfbf
| 2023b|| GCC/13.2.0 || rowspan="3" | || OpenBLAS/0.3.24,FlexiBLAS/3.3.1  || FFTW/3.3.10 
|-
| 2023a|| GCC/12.3.0 ||  OpenBLAS/0.3.23,FlexiBLAS/3.3.1  || FFTW/3.3.10 
|-
| 2022b|| GCC/12.2.0 ||  OpenBLAS/0.3.21,FlexiBLAS/3.2.1  || FFTW/3.3.10
|-
|GCC||13.2.0, 12.3.0, 12.2.0, 11.3.0, 11.2.0, 10.2.0, 8.3.0|| rowspan="2" | || rowspan="2" | || rowspan="2"  |  
|-
|GCCcore||13.2.0, 12.3.0, 12.2.0, 11.3.0, 11.2.0, 10.2.0, 8.3.0
|-
|-
<!--LIST END-->
<!--LIST END-->
|}
|}
*Compiler toolchains with GPU/CUDA support:


{| class="wikitable sortable" width="100%" border="1" cellspacing="0" cellpadding="2" align="left"
{| class="wikitable sortable" width="100%" border="1" cellspacing="0" cellpadding="2" align="left"
Line 48: Line 90:
! scope="col" width="100px" |Name
! scope="col" width="100px" |Name
! scope="col" width="100px" |Version
! scope="col" width="100px" |Version
! scope="col" width="100px" |Compiler(s)
! scope="col" width="100px" |GCCcore
! scope="col" width="100px" |Intel Compiler (icc, icpc, ifort)
! scope="col" width="100px" |MPI library
! scope="col" width="100px" |MPI library
! scope="col" width="100px" | Linear algebra library
! scope="col" width="100px" |MKL, FFT
! scope="col" width="100px" |FFT
|-
|-
<!--LIST BEGIN-->|-
<!--LIST BEGIN-->|-
|fosscuda
| rowspan="4" |intel
|2020b
|2023b
|GCC/10.2.0, CUDA/11.1.1
|GCCcore/13.2.0
|OpenMPI/4.0.5
|intel-compilers/2023.2.1
|OpenBLAS/0.3.12, ScaLAPACK/2.1.0
|impi/2021.10.0
| FFT/3.3.8
|imkl/2023.2.0 imkl-FFTW/2023.2.0
|-
|2023a
|GCCcore/12.3.0
|intel-compilers/2023.1.0
|impi/2021.9.0
|imkl/2023.1.0 imkl-FFTW/2023.1.0
|-
|-
|fosscuda||2019b||GCC/8.3.0, CUDA/10.1.243||OpenMPI/3.1.4 ||OpenBLAS/0.3.7, ScaLAPACK/2.0.2||FFT/3.3.8
|2022a
|GCCcore/11.3.0
|intel-compilers/2022.1.0
|impi/2021.6.0
|imkl/2022.1.0 imkl-FFTW/2022.1.0
|-
|-
|gompic
|2021b
|2020b
|GCCcore/11.2.0
|GCC/10.2.0, CUDA/11.1.1
|intel-compilers/2021.4.0
|OpenMPI/4.0.5
|impi/2021.4.0
|
|imkl/2021.4.0 imkl-FFTW/2021.4.0
|
|-
|-
|gompic||2019b||GCC/8.3.0, CUDA/10.1.243 || OpenMPI/3.1.4|| ||
| rowspan="4" |iimpi
|2023b
|GCCcore/13.2.0
|intel-compilers/2023.2.1  
|impi/2021.10.0
| rowspan="8" |
|-
|-
|gcccuda
|2023a
|2020b
|GCCcore/12.3.0
|GCC/10.2.0, CUDA/11.1.1
|intel-compilers/2023.1.0
|
|impi/2021.9.0
|
|
|-
|-
|gcccuda||2019b||GCC/8.3.0, CUDA/10.1.243|| || ||
|2022a
|GCCcore/11.3.0
|intel-compilers/2022.1.0
|impi/2021.6.0
|-
|2021b
|GCCcore/11.2.0
|intel-compilers/2021.4.0  
|impi/2021.4.0
|-
| rowspan="4" |intel-compilers
|2023.2.1  
|GCCcore/13.2.0
| rowspan="4" |
|-
|2023.1.0
|GCCcore/12.3.0
|-
|2022.1.0
|GCCcore/11.3.0
|-
|2021.4.0
|GCCcore/11.2.0
|-
|-
<!--LIST END-->
<!--LIST END-->
|}
|}
==Available intel toolchains on Sapelo2==


For a complete list of EasyBuild toolchains, please refer to [https://easybuild.readthedocs.io/en/latest/version-specific/toolchains.html#vsd-list-toolchains List_of_known_toolchains]
For a complete list of EasyBuild toolchains, please refer to [https://easybuild.readthedocs.io/en/latest/version-specific/toolchains.html#vsd-list-toolchains List_of_known_toolchains]
------
------
[[#top|Back to Top]]
[[#top|Back to Top]]
Line 97: Line 166:
==Toolchain compatibility==
==Toolchain compatibility==


*Software modules built by EasyBuild generally have names in a format like '''Name/Version-Toolchain'''. When you load more than one software module built by EasyBuild, please make sure that all modules are built with '''the same or compatible toolchains'''. You can check this by looking into software modules' names. If you load more than one module and some toolchains are incompatible, you will end up with failing dependencies of Lmod errors, like this:
*Software modules built by EasyBuild generally have names in a format like '''Name/Version-Toolchain'''. When you load more than one software module built by EasyBuild, please make sure that all modules are built with '''the same or compatible toolchains'''. You can check this by looking into software modules' names. If you load more than one module and some toolchains are incompatible, you will receive the following Lmod error:


<pre class="gscript">
<pre class="gscript">
Line 104: Line 173:
</pre>
</pre>


*Also please note that, when a '''conflicting module''' is loaded, one or more dependent modules of the software previously loaded could be replaced, and thus results in the software previously loaded being broken.
*When loading a '''conflicting module,''' one or more dependent modules of the software you previously loaded may be replaced. Most likely the software you loaded earlier got broken because of this.


*The diagram below shows toolchains belonging to GCC/8.3.0 family. They are all compatible with each other. The blue connection lines with arrows illustrate toolchain build hierarchy, for example, GCC-8.3.0 is built on the top of GCCcore-8.3.0.
*Table below gives examples of compatible toolchains. Multiple software modules built with compatible toolchains can be loaded together in one computational job or one interactive session:


[[File:Sapelo2 common compatible toolchains – GCC8.3 resized.jpg]]
*
 
{| class="wikitable"
==Dependency compatibility (to be added)==
|+
 
!Toolchain Family
*The diagram below gives examples showing what will happen when incompatible toolchains and conflicting modules are loaded:
!Compatible Toolchains
|-
| rowspan="6" |foss
|foss/2023b , gompi/2023b , gfbf/2023b, GCC/13.2.0 , GCCcore/13.2.0
|-
|foss/2023a , gompi/2023a , gfbf/2023a, GCC/12.3.0 , GCCcore/12.3.0
|-
|foss/2022b , gompi/2022b , gfbf/2022b, GCC/12.2.0 , GCCcore/12.2.0
|-
|foss/2022a , gompi/2022a , GCC/11.3.0 , GCCcore/11.3.0
|-
|foss/2021b , gompi/2021b , GCC/11.2.0 , GCCcore/11.2.0
|-
|foss/2019b , gompi/2019b , GCC/8.3.0 , GCCcore/8.3.0
|-
| rowspan="4" |intel
|intel/2023b , iimpi/2023b , intel-compilers/2023.2.1 , GCCcore/13.2.0
|-
|intel/2023a , iimpi/2023a , intel-compilers/2023.1.0 , GCCcore/12.3.0
|-
|intel/2022a , iimpi/2022a , intel-compilers/2022.1.0 , GCCcore/11.3.0
|-
|intel/2021b , iimpi/2021b , intel-compilers/2021.4.0 , GCCcore/11.2.0
|}




------
------
[[#top|Back to Top]]
[[#top|Back to Top]]

Latest revision as of 14:04, 5 August 2024


We use EasyBuild to build most of the software modules on the Sapelo2 cluster. Easybuild is a software build and installation framework that allows us to manage scientific software on High Performance Computing (HPC) systems in an efficient way. EasyBuild employs so-called compiler toolchains or, simply toolchains for short, in handling the build and installation processes. A typical toolchain consists of one or more compilers, usually put together with some libraries for specific functionality, e.g., for using an MPI stack for distributed computing, or which provide optimized routines for commonly used math operations, e.g., the well-known BLAS/LAPACK APIs for linear algebra routines. For detailed info about EasyBuild and its toolchains, please refer to EasyBuild and toolchains

Available foss and intel toolchains on Sapelo2

Name Version Compiler(s) MPI library Linear algebra library FFT
foss 2023b GCC/13.2.0 OpenMPI/4.1.6 FlexiBLAS/3.3.1, ScaLAPACK/2.2.0 FFTW/3.3.10, FFTW.MPI/3.3.10
2023a GCC/12.3.0 OpenMPI/4.1.5 FlexiBLAS/3.3.1, ScaLAPACK/2.2.0 FFTW/3.3.10, FFTW.MPI/3.3.10
2022b GCC/12.2.0 OpenMPI/4.1.4 FlexiBLAS/3.2.1, ScaLAPACK/2.2.0 FFTW/3.3.10, FFTW.MPI/3.3.10
2022a GCC/11.3.0 OpenMPI/4.1.4 FlexiBLAS/3.2.0, ScaLAPACK/2.2.0 FFTW/3.3.10, FFTW.MPI/3.3.10
2021b GCC/11.2.0 OpenMPI/4.1.1 FlexiBLAS/3.0.4, ScaLAPACK/2.1.0 FFTW/3.3.10
2019b GCC/8.3.0 OpenMPI/3.1.4 OpenBLAS/0.3.7, ScaLAPACK/2.0.2 FFTW/3.3.8
gompi 2023b GCC/13.2.0 OpenMPI/4.1.6
2023a GCC/12.3.0 OpenMPI/4.1.5
2022b GCC/12.2.0 OpenMPI/4.1.4
2022a GCC/11.3.0 OpenMPI/4.1.4
2021b GCC/11.2.0 OpenMPI/4.1.1
2019b GCC/8.3.0 OpenMPI/3.1.4
gfbf 2023b GCC/13.2.0 OpenBLAS/0.3.24,FlexiBLAS/3.3.1 FFTW/3.3.10
2023a GCC/12.3.0 OpenBLAS/0.3.23,FlexiBLAS/3.3.1 FFTW/3.3.10
2022b GCC/12.2.0 OpenBLAS/0.3.21,FlexiBLAS/3.2.1 FFTW/3.3.10
GCC 13.2.0, 12.3.0, 12.2.0, 11.3.0, 11.2.0, 10.2.0, 8.3.0
GCCcore 13.2.0, 12.3.0, 12.2.0, 11.3.0, 11.2.0, 10.2.0, 8.3.0
Name Version GCCcore Intel Compiler (icc, icpc, ifort) MPI library MKL, FFT
intel 2023b GCCcore/13.2.0 intel-compilers/2023.2.1 impi/2021.10.0 imkl/2023.2.0 imkl-FFTW/2023.2.0
2023a GCCcore/12.3.0 intel-compilers/2023.1.0 impi/2021.9.0 imkl/2023.1.0 imkl-FFTW/2023.1.0
2022a GCCcore/11.3.0 intel-compilers/2022.1.0 impi/2021.6.0 imkl/2022.1.0 imkl-FFTW/2022.1.0
2021b GCCcore/11.2.0 intel-compilers/2021.4.0 impi/2021.4.0 imkl/2021.4.0 imkl-FFTW/2021.4.0
iimpi 2023b GCCcore/13.2.0 intel-compilers/2023.2.1 impi/2021.10.0
2023a GCCcore/12.3.0 intel-compilers/2023.1.0 impi/2021.9.0
2022a GCCcore/11.3.0 intel-compilers/2022.1.0 impi/2021.6.0
2021b GCCcore/11.2.0 intel-compilers/2021.4.0 impi/2021.4.0
intel-compilers 2023.2.1 GCCcore/13.2.0
2023.1.0 GCCcore/12.3.0
2022.1.0 GCCcore/11.3.0
2021.4.0 GCCcore/11.2.0

For a complete list of EasyBuild toolchains, please refer to List_of_known_toolchains

Back to Top

Toolchain compatibility

  • Software modules built by EasyBuild generally have names in a format like Name/Version-Toolchain. When you load more than one software module built by EasyBuild, please make sure that all modules are built with the same or compatible toolchains. You can check this by looking into software modules' names. If you load more than one module and some toolchains are incompatible, you will receive the following Lmod error:
Lmod has detected the following error:  
These module(s) exist but cannot be loaded as requested
  • When loading a conflicting module, one or more dependent modules of the software you previously loaded may be replaced. Most likely the software you loaded earlier got broken because of this.
  • Table below gives examples of compatible toolchains. Multiple software modules built with compatible toolchains can be loaded together in one computational job or one interactive session:
Toolchain Family Compatible Toolchains
foss foss/2023b , gompi/2023b , gfbf/2023b, GCC/13.2.0 , GCCcore/13.2.0
foss/2023a , gompi/2023a , gfbf/2023a, GCC/12.3.0 , GCCcore/12.3.0
foss/2022b , gompi/2022b , gfbf/2022b, GCC/12.2.0 , GCCcore/12.2.0
foss/2022a , gompi/2022a , GCC/11.3.0 , GCCcore/11.3.0
foss/2021b , gompi/2021b , GCC/11.2.0 , GCCcore/11.2.0
foss/2019b , gompi/2019b , GCC/8.3.0 , GCCcore/8.3.0
intel intel/2023b , iimpi/2023b , intel-compilers/2023.2.1 , GCCcore/13.2.0
intel/2023a , iimpi/2023a , intel-compilers/2023.1.0 , GCCcore/12.3.0
intel/2022a , iimpi/2022a , intel-compilers/2022.1.0 , GCCcore/11.3.0
intel/2021b , iimpi/2021b , intel-compilers/2021.4.0 , GCCcore/11.2.0


Back to Top

Retrieved from "https://wiki.gacrc.uga.edu/index.php?title=Available_Toolchains_and_Toolchain_Compatibility&oldid=21993"