Difference between revisions of "MPI"

From Research Computing Center Wiki
Jump to navigation Jump to search
Line 1: Line 1:
 
[[Category:Sapelo2]][[Category:Teaching]]
 
[[Category:Sapelo2]][[Category:Teaching]]
  
−
===MPI Libraries for parallel jobs on Sapelo2===
+
==MPI Libraries for parallel jobs on Sapelo2==
  
 
All compute nodes on Sapelo2 have Infiniband (IB) interconnect. For a subset of ('''older''') compute nodes, internodal communication among them is provided by a '''QDR''' Infiniband network (40Gbps). For another subset of ('''newer''') compute nodes, these communications are provided by an '''EDR''' Infiniband network (100Gbps).  
 
All compute nodes on Sapelo2 have Infiniband (IB) interconnect. For a subset of ('''older''') compute nodes, internodal communication among them is provided by a '''QDR''' Infiniband network (40Gbps). For another subset of ('''newer''') compute nodes, these communications are provided by an '''EDR''' Infiniband network (100Gbps).  
Line 59: Line 59:
  
 
Once the appropriate module is loaded, you can compile code with '''mpicc, mpicxx,''' '''mpic++,''' '''mpifort,''' '''mpif90,''' '''mpif77,''' etc. and you can run applications that were linked to the '''MPI libraries''' loaded by the module.
 
Once the appropriate module is loaded, you can compile code with '''mpicc, mpicxx,''' '''mpic++,''' '''mpifort,''' '''mpif90,''' '''mpif77,''' etc. and you can run applications that were linked to the '''MPI libraries''' loaded by the module.
−
 
−
 
 
====OpenMPI====
 
====OpenMPI====
  
Line 148: Line 146:
 
[[#top|Back to Top]]
 
[[#top|Back to Top]]
  
−
===MPI Libraries for parallel jobs on the teaching cluster===
+
==MPI Libraries for parallel jobs on the teaching cluster==
  
−
All compute nodes on the teaching cluster are connected via a gigabit Ethernet network. MPI libraries currently installed on the teaching cluster are listed below.
+
All compute nodes on the teaching cluster are connected via a '''gigabit Ethernet''' network. MPI libraries currently installed on the teaching cluster are listed below.
  
−
====OpenMPI====
+
====MVAPICH2====
  
−
You can find all OpenMPI modules available on the teaching cluster by running the following command on a teaching cluster node:
+
You can find all MVAPICH2 modules available on the teaching cluster by running the following command on a Sapelo2 node:
 
<pre class="gcommand">
 
<pre class="gcommand">
−
module spider openmpi
+
module spider MVAPICH2
 
</pre>
 
</pre>
 +
Module names that have an -QDR or -EDR extension only work for the corresponding IB network. Module names without such an extension can be used on either IB network. The module names have the format MVAPICH2/''Version''-''CompilerToolchain''-''ToolchainVersion''.
 +
 +
For example, these are some of the modules available:
 +
 +
<pre class="gcomment">
 +
zhuofei@teach-sub1 ~$ ml spider MVAPICH2
 +
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  MVAPICH2:
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
    Description:
 +
      This is an MPI 3.1 implementation. It is based on MPICH2 and MVICH.
 +
 +
    Versions:
 +
        MVAPICH2/2.3.2-GCC-7.3.0-2.30-EDR
 +
        MVAPICH2/2.3.2-GCC-8.3.0-EDR
 +
        MVAPICH2/2.3.2-iccifort-2013_sp1.0.080-EDR
 +
        MVAPICH2/2.3.2-iccifort-2015.2.164-GCC-4.8.5-EDR
 +
        MVAPICH2/2.3.2-iccifort-2018.1.163-GCC-6.4.0-2.28-EDR
 +
        MVAPICH2/2.3.2-iccifort-2019.5.281-EDR
 +
        MVAPICH2/2.3.4-GCC-7.3.0-2.30
 +
    Other possible modules matches:
 +
        gmvapich2
 +
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  To find other possible module matches execute:
 +
 +
      $ module -r spider '.*MVAPICH2.*'
  
−
These are some of the modules available:
+
------------------------------------------------------------------------------------------------------------------------------------
 +
  For detailed information about a specific "MVAPICH2" package (including how to load the modules) use the module's full name.
 +
  Note that names that have a trailing (E) are extensions provided by other modules.
 +
  For example:
  
−
* OpenMPI 2.1.2, using GNU 6.4.0 compilers, available in module OpenMPI/2.1.2-GCC-6.4.0-2.28. To use it, load the module with
+
    $ module spider MVAPICH2/2.3.4-GCC-7.3.0-2.30
−
<pre class="gcommand">
+
------------------------------------------------------------------------------------------------------------------------------------
−
module load OpenMPI/2.1.2-GCC-6.4.0-2.28
 
 
</pre>
 
</pre>
  
−
* OpenMPI 2.0.2, using GNU 6.3.0 compilers, available in module OpenMPI/2.0.2-GCC-6.3.0-2.27. To use it, load the module with
+
Once the appropriate module is loaded, you can compile code with '''mpicc, mpicxx,''' '''mpic++,''' '''mpifort,''' '''mpif90,''' '''mpif77,''' etc. and you can run applications that were linked to the '''MPI libraries''' loaded by the module.
 +
====OpenMPI====
 +
 
 +
You can find all OpenMPI modules available on the teaching cluster by running the following command on a Sapelo2 node:
 
<pre class="gcommand">
 
<pre class="gcommand">
−
module load OpenMPI/2.0.2-GCC-6.3.0-2.27
+
module spider OpenMPI
 
</pre>
 
</pre>
  
−
* OpenMPI 1.10.3, using GNU 5.4.0 compilers, available in module OpenMPI/1.10.3-GCC-5.4.0-2.26. To use it, load the module with
+
The module names have the format OpenMPI/''Version''-''CompilerToolchain''-''ToolchainVersion''.
−
<pre class="gcommand">
+
 
−
module load OpenMPI/1.10.3-GCC-5.4.0-2.26
+
For example, these are some of the modules available:
 +
 
 +
<pre class="gcomment">
 +
zhuofei@teach-sub1 ~$ ml spider OpenMPI
 +
 
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  OpenMPI:
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
    Description:
 +
      The Open MPI Project is an open source MPI-3 implementation.
 +
 
 +
    Versions:
 +
        OpenMPI/1.8.4-iccifort-2013_sp1.0.080
 +
        OpenMPI/1.8.4-iccifort-2015.2.164-GCC-4.8.5
 +
        OpenMPI/2.1.2-iccifort-2018.1.163-GCC-6.4.0-2.28
 +
        OpenMPI/3.1.1-gcccuda-2018b
 +
        OpenMPI/3.1.4-GCC-8.3.0
 +
        OpenMPI/3.1.4-gcccuda-2019b
 +
 
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  For detailed information about a specific "OpenMPI" package (including how to load the modules) use the module's full name.
 +
  Note that names that have a trailing (E) are extensions provided by other modules.
 +
  For example:
 +
 
 +
    $ module spider OpenMPI/3.1.4-gcccuda-2019b
 +
------------------------------------------------------------------------------------------------------------------------------------
 
</pre>
 
</pre>
 +
 +
Once the appropriate module is loaded, you can compile code with '''mpicc, mpiCC, mpicxx,''' '''mpic++,''' '''mpifort,''' '''mpif90,''' '''mpif77,''' etc. and you can run applications that were linked to the '''MPI libraries''' loaded by the module.
 +
 +
=== Intel MPI ===
 +
You can find all Intel MPI modules available on the teaching cluster by running the following command on a Sapelo2 node:<pre class="gcommand">
 +
module spider impi
 +
</pre>The module names have the format impi/''Version''-''CompilerToolchain''-''ToolchainVersion''.
 +
 +
For example, these are some of the modules available:<pre class="gcomment">
 +
zhuofei@teach-sub1 ~$ ml spider impi
 +
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  impi: impi/2018.5.288-iccifort-2019.5.281
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
    Description:
 +
      Intel MPI Library, compatible with MPICH ABI
 +
 +
 +
    Other possible modules matches:
 +
        iimpi
 +
 +
    This module can be loaded directly: module load impi/2018.5.288-iccifort-2019.5.281
 +
 +
    Help:
 +
      Description
 +
      ===========
 +
      Intel MPI Library, compatible with MPICH ABI
 +
     
 +
     
 +
      More information
 +
      ================
 +
      - Homepage: https://software.intel.com/en-us/intel-mpi-library/
 +
     
 +
------------------------------------------------------------------------------------------------------------------------------------
 +
  To find other possible module matches execute:
 +
 +
      $ module -r spider '.*impi.*'
 +
</pre>Once the appropriate module is loaded, you can compile code with '''mpicc, mpiicc, mpicxx, mpiicpc,''' '''mpiifort,''' '''mpif90,''' '''mpif77,''' etc. and you can run applications that were linked to the '''MPI libraries''' loaded by the module.
 +
 +
[[#top|Back to Top]]

Revision as of 11:19, 8 November 2021


MPI Libraries for parallel jobs on Sapelo2

All compute nodes on Sapelo2 have Infiniband (IB) interconnect. For a subset of (older) compute nodes, internodal communication among them is provided by a QDR Infiniband network (40Gbps). For another subset of (newer) compute nodes, these communications are provided by an EDR Infiniband network (100Gbps).

Various IB-enabled MPI libraries are available and users can set the environment variables for the MPI library of choice by loading the corresponding module file. Some of the MPI libraries were installed specifically for the QDR network or for the EDR network and those module files will have an -QDR or -EDR extension in their names.

For more information on Environment Modules, please see the Lmod page.

The following MPI libraries are available:

MVAPICH2

You can find all MVAPICH2 modules available on Sapelo2 by running the following command on a Sapelo2 node: 

module spider MVAPICH2

Module names that have an -QDR or -EDR extension only work for the corresponding IB network. Module names without such an extension can be used on either IB network. The module names have the format MVAPICH2/Version-CompilerToolchain-ToolchainVersion.

For example, these are some of the modules available: 

[shtsai@ss-sub1 ~]$ module spider MVAPICH2

-----------------------------------------------------------------------------------------------------------------------------------------------
  MVAPICH2:
-----------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      This is an MPI 3.1 implementation. It is based on MPICH2 and MVICH.

     Versions:
        MVAPICH2/2.3.2-GCC-7.3.0-2.30-EDR
        MVAPICH2/2.3.2-GCC-8.3.0-EDR
        MVAPICH2/2.3.2-iccifort-2013_sp1.0.080-EDR
        MVAPICH2/2.3.2-iccifort-2015.2.164-GCC-4.8.5-EDR
        MVAPICH2/2.3.2-iccifort-2018.1.163-GCC-6.4.0-2.28-EDR
        MVAPICH2/2.3.2-iccifort-2019.5.281-EDR
        MVAPICH2/2.3.4-GCC-7.3.0-2.30
        MVAPICH2/2.3.4-GCC-8.3.0
        MVAPICH2/2.3.4-iccifort-2019.5.281
        MVAPICH2/2.3.6-GCC-9.3.0
     Other possible modules matches:
        gmvapich2

-----------------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*MVAPICH2.*'

-----------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "MVAPICH2" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider MVAPICH2/2.3.6-GCC-9.3.0
-----------------------------------------------------------------------------------------------------------------------------------------------

Once the appropriate module is loaded, you can compile code with mpicc, mpicxx, mpic++, mpifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

OpenMPI

You can find all OpenMPI modules available on Sapelo2 by running the following command on a Sapelo2 node: 

module spider OpenMPI

Module names that have an -QDR or -EDR extension only work for the corresponding IB network. Module names without such an extension can be used on either IB network. The module names have the format OpenMPI/Version-CompilerToolchain-ToolchainVersion.

For example, these are some of the modules available: 

[shtsai@ss-sub1 ~]$ module spider OpenMPI

-----------------------------------------------------------------------------------------------------------------------------------------------
  OpenMPI:
-----------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      The Open MPI Project is an open source MPI-3 implementation.

     Versions:
        OpenMPI/1.6.5
        OpenMPI/1.8.4-iccifort-2013_sp1.0.080
        OpenMPI/1.8.4-iccifort-2015.2.164-GCC-4.8.5
        OpenMPI/2.1.2-iccifort-2018.1.163-GCC-6.4.0-2.28-test
        OpenMPI/2.1.2-iccifort-2018.1.163-GCC-6.4.0-2.28
        OpenMPI/3.1.1-gcccuda-2018b
        OpenMPI/3.1.1-iccifort-2018.1.163-GCC-6.4.0-2.28
        OpenMPI/3.1.4-GCC-8.3.0-nosrun
        OpenMPI/3.1.4-GCC-8.3.0-test
        OpenMPI/3.1.4-GCC-8.3.0
        OpenMPI/3.1.4-gcccuda-2019b
        OpenMPI/4.0.3-GCC-9.3.0
        OpenMPI/4.0.5-GCC-10.2.0
        OpenMPI/4.0.5-gcccuda-2020b
        OpenMPI/4.1.1-GCC-10.3.0-EDR
        OpenMPI/4.1.1-GCC-10.3.0-QDR
        OpenMPI/4.1.1-GCC-10.3.0

-----------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "OpenMPI" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider OpenMPI/4.1.1-GCC-10.3.0
-----------------------------------------------------------------------------------------------------------------------------------------------

Once the appropriate module is loaded, you can compile code with mpicc, mpiCC, mpicxx, mpic++, mpifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

Intel MPI

You can find all Intel MPI modules available on Sapelo2 by running the following command on a Sapelo2 node:

module spider impi

The module names have the format impi/Version-CompilerToolchain-ToolchainVersion. For example, these are some of the modules available:

[shtsai@ss-sub1 ~]$ module spider impi

------------------------------------------------------------------------------------------------------------------------------------
  impi:
------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Intel MPI Library, compatible with MPICH ABI

     Versions:
        impi/2018.5.288-iccifort-2019.5.281
        impi/2019.9.304-iccifort-2020.4.304
     Other possible modules matches:
        iimpi

------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*impi.*'

------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "impi" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider impi/2019.9.304-iccifort-2020.4.304
------------------------------------------------------------------------------------------------------------------------------------

Once the appropriate module is loaded, you can compile code with mpicc, mpiicc, mpicxx, mpiicpc, mpiifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

Back to Top

MPI Libraries for parallel jobs on the teaching cluster

All compute nodes on the teaching cluster are connected via a gigabit Ethernet network. MPI libraries currently installed on the teaching cluster are listed below.

MVAPICH2

You can find all MVAPICH2 modules available on the teaching cluster by running the following command on a Sapelo2 node: 

module spider MVAPICH2

Module names that have an -QDR or -EDR extension only work for the corresponding IB network. Module names without such an extension can be used on either IB network. The module names have the format MVAPICH2/Version-CompilerToolchain-ToolchainVersion.

For example, these are some of the modules available: 

zhuofei@teach-sub1 ~$ ml spider MVAPICH2

------------------------------------------------------------------------------------------------------------------------------------
  MVAPICH2:
------------------------------------------------------------------------------------------------------------------------------------
    Description:
      This is an MPI 3.1 implementation. It is based on MPICH2 and MVICH.

     Versions:
        MVAPICH2/2.3.2-GCC-7.3.0-2.30-EDR
        MVAPICH2/2.3.2-GCC-8.3.0-EDR
        MVAPICH2/2.3.2-iccifort-2013_sp1.0.080-EDR
        MVAPICH2/2.3.2-iccifort-2015.2.164-GCC-4.8.5-EDR
        MVAPICH2/2.3.2-iccifort-2018.1.163-GCC-6.4.0-2.28-EDR
        MVAPICH2/2.3.2-iccifort-2019.5.281-EDR
        MVAPICH2/2.3.4-GCC-7.3.0-2.30
     Other possible modules matches:
        gmvapich2

------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*MVAPICH2.*'

------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "MVAPICH2" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider MVAPICH2/2.3.4-GCC-7.3.0-2.30
------------------------------------------------------------------------------------------------------------------------------------

Once the appropriate module is loaded, you can compile code with mpicc, mpicxx, mpic++, mpifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

OpenMPI

You can find all OpenMPI modules available on the teaching cluster by running the following command on a Sapelo2 node: 

module spider OpenMPI

The module names have the format OpenMPI/Version-CompilerToolchain-ToolchainVersion.

For example, these are some of the modules available: 

zhuofei@teach-sub1 ~$ ml spider OpenMPI

------------------------------------------------------------------------------------------------------------------------------------
  OpenMPI:
------------------------------------------------------------------------------------------------------------------------------------
    Description:
      The Open MPI Project is an open source MPI-3 implementation.

     Versions:
        OpenMPI/1.8.4-iccifort-2013_sp1.0.080
        OpenMPI/1.8.4-iccifort-2015.2.164-GCC-4.8.5
        OpenMPI/2.1.2-iccifort-2018.1.163-GCC-6.4.0-2.28
        OpenMPI/3.1.1-gcccuda-2018b
        OpenMPI/3.1.4-GCC-8.3.0
        OpenMPI/3.1.4-gcccuda-2019b

------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "OpenMPI" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider OpenMPI/3.1.4-gcccuda-2019b
------------------------------------------------------------------------------------------------------------------------------------

Once the appropriate module is loaded, you can compile code with mpicc, mpiCC, mpicxx, mpic++, mpifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

Intel MPI

You can find all Intel MPI modules available on the teaching cluster by running the following command on a Sapelo2 node:

module spider impi

The module names have the format impi/Version-CompilerToolchain-ToolchainVersion. For example, these are some of the modules available:

zhuofei@teach-sub1 ~$ ml spider impi

------------------------------------------------------------------------------------------------------------------------------------
  impi: impi/2018.5.288-iccifort-2019.5.281
------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Intel MPI Library, compatible with MPICH ABI


     Other possible modules matches:
        iimpi

    This module can be loaded directly: module load impi/2018.5.288-iccifort-2019.5.281

    Help:
      Description
      ===========
      Intel MPI Library, compatible with MPICH ABI
      
      
      More information
      ================
       - Homepage: https://software.intel.com/en-us/intel-mpi-library/
      
------------------------------------------------------------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*impi.*'

Once the appropriate module is loaded, you can compile code with mpicc, mpiicc, mpicxx, mpiicpc, mpiifort, mpif90, mpif77, etc. and you can run applications that were linked to the MPI libraries loaded by the module.

Back to Top

Retrieved from "https://wiki.gacrc.uga.edu/index.php?title=MPI&oldid=20682"