Python-Sapelo2: Difference between revisions

From Research Computing Center Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(25 intermediate revisions by 3 users not shown)
Line 5: Line 5:
Sapelo2
Sapelo2
=== Version ===
=== Version ===
2.7.14, 2.7.15, 2.7.16, 3.7.4, 3.8.2
2.7.18, 3.7.4, 3.8.6, 3.9.6, 3.10.4, 3.10.8, 3.11.3, 3.11.5


=== Author / Distributor ===
=== Author / Distributor ===
Line 16: Line 16:
=== Running Program ===
=== Running Program ===
   
   
Also refer to [[Running Jobs on Sapelo]]
Also refer to [[Running Jobs on Sapelo2]]


We have a set of pythons installed on Sapelo2:
There is no default python command on the system. The OS comes with a default /usr/bin/python2 (v. 2.7.18) and a default /usr/bin/python3 (v. 3.6.8). Scripts that have the first line:
<pre class="gscript">
#!/usr/bin/python
</pre>
will not work on Sapelo2 (with the Rocky 8 OS). We recommend that you change this line to
<pre class="gscript">
#!/usr/bin/env python
</pre>
and load one of the Python modules before running the script.
 
 
We have a set of Python versions installed on Sapelo2 via environment modules. To find out which Python modules are installed, please run the following command on the cluster:
 
<pre class="gscript">
ml spider Python
</pre>


For example, this output could look like this:
<pre class="gscript">
<pre class="gscript">
ml spider Python
ml spider Python
Line 29: Line 45:


     Versions:
     Versions:
         Python/2.7.14-GCCcore-6.4.0-bare
         Python/2.7.18-GCCcore-11.2.0-bare
         Python/2.7.15-fosscuda-2018b
         Python/2.7.18-GCCcore-11.2.0
         Python/2.7.16-GCCcore-8.3.0
        Python/2.7.18-GCCcore-11.3.0-bare
         Python/2.7.18-GCCcore-11.3.0
         Python/3.7.4-GCCcore-8.3.0
         Python/3.7.4-GCCcore-8.3.0
         Python/3.8.2-GCCcore-8.3.0
         Python/3.8.6-GCCcore-10.2.0
        Python/3.9.6-GCCcore-11.2.0-bare
        Python/3.9.6-GCCcore-11.2.0
        Python/3.10.4-GCCcore-11.3.0-bare
        Python/3.10.4-GCCcore-11.3.0
        Python/3.10.8-GCCcore-12.2.0-bare
        Python/3.10.8-GCCcore-12.2.0
        Python/3.11.3-GCCcore-12.3.0
        Python/3.11.5-GCCcore-13.2.0
     Other possible modules matches:
     Other possible modules matches:
         Biopython  Boost.Python  GitPython  IPython  bx-python  netcdf4-python  openslide-python
         Biopython  Boost.Python  GitPython  IPython  bx-python  flatbuffers-python  ...


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------
   To find other possible module matches execute:
   To find other possible module matches execute:


       $ module -r spider '.*Python.*'
       $ module -r spider '.*Python.*'


------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------
   For detailed information about a specific "Python" package (including how to load the modules) use the module's full name.
   For detailed information about a specific "Python" package (including how to load the modules) use the module's full name.
   Note that names that have a trailing (E) are extensions provided by other modules.
   Note that names that have a trailing (E) are extensions provided by other modules.
   For example:
   For example:


     $ module spider Python/3.8.2-GCCcore-8.3.0
     $ module spider Python/3.11.5-GCCcore-13.2.0
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------
 
</pre>  
</pre>  


----
[[#top|Back to Top]]
===About Python packages===
Some common third-party Python packages are installed individually as separate modules on the cluster.
====numpy, scipy, pandas====
The numpy, scipy, pandas, mpi4py, mpmath, and deap packages are installed in SciPy-bundle module:
<pre class="gscript">
ml spider SciPy-bundle   
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  SciPy-bundle:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Bundle of Python packages for scientific software
    Versions:
        SciPy-bundle/2021.10-foss-2021b-Python-2.7.18
        SciPy-bundle/2021.10-foss-2021b
        SciPy-bundle/2021.10-foss-2022a-Python-2.7.18
        SciPy-bundle/2022.05-foss-2022a
----------------------------------------------------------------------------------
  For detailed information about a specific "SciPy-bundle" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:
    $ module spider SciPy-bundle/2022.05-foss-2022a
----------------------------------------------------------------------------------
</pre>
Note that, when you load SciPy-bundle module, a specific version of Python that the package is built for will be loaded at first. This means that you don't have to load the Python module separately from loading SciPy-bundle module. For example:
*Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2021b is loaded.
*Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when SciPy-bundle/2022.05-foss-2022a is loaded.
*Python/2.7.18-GCCcore-11.2.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2021b-Python-2.7.18 is loaded.
*Python/2.7.18-GCCcore-11.3.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2022a-Python-2.7.18 is loaded.
----
[[#top|Back to Top]]
====matplotlib====
The matplotlib package is installed in matplotlib module.
<pre class="gscript">
ml spider matplotlib
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  matplotlib:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python
      scripts, the python and ipython shell, web application servers, and six graphical user interface toolkits.
    Versions:
        matplotlib/2.2.5-foss-2021b-Python-2.7.18
        matplotlib/2.2.5-foss-2022a-Python-2.7.18
        matplotlib/3.4.3-foss-2021b
        matplotlib/3.5.2-foss-2021b
        matplotlib/3.5.2-foss-2022a
----------------------------------------------------------------------------------
  For detailed information about a specific "matplotlib" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:
    $ module spider matplotlib/3.5.2-foss-2022a
----------------------------------------------------------------------------------
</pre>
Same as SciPy-bundle, when you load matplotlib module, a specific version of Python that matplotlib is built for will be loaded at first, which means you don't have to load the Python module separately from loading matplotlib module. For example:
*Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when you load matplotlib/3.4.3-foss-2021b or matplotlib/3.5.2-foss-2021b.
*Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when you load matplotlib/3.5.2-foss-2022a.
*Python/2.7.18-GCCcore-11.2.0 will be automatically loaded when you load matplotlib/2.2.5-foss-2021b-Python-2.7.18.
*Python/2.7.18-GCCcore-11.3.0 will be automatically loaded when you load matplotlib/2.2.5-foss-2022a-Python-2.7.18.
----
[[#top|Back to Top]]
====Biopython====
The Biopython package is installed in Biopython module.
<pre class="gscript">
ml spider biopython
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Biopython:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Biopython is a set of freely available tools for biological computation written in Python by an international team of developers. It is a distributed collaborative effort to develop Python libraries
      and applications which address the needs of current and future work in bioinformatics.
    Versions:
        Biopython/1.79-foss-2021b
        Biopython/1.79-foss-2022a
----------------------------------------------------------------------------------
  For detailed information about a specific "Biopython" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:
    $ module spider Biopython/1.79-foss-2022a
----------------------------------------------------------------------------------
</pre>


'''Example of using Python3'''
Same as above SciPy-bundle and matplotlib, when you load Biopython module, a specific version of Python that the package is built for will be loaded at first. You don't have to load the Python module separately from loading Biopython module. For ecample:


To use Python 3.8.2, you can use the following in the first line of your Python code:
*Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when Biopython/1.79-foss-2021b is loaded.
 
*Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when Biopython/1.79-foss-2022a is loaded.
 
 
----
[[#top|Back to Top]]
 
===Examples===
 
====Example of using Python3====
 
To use Python 3.9.6, you can use the following in the first line of your Python code:


<pre class="gscript">
<pre class="gscript">
#!/apps/eb/Python/3.8.2-GCCcore-8.3.0/bin/python3
#!/apps/eb/Python/3.9.6-GCCcore-11.2.0/bin/python3
</pre>
</pre>


Line 67: Line 222:


<pre class="gcommand">
<pre class="gcommand">
module load Python/3.8.2-GCCcore-8.3.0
module load Python/3.9.6-GCCcore-11.2.0


python myProgram.py
python myProgram.py
</pre>
</pre>


'''Example of using Python2'''
====Example of using Python2====


To use Python 2.7.16, you can use the following in the first line of your Python code:
To use Python 2.7.18, you can use the following in the first line of your Python code:


<pre class="gscript">
<pre class="gscript">
#!/apps/eb/Python/2.7.16-GCCcore-8.3.0/bin/python
#!/apps/eb/Python/2.7.18-GCCcore-11.2.0/bin/python
</pre>
</pre>


Line 87: Line 242:


<pre class="gcommand">
<pre class="gcommand">
module load Python/2.7.16-GCCcore-8.3.0
module load Python/2.7.18-GCCcore-11.2.0


python myProgram.py
python myProgram.py
</pre>
</pre>
----
[[#top|Back to Top]]


=== Documentation ===
=== Documentation ===

Latest revision as of 12:01, 12 September 2024

Category

Programming

Program On

Sapelo2

Version

2.7.18, 3.7.4, 3.8.6, 3.9.6, 3.10.4, 3.10.8, 3.11.3, 3.11.5

Author / Distributor

The Python Software Foundation

Description

Python is a programming language.

Running Program

Also refer to Running Jobs on Sapelo2

There is no default python command on the system. The OS comes with a default /usr/bin/python2 (v. 2.7.18) and a default /usr/bin/python3 (v. 3.6.8). Scripts that have the first line:

#!/usr/bin/python

will not work on Sapelo2 (with the Rocky 8 OS). We recommend that you change this line to

#!/usr/bin/env python

and load one of the Python modules before running the script.


We have a set of Python versions installed on Sapelo2 via environment modules. To find out which Python modules are installed, please run the following command on the cluster:

ml spider Python

For example, this output could look like this:

ml spider Python

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Python:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      Python is a programming language that lets you work more quickly and integrate your systems more effectively.

     Versions:
        Python/2.7.18-GCCcore-11.2.0-bare
        Python/2.7.18-GCCcore-11.2.0
        Python/2.7.18-GCCcore-11.3.0-bare
        Python/2.7.18-GCCcore-11.3.0
        Python/3.7.4-GCCcore-8.3.0
        Python/3.8.6-GCCcore-10.2.0
        Python/3.9.6-GCCcore-11.2.0-bare
        Python/3.9.6-GCCcore-11.2.0
        Python/3.10.4-GCCcore-11.3.0-bare
        Python/3.10.4-GCCcore-11.3.0
        Python/3.10.8-GCCcore-12.2.0-bare
        Python/3.10.8-GCCcore-12.2.0
        Python/3.11.3-GCCcore-12.3.0
        Python/3.11.5-GCCcore-13.2.0
     Other possible modules matches:
        Biopython  Boost.Python  GitPython  IPython  bx-python  flatbuffers-python  ...

----------------------------------------------------------------------------------
  To find other possible module matches execute:

      $ module -r spider '.*Python.*'

----------------------------------------------------------------------------------
  For detailed information about a specific "Python" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider Python/3.11.5-GCCcore-13.2.0
----------------------------------------------------------------------------------


Back to Top

About Python packages

Some common third-party Python packages are installed individually as separate modules on the cluster.

numpy, scipy, pandas

The numpy, scipy, pandas, mpi4py, mpmath, and deap packages are installed in SciPy-bundle module:

ml spider SciPy-bundle    

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  SciPy-bundle:
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Bundle of Python packages for scientific software

     Versions:
        SciPy-bundle/2021.10-foss-2021b-Python-2.7.18
        SciPy-bundle/2021.10-foss-2021b
        SciPy-bundle/2021.10-foss-2022a-Python-2.7.18
        SciPy-bundle/2022.05-foss-2022a

----------------------------------------------------------------------------------
  For detailed information about a specific "SciPy-bundle" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider SciPy-bundle/2022.05-foss-2022a
----------------------------------------------------------------------------------

Note that, when you load SciPy-bundle module, a specific version of Python that the package is built for will be loaded at first. This means that you don't have to load the Python module separately from loading SciPy-bundle module. For example:

  • Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2021b is loaded.
  • Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when SciPy-bundle/2022.05-foss-2022a is loaded.
  • Python/2.7.18-GCCcore-11.2.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2021b-Python-2.7.18 is loaded.
  • Python/2.7.18-GCCcore-11.3.0 will be automatically loaded when SciPy-bundle/2021.10-foss-2022a-Python-2.7.18 is loaded.

Back to Top

matplotlib

The matplotlib package is installed in matplotlib module.

ml spider matplotlib

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  matplotlib:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      matplotlib is a python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms. matplotlib can be used in python
      scripts, the python and ipython shell, web application servers, and six graphical user interface toolkits.

     Versions:
        matplotlib/2.2.5-foss-2021b-Python-2.7.18
        matplotlib/2.2.5-foss-2022a-Python-2.7.18
        matplotlib/3.4.3-foss-2021b
        matplotlib/3.5.2-foss-2021b
        matplotlib/3.5.2-foss-2022a

----------------------------------------------------------------------------------
  For detailed information about a specific "matplotlib" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider matplotlib/3.5.2-foss-2022a
----------------------------------------------------------------------------------

Same as SciPy-bundle, when you load matplotlib module, a specific version of Python that matplotlib is built for will be loaded at first, which means you don't have to load the Python module separately from loading matplotlib module. For example:

  • Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when you load matplotlib/3.4.3-foss-2021b or matplotlib/3.5.2-foss-2021b.
  • Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when you load matplotlib/3.5.2-foss-2022a.
  • Python/2.7.18-GCCcore-11.2.0 will be automatically loaded when you load matplotlib/2.2.5-foss-2021b-Python-2.7.18.
  • Python/2.7.18-GCCcore-11.3.0 will be automatically loaded when you load matplotlib/2.2.5-foss-2022a-Python-2.7.18.

Back to Top

Biopython

The Biopython package is installed in Biopython module.

ml spider biopython

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  Biopython:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Biopython is a set of freely available tools for biological computation written in Python by an international team of developers. It is a distributed collaborative effort to develop Python libraries
      and applications which address the needs of current and future work in bioinformatics. 

     Versions:
        Biopython/1.79-foss-2021b
        Biopython/1.79-foss-2022a

----------------------------------------------------------------------------------
  For detailed information about a specific "Biopython" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider Biopython/1.79-foss-2022a
----------------------------------------------------------------------------------

Same as above SciPy-bundle and matplotlib, when you load Biopython module, a specific version of Python that the package is built for will be loaded at first. You don't have to load the Python module separately from loading Biopython module. For ecample:

  • Python/3.9.6-GCCcore-11.2.0 will be automatically loaded when Biopython/1.79-foss-2021b is loaded.
  • Python/3.10.4-GCCcore-11.3.0 will be automatically loaded when Biopython/1.79-foss-2022a is loaded.



Back to Top

Examples

Example of using Python3

To use Python 3.9.6, you can use the following in the first line of your Python code:

#!/apps/eb/Python/3.9.6-GCCcore-11.2.0/bin/python3

Or use the following line in the first line of your Python code:

#!/usr/bin/env python

an then you run your program as:

module load Python/3.9.6-GCCcore-11.2.0

python myProgram.py

Example of using Python2

To use Python 2.7.18, you can use the following in the first line of your Python code:

#!/apps/eb/Python/2.7.18-GCCcore-11.2.0/bin/python

Or use the following line in the first line of your Python code:

#!/usr/bin/env python

an then you run your program as:

module load Python/2.7.18-GCCcore-11.2.0

python myProgram.py

Back to Top

Documentation

Please see http://www.python.org/

Installation

EasyBuild recipes

System

64-bit Linux