Maker-Sapelo2: Difference between revisions

From Research Computing Center Wiki
Jump to navigation Jump to search
No edit summary
 
(9 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[Category:Sap2test]][[Category:Software]][[Category:Bioinformatics]]   
[[Category:Sapelo2]][[Category:Software]][[Category:Bioinformatics]]   
=== Category ===
=== Category ===


Line 6: Line 6:
=== Program On ===
=== Program On ===


Sap2test
Sapelo2


=== Version ===
=== Version ===
   
   
3.01.03
3.01.04
   
   
=== Author / Distributor ===
=== Author / Distributor ===
Line 24: Line 24:
=== Running Program ===
=== Running Program ===
   
   
Also refer to [[Running Jobs on Sap2test]]
Also refer to [[Running Jobs on Sapelo2]]


Version 3.01.03 is installed at /apps/gb/Maker/3.01.03
Version 3.01.04 is installed at /apps/eb/Maker/3.01.04-foss-2022a


Example to prepare the configuration files.   
Example to prepare the configuration files.   
<pre class="gscript">
<pre class="gscript">
module load Maker/3.01.03
module load MAKER/3.01.04-foss-2022a
maker -CTL
maker -CTL
</pre>
</pre>


Command "maker -CTL" will generate the configuration files maker_bopts.ctl, maker_evm.ctl, maker_exe.ctl, maker_opts.ctl.  
Command "maker -CTL" will generate the configuration files '''maker_bopts.ctl''', '''maker_evm.ctl''', '''maker_exe.ctl''', and '''maker_opts.ctl''' in your working folder. You need to configure those configuration files before you run maker. Examples can be found at [http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#Running_MAKER_with_example_data Running_MAKER_with_example_data].
 
Maker example can be found at [http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#Running_MAKER_with_example_data Running_MAKER_with_example_data]
 


==== Running maker in a MPI job ====
==== Running maker in a MPI job ====


Here is an example of submission script for running maker in a MPI job of 60 MPI processes running on 5 nodes:  
Here is an example of submission script (sub.sh) for running maker in a MPI job of 60 MPI processes running on 5 nodes:  


<pre class="gscript">
<pre class="gscript">
Line 50: Line 47:
#SBATCH --ntasks-per-node=12
#SBATCH --ntasks-per-node=12
#SBATCH --cpus-per-task=1
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --mem-per-cpu=1G
#SBATCH --time=8:00:00
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --output=log.o%j
Line 60: Line 57:
cd $SLURM_SUBMIT_DIR
cd $SLURM_SUBMIT_DIR


ml Maker/3.01.03
ml MAKER/3.01.04-foss-2022a


mpirun -n 60 maker
mpirun -n 60 maker
</pre>  
</pre>  


==== Running maker in a memory-sharing threaded job ====
==== Running maker in a memory-sharing threaded job ====


Here is an example of submission script for running maker in a threaded job of 16 threads running on 1 node:
Here is an example of submission script (sub.sh) for running maker in a threaded job of 16 threads running on 1 node:


<pre class="gscript">
<pre class="gscript">
Line 76: Line 72:
#SBATCH --ntasks=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --cpus-per-task=16
#SBATCH --mem=10G
#SBATCH --mem=20G
#SBATCH --time=1:00:00
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --output=log.o%j
#SBATCH --export=NONE
#SBATCH --export=NONE
Line 83: Line 79:
cd $SLURM_SUBMIT_DIR
cd $SLURM_SUBMIT_DIR


ml Maker/3.01.03
ml MAKER/3.01.04-foss-2022a


maker -cpu 16
maker -cpu 16
</pre>  
</pre>  


==== Running maker in a single-core job ====


Here is an example of submission script (sub.sh) for running maker in a single-core job:


Then submit the job by:
<pre class="gscript">
<pre class="gcommand">
#!/bin/bash
qsub ./sub.sh
#SBATCH --partition=batch
</pre>
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --mem=10G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE


cd $SLURM_SUBMIT_DIR


Version 2.31.9_conda is installed at /usr/local/apps/gb/maker/2.31.9_conda via Conda. The sample data and sample control files are at /usr/local/apps/gb/maker/2.31.9_conda/data
ml MAKER/3.01.04-foss-2022a
 
Example of the conda version script sub.sh to run on at queue.
First step is to prepare the configuration files. Here uses <u>/scratch/yhuang/08202018a</u> as an example of working dir: 
<div class="gscript2">
ml Miniconda3/4.4.10<br>
. /usr/local/apps/eb/Miniconda3/4.4.10/etc/profile.d/conda.sh<br>
conda activate /usr/local/apps/gb/maker/2.31.9_conda<br>
cd <u>/scratch/yhuang/08202018a</u><br>
maker -CTL <br>
cp /usr/local/apps/gb/maker/2.31.9_conda/data/maker_exe.ctl /usr/local/apps/gb/maker/2.31.9_conda/data/maker_opts.ctl .
</div>
Command "maker -CTL" will generate the configuration files. At the central location, the maker_exe.ctl and maker_opts.ctl are further configured for more application dependencies. So here copies from the central files.
 
Here is the example of submission script:
<div class="gscript2">
<nowiki>#</nowiki>PBS -S /bin/bash<br>
<nowiki>#</nowiki>PBS -N j_<!-- NAME BEGIN -->maker<!-- NAME END --><br>
<nowiki>#</nowiki>PBS -q batch<br>
<nowiki>#</nowiki>PBS -l nodes=<u>1</u>:ppn=<u>1</u>:<u>AMD</u><br>
<nowiki>#</nowiki>PBS -l walltime=<u>8:00:00</u><br>
<nowiki>#</nowiki>PBS -l mem=<u>10gb</u><br>
<nowiki>#</nowiki>PBS -M <u>username@uga.edu</u><br> 
<nowiki>#</nowiki>PBS -m abe<br>
 
ml Miniconda3/4.4.10<br>
. /usr/local/apps/eb/Miniconda3/4.4.10/etc/profile.d/conda.sh<br>
conda activate /usr/local/apps/gb/maker/2.31.9_conda<br>
mkdir -p /tmp/lscratch/$PBS_JOBID/tmp<br>
maker <u>[options] -TMP /tmp/lscratch/$PBS_JOBID/tmp <maker_opts> <maker_bopts> <maker_exe></u><br>
conda deactivate<br>
</div>


maker
</pre>


Then submit the job by:
Then submit the job by:
Line 134: Line 109:
qsub ./sub.sh
qsub ./sub.sh
</pre>
</pre>
Version 2.31.10 is installed in /usr/local/apps/gb/Maker/2.31.10
To use this version of Maker, please first load the module with
<pre class="gscript">
module load Maker/2.31.10-foss-2016b 
</pre>
This module will automatically load its dependencies. This version was compiled with MPI support and loading this module will also load the appropriate OpenMPI modules and set LD_PRELOAD to point to the correct MPI library.


=== Documentation ===
=== Documentation ===
Version 3.01.02-beta


<pre class="gcommand">
Maker wiki site is at https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/Main_Page
MAKER version 3.01.02
 
Usage:
 
    maker [options] <maker_opts> <maker_bopts> <maker_exe>
 
 
Description:
 
    MAKER is a program that produces gene annotations in GFF3 format using
    evidence such as EST alignments and protein homology. MAKER can be used to
    produce gene annotations for new genomes as well as update annotations
    from existing genome databases.
 
    The three input arguments are control files that specify how MAKER should
    behave. All options for MAKER should be set in the control files, but a
    few can also be set on the command line. Command line options provide a
    convenient machanism to override commonly altered control file values.
    MAKER will automatically search for the control files in the current
    working directory if they are not specified on the command line.
 
    Input files listed in the control options files must be in fasta format
    unless otherwise specified. Please see MAKER documentation to learn more
    about control file  configuration.  MAKER will automatically try and
    locate the user control files in the current working directory if these
    arguments are not supplied when initializing MAKER.
 
    It is important to note that MAKER does not try and recalculated data that
    it has already calculated.  For example, if you run an analysis twice on
    the same dataset you will notice that MAKER does not rerun any of the
    BLAST analyses, but instead uses the blast analyses stored from the
    previous run. To force MAKER to rerun all analyses, use the -f flag.
 
    MAKER also supports parallelization via MPI on computer clusters. Just
    launch MAKER via mpiexec (i.e. mpiexec -n 40 maker). MPI support must be
    configured during the MAKER installation process for this to work though
   
 
Options:
 
    -genome|g <file>    Overrides the genome file path in the control files
 
    -RM_off|R          Turns all repeat masking options off.
 
    -datastore/         Forcably turn on/off MAKER's two deep directory
      nodatastore        structure for output. Always on by default.
 
    -old_struct        Use the old directory styles (MAKER 2.26 and lower)
 
    -base    <string>  Set the base name MAKER uses to save output files.
                        MAKER uses the input genome file name by default.
 
    -tries|t <integer>  Run contigs up to the specified number of tries.
 
    -cpus|c  <integer>  Tells how many cpus to use for BLAST analysis.
                        Note: this is for BLAST and not for MPI!
 
    -force|f            Forces MAKER to delete old files before running again.
                        This will require all blast analyses to be rerun.
 
    -again|a            recaculate all annotations and output files even if no
                        settings have changed. Does not delete old analyses.
 
    -quiet|q            Regular quiet. Only a handlful of status messages.
 
    -qq                Even more quiet. There are no status messages.
 
    -dsindex            Quickly generate datastore index file. Note that this
                        will not check if run settings have changed on contigs
 
    -nolock            Turn off file locks. May be usful on some file systems,
                        but can cause race conditions if running in parallel.
 
    -TMP                Specify temporary directory to use.
 
    -CTL                Generate empty control files in the current directory.
 
    -OPTS              Generates just the maker_opts.ctl file.
 
    -BOPTS              Generates just the maker_bopts.ctl file.
 
    -EXE                Generates just the maker_exe.ctl file.
 
    -MWAS    <option>  Easy way to control mwas_server for web-based GUI
 
                              options:  STOP
                                        START
                                        RESTART
 
    -version            Prints the MAKER version.
 
    -help|?            Prints this usage statement.
 
</pre>
 
Version 2.31.9_cond


<pre class="gcommand">
<pre class="gcommand">
 
ml MAKER/3.01.04-foss-2022a
ml Miniconda3/4.4.10
conda activate /usr/local/apps/gb/maker/2.31.9_conda
maker -h
maker -h


MAKER version 2.31.9
MAKER version 3.01.04


Usage:
Usage:
Line 308: Line 176:


     -force|f            Forces MAKER to delete old files before running again.
     -force|f            Forces MAKER to delete old files before running again.
                        This will require all blast analyses to be rerun.
This will require all blast analyses to be rerun.


     -again|a            recaculate all annotations and output files even if no
     -again|a            recaculate all annotations and output files even if no
                        settings have changed. Does not delete old analyses.
settings have changed. Does not delete old analyses.


     -quiet|q            Regular quiet. Only a handlful of status messages.
     -quiet|q            Regular quiet. Only a handlful of status messages.
Line 348: Line 216:
=== Installation ===
=== Installation ===
   
   
Conda
Source download from http://www.yandell-lab.org/
   
   
=== System ===
=== System ===
64-bit Linux
64-bit Linux

Latest revision as of 10:06, 6 September 2023

Category

Bioinformatics

Program On

Sapelo2

Version

3.01.04

Author / Distributor

Yandell-lab

Description

More details from http://gmod.org/wiki/MAKER_Tutorial#What_does_MAKER_do.3F

MAKER is an easy-to-use genome annotation pipeline designed to be usable by small research groups with little bioinformatics experience; however, MAKER is also designed to be scalable and is appropriate for projects of any size even including use by large sequence centers. MAKER can be used for de novo annotation of newly sequenced genomes, for updating existing annotations to reflect new evidence, or just to combine annotations, evidence, and quality control statistics for use in other GMOD programs like GBrowse, JBrowse, Chado, and Apollo.

Running Program

Also refer to Running Jobs on Sapelo2

Version 3.01.04 is installed at /apps/eb/Maker/3.01.04-foss-2022a

Example to prepare the configuration files.

module load MAKER/3.01.04-foss-2022a
maker -CTL

Command "maker -CTL" will generate the configuration files maker_bopts.ctl, maker_evm.ctl, maker_exe.ctl, and maker_opts.ctl in your working folder. You need to configure those configuration files before you run maker. Examples can be found at Running_MAKER_with_example_data.

Running maker in a MPI job

Here is an example of submission script (sub.sh) for running maker in a MPI job of 60 MPI processes running on 5 nodes:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=maker-mpi
#SBATCH --ntasks=60
#SBATCH --ntasks-per-node=12
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

export OMPI_MCA_mpi_warn_on_fork=0
unset SLURM_EXPORT_ENV

cd $SLURM_SUBMIT_DIR

ml MAKER/3.01.04-foss-2022a

mpirun -n 60 maker

Running maker in a memory-sharing threaded job

Here is an example of submission script (sub.sh) for running maker in a threaded job of 16 threads running on 1 node:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=20G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

cd $SLURM_SUBMIT_DIR

ml MAKER/3.01.04-foss-2022a

maker -cpu 16

Running maker in a single-core job

Here is an example of submission script (sub.sh) for running maker in a single-core job:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --mem=10G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

cd $SLURM_SUBMIT_DIR

ml MAKER/3.01.04-foss-2022a

maker

Then submit the job by:

qsub ./sub.sh

Documentation

Maker wiki site is at https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/Main_Page

ml MAKER/3.01.04-foss-2022a
maker -h

MAKER version 3.01.04

Usage:

     maker [options] <maker_opts> <maker_bopts> <maker_exe>


Description:

     MAKER is a program that produces gene annotations in GFF3 format using
     evidence such as EST alignments and protein homology. MAKER can be used to
     produce gene annotations for new genomes as well as update annotations
     from existing genome databases.

     The three input arguments are control files that specify how MAKER should
     behave. All options for MAKER should be set in the control files, but a
     few can also be set on the command line. Command line options provide a
     convenient machanism to override commonly altered control file values.
     MAKER will automatically search for the control files in the current
     working directory if they are not specified on the command line.

     Input files listed in the control options files must be in fasta format
     unless otherwise specified. Please see MAKER documentation to learn more
     about control file  configuration.  MAKER will automatically try and
     locate the user control files in the current working directory if these
     arguments are not supplied when initializing MAKER.

     It is important to note that MAKER does not try and recalculated data that
     it has already calculated.  For example, if you run an analysis twice on
     the same dataset you will notice that MAKER does not rerun any of the
     BLAST analyses, but instead uses the blast analyses stored from the
     previous run. To force MAKER to rerun all analyses, use the -f flag.

     MAKER also supports parallelization via MPI on computer clusters. Just
     launch MAKER via mpiexec (i.e. mpiexec -n 40 maker). MPI support must be
     configured during the MAKER installation process for this to work though
     

Options:

     -genome|g <file>    Overrides the genome file path in the control files

     -RM_off|R           Turns all repeat masking options off.

     -datastore/         Forcably turn on/off MAKER's two deep directory
      nodatastore        structure for output.  Always on by default.

     -old_struct         Use the old directory styles (MAKER 2.26 and lower)

     -base    <string>   Set the base name MAKER uses to save output files.
                         MAKER uses the input genome file name by default.

     -tries|t <integer>  Run contigs up to the specified number of tries.

     -cpus|c  <integer>  Tells how many cpus to use for BLAST analysis.
                         Note: this is for BLAST and not for MPI!

     -force|f            Forces MAKER to delete old files before running again.
			 This will require all blast analyses to be rerun.

     -again|a            recaculate all annotations and output files even if no
			 settings have changed. Does not delete old analyses.

     -quiet|q            Regular quiet. Only a handlful of status messages.

     -qq                 Even more quiet. There are no status messages.

     -dsindex            Quickly generate datastore index file. Note that this
                         will not check if run settings have changed on contigs

     -nolock             Turn off file locks. May be usful on some file systems,
                         but can cause race conditions if running in parallel.

     -TMP                Specify temporary directory to use.

     -CTL                Generate empty control files in the current directory.

     -OPTS               Generates just the maker_opts.ctl file.

     -BOPTS              Generates just the maker_bopts.ctl file.

     -EXE                Generates just the maker_exe.ctl file.

     -MWAS    <option>   Easy way to control mwas_server for web-based GUI

                              options:  STOP
                                        START
                                        RESTART

     -version            Prints the MAKER version.

     -help|?             Prints this usage statement.

Back to Top

Installation

Source download from http://www.yandell-lab.org/

System

64-bit Linux