Maker-Sapelo2: Difference between revisions

From Research Computing Center Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 37: Line 37:


Maker example can be found at [http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#Running_MAKER_with_example_data Running_MAKER_with_example_data]
Maker example can be found at [http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#Running_MAKER_with_example_data Running_MAKER_with_example_data]


==== Running maker in a MPI job ====
==== Running maker in a MPI job ====


Here is an example of submission script for running maker in a MPI job of 60 MPI processes running on 5 nodes:  
Here is an example of submission script (sub.sh) for running maker in a MPI job of 60 MPI processes running on 5 nodes:  


<pre class="gscript">
<pre class="gscript">
Line 50: Line 49:
#SBATCH --ntasks-per-node=12
#SBATCH --ntasks-per-node=12
#SBATCH --cpus-per-task=1
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --mem-per-cpu=1G
#SBATCH --time=8:00:00
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --output=log.o%j
Line 64: Line 63:
mpirun -n 60 maker
mpirun -n 60 maker
</pre>  
</pre>  


==== Running maker in a memory-sharing threaded job ====
==== Running maker in a memory-sharing threaded job ====


Here is an example of submission script for running maker in a threaded job of 16 threads running on 1 node:
Here is an example of submission script (sub.sh) for running maker in a threaded job of 16 threads running on 1 node:


<pre class="gscript">
<pre class="gscript">
Line 76: Line 74:
#SBATCH --ntasks=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --cpus-per-task=16
#SBATCH --mem=10G
#SBATCH --mem=20G
#SBATCH --time=1:00:00
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --output=log.o%j
#SBATCH --export=NONE
#SBATCH --export=NONE
Line 88: Line 86:
</pre>  
</pre>  


==== Running maker in a single-core job ====


Here is an example of submission script (sub.sh) for running maker in a single-core job:


Then submit the job by:
<pre class="gscript">
<pre class="gcommand">
#!/bin/bash
qsub ./sub.sh
#SBATCH --partition=batch
</pre>
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --mem=10G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE


cd $SLURM_SUBMIT_DIR


Version 2.31.9_conda is installed at /usr/local/apps/gb/maker/2.31.9_conda via Conda. The sample data and sample control files are at /usr/local/apps/gb/maker/2.31.9_conda/data
ml Maker/3.01.03
 
Example of the conda version script sub.sh to run on at queue.
First step is to prepare the configuration files. Here uses <u>/scratch/yhuang/08202018a</u> as an example of working dir: 
<div class="gscript2">
ml Miniconda3/4.4.10<br>
. /usr/local/apps/eb/Miniconda3/4.4.10/etc/profile.d/conda.sh<br>
conda activate /usr/local/apps/gb/maker/2.31.9_conda<br>
cd <u>/scratch/yhuang/08202018a</u><br>
maker -CTL <br>
cp /usr/local/apps/gb/maker/2.31.9_conda/data/maker_exe.ctl /usr/local/apps/gb/maker/2.31.9_conda/data/maker_opts.ctl .
</div>
Command "maker -CTL" will generate the configuration files. At the central location, the maker_exe.ctl and maker_opts.ctl are further configured for more application dependencies. So here copies from the central files.
 
Here is the example of submission script:
<div class="gscript2">
<nowiki>#</nowiki>PBS -S /bin/bash<br>
<nowiki>#</nowiki>PBS -N j_<!-- NAME BEGIN -->maker<!-- NAME END --><br>
<nowiki>#</nowiki>PBS -q batch<br>
<nowiki>#</nowiki>PBS -l nodes=<u>1</u>:ppn=<u>1</u>:<u>AMD</u><br>
<nowiki>#</nowiki>PBS -l walltime=<u>8:00:00</u><br>
<nowiki>#</nowiki>PBS -l mem=<u>10gb</u><br>
<nowiki>#</nowiki>PBS -M <u>username@uga.edu</u><br> 
<nowiki>#</nowiki>PBS -m abe<br>
 
ml Miniconda3/4.4.10<br>
. /usr/local/apps/eb/Miniconda3/4.4.10/etc/profile.d/conda.sh<br>
conda activate /usr/local/apps/gb/maker/2.31.9_conda<br>
mkdir -p /tmp/lscratch/$PBS_JOBID/tmp<br>
maker <u>[options] -TMP /tmp/lscratch/$PBS_JOBID/tmp <maker_opts> <maker_bopts> <maker_exe></u><br>
conda deactivate<br>
</div>


maker
</pre>


Then submit the job by:
Then submit the job by:
Line 135: Line 112:
</pre>
</pre>


Version 2.31.10 is installed in /usr/local/apps/gb/Maker/2.31.10
To use this version of Maker, please first load the module with
<pre class="gscript">
module load Maker/2.31.10-foss-2016b 
</pre>
This module will automatically load its dependencies. This version was compiled with MPI support and loading this module will also load the appropriate OpenMPI modules and set LD_PRELOAD to point to the correct MPI library.


=== Documentation ===
=== Documentation ===
Version 3.01.02-beta


<pre class="gcommand">
Maker wiki site is at https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/Main_Page
MAKER version 3.01.02
 
Usage:
 
    maker [options] <maker_opts> <maker_bopts> <maker_exe>
 
 
Description:
 
    MAKER is a program that produces gene annotations in GFF3 format using
    evidence such as EST alignments and protein homology. MAKER can be used to
    produce gene annotations for new genomes as well as update annotations
    from existing genome databases.
 
    The three input arguments are control files that specify how MAKER should
    behave. All options for MAKER should be set in the control files, but a
    few can also be set on the command line. Command line options provide a
    convenient machanism to override commonly altered control file values.
    MAKER will automatically search for the control files in the current
    working directory if they are not specified on the command line.
 
    Input files listed in the control options files must be in fasta format
    unless otherwise specified. Please see MAKER documentation to learn more
    about control file  configuration.  MAKER will automatically try and
    locate the user control files in the current working directory if these
    arguments are not supplied when initializing MAKER.
 
    It is important to note that MAKER does not try and recalculated data that
    it has already calculated.  For example, if you run an analysis twice on
    the same dataset you will notice that MAKER does not rerun any of the
    BLAST analyses, but instead uses the blast analyses stored from the
    previous run. To force MAKER to rerun all analyses, use the -f flag.
 
    MAKER also supports parallelization via MPI on computer clusters. Just
    launch MAKER via mpiexec (i.e. mpiexec -n 40 maker). MPI support must be
    configured during the MAKER installation process for this to work though
   
 
Options:
 
    -genome|g <file>    Overrides the genome file path in the control files
 
    -RM_off|R          Turns all repeat masking options off.
 
    -datastore/         Forcably turn on/off MAKER's two deep directory
      nodatastore        structure for output. Always on by default.
 
    -old_struct        Use the old directory styles (MAKER 2.26 and lower)
 
    -base    <string>  Set the base name MAKER uses to save output files.
                        MAKER uses the input genome file name by default.
 
    -tries|t <integer>  Run contigs up to the specified number of tries.
 
    -cpus|c  <integer>  Tells how many cpus to use for BLAST analysis.
                        Note: this is for BLAST and not for MPI!
 
    -force|f            Forces MAKER to delete old files before running again.
                        This will require all blast analyses to be rerun.
 
    -again|a            recaculate all annotations and output files even if no
                        settings have changed. Does not delete old analyses.
 
    -quiet|q            Regular quiet. Only a handlful of status messages.
 
    -qq                Even more quiet. There are no status messages.
 
    -dsindex            Quickly generate datastore index file. Note that this
                        will not check if run settings have changed on contigs
 
    -nolock            Turn off file locks. May be usful on some file systems,
                        but can cause race conditions if running in parallel.
 
    -TMP                Specify temporary directory to use.
 
    -CTL                Generate empty control files in the current directory.
 
    -OPTS              Generates just the maker_opts.ctl file.
 
    -BOPTS              Generates just the maker_bopts.ctl file.
 
    -EXE                Generates just the maker_exe.ctl file.
 
    -MWAS    <option>  Easy way to control mwas_server for web-based GUI
 
                              options:  STOP
                                        START
                                        RESTART
 
    -version            Prints the MAKER version.
 
    -help|?            Prints this usage statement.
 
</pre>
 
Version 2.31.9_cond


<pre class="gcommand">
<pre class="gcommand">
 
ml Maker/3.01.03
ml Miniconda3/4.4.10
conda activate /usr/local/apps/gb/maker/2.31.9_conda
maker -h
maker -h


MAKER version 2.31.9
MAKER version 3.01.03


Usage:
Usage:
Line 308: Line 179:


     -force|f            Forces MAKER to delete old files before running again.
     -force|f            Forces MAKER to delete old files before running again.
                        This will require all blast analyses to be rerun.
This will require all blast analyses to be rerun.


     -again|a            recaculate all annotations and output files even if no
     -again|a            recaculate all annotations and output files even if no
                        settings have changed. Does not delete old analyses.
settings have changed. Does not delete old analyses.


     -quiet|q            Regular quiet. Only a handlful of status messages.
     -quiet|q            Regular quiet. Only a handlful of status messages.
Line 342: Line 213:


     -help|?            Prints this usage statement.
     -help|?            Prints this usage statement.
</pre>
</pre>


Line 348: Line 218:
=== Installation ===
=== Installation ===
   
   
Conda
Source download from http://www.yandell-lab.org/
   
   
=== System ===
=== System ===
64-bit Linux
64-bit Linux

Revision as of 16:59, 8 September 2020

Category

Bioinformatics

Program On

Sap2test

Version

3.01.03

Author / Distributor

Yandell-lab

Description

More details from http://gmod.org/wiki/MAKER_Tutorial#What_does_MAKER_do.3F

MAKER is an easy-to-use genome annotation pipeline designed to be usable by small research groups with little bioinformatics experience; however, MAKER is also designed to be scalable and is appropriate for projects of any size even including use by large sequence centers. MAKER can be used for de novo annotation of newly sequenced genomes, for updating existing annotations to reflect new evidence, or just to combine annotations, evidence, and quality control statistics for use in other GMOD programs like GBrowse, JBrowse, Chado, and Apollo.

Running Program

Also refer to Running Jobs on Sap2test

Version 3.01.03 is installed at /apps/gb/Maker/3.01.03

Example to prepare the configuration files.

module load Maker/3.01.03
maker -CTL

Command "maker -CTL" will generate the configuration files maker_bopts.ctl, maker_evm.ctl, maker_exe.ctl, maker_opts.ctl.

Maker example can be found at Running_MAKER_with_example_data

Running maker in a MPI job

Here is an example of submission script (sub.sh) for running maker in a MPI job of 60 MPI processes running on 5 nodes:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=maker-mpi
#SBATCH --ntasks=60
#SBATCH --ntasks-per-node=12
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

export OMPI_MCA_mpi_warn_on_fork=0
unset SLURM_EXPORT_ENV

cd $SLURM_SUBMIT_DIR

ml Maker/3.01.03

mpirun -n 60 maker

Running maker in a memory-sharing threaded job

Here is an example of submission script (sub.sh) for running maker in a threaded job of 16 threads running on 1 node:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=20G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

cd $SLURM_SUBMIT_DIR

ml Maker/3.01.03

maker -cpu 16

Running maker in a single-core job

Here is an example of submission script (sub.sh) for running maker in a single-core job:

#!/bin/bash
#SBATCH --partition=batch
#SBATCH --job-name=Maker
#SBATCH --ntasks=1
#SBATCH --mem=10G
#SBATCH --time=8:00:00
#SBATCH --output=log.o%j
#SBATCH --export=NONE

cd $SLURM_SUBMIT_DIR

ml Maker/3.01.03

maker

Then submit the job by:

qsub ./sub.sh


Documentation

Maker wiki site is at https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/Main_Page

ml Maker/3.01.03 
maker -h

MAKER version 3.01.03

Usage:

     maker [options] <maker_opts> <maker_bopts> <maker_exe>


Description:

     MAKER is a program that produces gene annotations in GFF3 format using
     evidence such as EST alignments and protein homology. MAKER can be used to
     produce gene annotations for new genomes as well as update annotations
     from existing genome databases.

     The three input arguments are control files that specify how MAKER should
     behave. All options for MAKER should be set in the control files, but a
     few can also be set on the command line. Command line options provide a
     convenient machanism to override commonly altered control file values.
     MAKER will automatically search for the control files in the current
     working directory if they are not specified on the command line.

     Input files listed in the control options files must be in fasta format
     unless otherwise specified. Please see MAKER documentation to learn more
     about control file  configuration.  MAKER will automatically try and
     locate the user control files in the current working directory if these
     arguments are not supplied when initializing MAKER.

     It is important to note that MAKER does not try and recalculated data that
     it has already calculated.  For example, if you run an analysis twice on
     the same dataset you will notice that MAKER does not rerun any of the
     BLAST analyses, but instead uses the blast analyses stored from the
     previous run. To force MAKER to rerun all analyses, use the -f flag.

     MAKER also supports parallelization via MPI on computer clusters. Just
     launch MAKER via mpiexec (i.e. mpiexec -n 40 maker). MPI support must be
     configured during the MAKER installation process for this to work though
     

Options:

     -genome|g <file>    Overrides the genome file path in the control files

     -RM_off|R           Turns all repeat masking options off.

     -datastore/         Forcably turn on/off MAKER's two deep directory
      nodatastore        structure for output.  Always on by default.

     -old_struct         Use the old directory styles (MAKER 2.26 and lower)

     -base    <string>   Set the base name MAKER uses to save output files.
                         MAKER uses the input genome file name by default.

     -tries|t <integer>  Run contigs up to the specified number of tries.

     -cpus|c  <integer>  Tells how many cpus to use for BLAST analysis.
                         Note: this is for BLAST and not for MPI!

     -force|f            Forces MAKER to delete old files before running again.
			 This will require all blast analyses to be rerun.

     -again|a            recaculate all annotations and output files even if no
			 settings have changed. Does not delete old analyses.

     -quiet|q            Regular quiet. Only a handlful of status messages.

     -qq                 Even more quiet. There are no status messages.

     -dsindex            Quickly generate datastore index file. Note that this
                         will not check if run settings have changed on contigs

     -nolock             Turn off file locks. May be usful on some file systems,
                         but can cause race conditions if running in parallel.

     -TMP                Specify temporary directory to use.

     -CTL                Generate empty control files in the current directory.

     -OPTS               Generates just the maker_opts.ctl file.

     -BOPTS              Generates just the maker_bopts.ctl file.

     -EXE                Generates just the maker_exe.ctl file.

     -MWAS    <option>   Easy way to control mwas_server for web-based GUI

                              options:  STOP
                                        START
                                        RESTART

     -version            Prints the MAKER version.

     -help|?             Prints this usage statement.

Back to Top

Installation

Source download from http://www.yandell-lab.org/

System

64-bit Linux