Flankophile-Sapelo2

Category

Pipeline

Program On

Bitbucket

Version

0.2.10

Author / Distributor

Genomic Epidemiology

Running Program

Please also refer to Running Jobs on Sapelo2.

Requirements

Flankophile requires the following:

1. The Flankophile pipeline
   1. Contains the pipeline's files and all required conda environments.
2. Input sequence data in FASTA (.fa) format
   1. One or more FASTA files that Flankophile should use as input.
3. A tab-separated value (.tsv) metadata file
   1. Describes the assembly names, their filepaths, and the genus for each input FASTA file.
4. An analysis configuration file in YAML (.yaml)
   1. Describes the analysis for Flankophile to perform and the parameters to use during analysis.
   2. Should be located at the root of the Flankophile repository directory.

Setup

The steps to setup Flankophile are as follows:

1. Enter into an interactive session The installation of Flankophile involves an amount of resource utilization that should be performed by a compute node and not a submit node.
   1. interact --cpus-per-task 16 --mem 32GB
2. Change the working directory to your user's /scratch directory Like other pipelines, the Flankophile installation directory should be in a location suited for large amounts of sustained I/O. Do not install Flankophile in the /home directory; the filesystem that hosts /home is not suited for this workload.
   1. cd /scratch/$USER
3. Clone the Flankophile repository at the desired commit The versions of Flankophile are not labeled with tags in the project's Bitbucket page. Instead, the version is stated in the commit message; this guide will use the commit corresponding to version 0.2.10 (34f286e).
   1. git clone https://bitbucket.org/genomicepidemiology/flankophile/src/34f286e00088d12019dc30cb2c38c103f0efe506/ flankophile
      1. The ...0efe506/ flankophile at the end instructs git to clone the repository into the directory, flankophile.
4. Change the working directory to the newly cloned flankophile/ repository
   1. cd flankophile
5. Load the Snakemake module.
   1. module load snakemake/6.9.1-Mamba-4.11.0-4
6. Create the conda environments that Flankophile will use at runtime In some tests, creating the conda environments at runtime causes the pipeline to fail due to missing or incompatible dependencies. To avoid this, create the conda environments ahead of time. This step will take ~15-25 minutes to complete.
   1. snakemake --use-conda --conda-create-envs-only --cores 16

Example Files

As listed above, Flankophile requires input files, a metadata file, and a config file to run.

Example Input Files

Unfortunately, the Flankophile pipeline does not come with any input files to use as examples.

Example .tsv Metadata File

The following text is an example of a metadata file's contents:

/scratch/$USER/flankophile/metadata.tsv

#assembly_name    path    metadata
assembly32    /scratch/$USER/e_coli-assembly32.fa    Escherichia
assembly25    /scratch/$USER/p_vulgaris-assembly25.fa    Proteus

When creating a metadata.tsv file for your own data, be sure to take note of its filepath. This will be listed after input_list: in the config.yaml file, shown below.

Example .yaml Configuration File

The following text is an example of a Flankophile configuration file's contents:

/scratch/$USER/flankophile/config.yaml

# FLANKOPHILE version 0.2.10
# Alix Vincent Thorn


## 1 #######################################################################################

database: "input/example_input_files/ResFinder_08_02_2022.fa"

input_list: "/scratch/$USER/flankophile/metadata.tsv"     # Edit this!

min_coverage_abricate: "98"                   # Minimum coverage in percentage compared to reference sequence.

min_identity_abricate: "98"                   # Minimum percentage identity compared to reference sequence.

## 2 ######################################################################################

flank_length_upstreams: "1500"

flank_length_downstreams: "1500"

## 3 #######################################################################################

cluster_identity_and_length_diff: "0.98"      # CD-HIT parameters = c and s.  0.98 = cluster at 98 % identity and a maximum length difference of 98 %

## 4  ######################################################################################

k-mer_size: "16"                              # Kmersize used by kma index. Try with "16" if in doubt.

distance_measure: "1"                         # Choose measurement for making distance matrix



# Distance calculation methods:
#
#        1      k-mer hamming distance
#       64      Jaccard distance
#      256      Cosine distance
#     4096      Chi-square distance

This configuration file contains the path to the metadata file, which contains the paths to the input data.

When creating a config.yaml file for your own data, be sure to include the location of the metadata.tsv file you created.

Starting the Pipeline

Once the required files are in place, using either the above examples or your own inputs, the pipeline can be started in either an interactive session or within a job submission script.

In an Interactive Session

The procedure to start the Flankophile pipeline in an interactive session is as follows:

1. Enter into an interactive session if not already inside one.
   1. interact --cpus-per-task 16 --mem 32GB
2. Change the working directory to the location of the flankophile/ directory. (The following command assumes it is in the base of your /scratch directory)
   1. cd /scratch/$USER/flankophile/
3. Load the snakemake module into your environment. (Flankophile requires snakemake 6.9.1 with Miniconda/Mamba 4.11.0)
   1. module load snakemake/6.9.1-Mamba-4.11.0-4
4. Identify the name of the Conda environment to activate.
   1. ls .snakemake/conda/
   2. This will be a random series of numbers and letters.
5. Activate the Conda environment that Flankophile will use.
   1. source activate /scratch/$USER/flankophile/.snakemake/conda/50e62e607e6a24bb70ce9a5a50888445
   2. Make sure to adjust the command according to the name of the environment, identified previously.
6. Run the pipeline.
   1. snakemake --use-conda --cores 16

In a Job Submission Script

The following is an example of a job submission script that runs the Flankophile pipeline:

#!/bin/bash
#SBATCH --job-name=Flankophile
#SBATCH --partition=batch
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=32gb
#SBATCH --time=01:00:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err

module load snakemake/6.9.1-Mamba-4.11.0-4

# The following assumes flankophile/ was copied into the base of your user's /scratch 
cd /scratch/$USER/flankophile/
source activate /source/$USER/flankophile/.snakemake/conda/50e62e607e6a24bb70ce9a5a50888445

snakemake --use-conda --cores 16

Please note that you need to identify the correct name of the Conda environment just as you would if running the pipeline from within an interactive session.

Assuming flankophile/ was cloned into your user's /scratch directory, use the following steps to identify the name of the Conda environment:

1. Identify the name of the Conda environment to activate.
   1. ls /scratch/$USER/flankophile/.snakemake/conda/

Make sure to adjust the job submission script so that the correct Conda environment is loaded.

Running Pipeline Using /lscratch

To learn more about lscratch usage, please refer to the sections from our wiki, How to run a job using the local scratch /lscratch on a compute node and lscratch file system.

Below is an outline describing how Flankophile can be run using a compute node's /lscratch space. It assumes the user has a directory named /home/$USER/flankophile/ that contains config.yaml, metadata.csv, and the input files. Keeping in line with the previous examples, the names of the input files would then be p_vulgaris-assembly25.fa and e_coli-assembly32.fa. The flankophile/ directory in the user's home would contain these input files as well as the job submission script, flankophile-sub.sh.

List of Files for /lscratch Usage:

The following is a list of the files and their expected locations for this guide:

1. /home/$USER/flankophile/config.yaml
2. /home/$USER/flankophile/metadata.tsv
3. /home/$USER/flankophile/p_vulgaris-assembly25.fa
4. /home/$USER/flankophile/e_coli-assembly32.fa
5. /home/$USER/flankophile/flankophile-sub.sh

/home/$USER/flankophile/config.yaml

This configuration file differs from the above Example .yaml Configuration File by the specified path to the metadata.tsv file. Instead of listing the path to the metadata.tsv file as a subdirectory of the user's /scratch directory, this configuration file should list the path to the metadata.tsv file as a subdirectory of the user's /home directory. It is acceptable to store the metadata.tsv and input files in the /home directory because Flankophile will only read from these files during its execution; any heavy writing will occur in the /lscratch directory where the pipeline and its dependencies will be installed (see Home_file_system for further information on proper usage of /home storage).

Make sure to replace any instances of $USER with your actual username:

# FLANKOPHILE version 0.2.10
# Alix Vincent Thorn


## 1 #######################################################################################

database: "input/example_input_files/ResFinder_08_02_2022.fa"

input_list: "/home/$USER/flankophile/metadata.tsv"     # Edit this!

min_coverage_abricate: "98"                   # Minimum coverage in percentage compared to reference sequence.

min_identity_abricate: "98"                   # Minimum percentage identity compared to reference sequence.

## 2 ######################################################################################

flank_length_upstreams: "1500"

flank_length_downstreams: "1500"

## 3 #######################################################################################

cluster_identity_and_length_diff: "0.98"      # CD-HIT parameters = c and s.  0.98 = cluster at 98 % identity and a maximum length difference of 98 %

## 4  ######################################################################################

k-mer_size: "16"                              # Kmersize used by kma index. Try with "16" if in doubt.

distance_measure: "1"                         # Choose measurement for making distance matrix



# Distance calculation methods:
#
#        1      k-mer hamming distance
#       64      Jaccard distance
#      256      Cosine distance
#     4096      Chi-square distance

/home/$USER/flankophile/metadata.tsv

Similar to the config.yaml file, this metadata.tsv file differs from the above Example .tsv Metadata File by the specified paths it contains. Note also that the paths it contains are in a subdirectory of the /home directory, flankophile/, instead of in the /home directory directly.

Make sure to replace any instances of $USER with your actual username, and to specify the paths to your actual data:

#assembly_name    path    metadata
assembly32    /home/$USER/flankophile/e_coli-assembly32.fa    Escherichia
assembly25    /home/$USER/flankophile/p_vulgaris-assembly25.fa    Proteus

/home/$USER/flankophile/flankophile-sub.sh

The major differences between running Flankophile using the /lscratch space and running Flankophile elsewhere is that the working directory inside /lscratch must be created before execution and removed after execution. These steps are handled by the following job submission script:

#!/usr/bin/env bash
#SBATCH --job-name=Flankophile
#SBATCH --partition=batch
#SBATCH --nodes=1
#SBATCH --gres=lscratch:30
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=32G
#SBATCH --time=05:00:00
#SBATCH --output=log.%j.out
#SBATCH --error=log.%j.err

cd $SLURM_SUBMIT_DIR

# Step 1 - Create working directory in /lscratch
mkdir -p /lscratch/${USER}/${SLURM_JOB_ID}

# Step 2 - cd to /lscratch working directory
cd /lscratch/${USER}/${SLURM_JOB_ID}

# Step 3 - Clone and cd into pipeline
git clone https://bitbucket.org/genomicepidemiology/flankophile/src/34f286e00088d12019dc30cb2c38c103f0efe506/ flankophile
cd flankophile

# Step 4 - Load snakemake then build required environments
module load snakemake/6.9.1-Mamba-4.11.0-4
snakemake --use-conda --conda-create-envs-only --cores $SLURM_CPUS_PER_TASK

# Step 5 - Copy config.yaml into /lscratch working directory
cp ${SLURM_SUBMIT_DIR}/config.yaml ./

# Step 6 - Activate newly created environment
source activate $(ls -d .snakemake/conda/*/)

# Step 7 - Run the pipeline
snakemake --use-conda --cores $SLURM_CPUS_PER_TASK

# Step 8 - Copy the output directory into submit directory
cp -r output/ ${SLURM_SUBMIT_DIR}/

# Step 9 - Clean up by removing the /lscratch working directory
rm -rf /lscratch/${USER}/${SLURM_JOB_ID}

Step-by-Step Breakdown of flankophile-sub.sh

1. Step 1 - Create working directory in /lscratch

   mkdir -p /lscratch/${USER}/${SLURM_JOB_ID}
   

   1. The above command creates a directory for the job to use.
2. Step 2 - cd to /lscratch working directory

   cd /lscratch/${USER}/${SLURM_JOB_ID}
   

   1. The above command changes the current working directory to the directory created in the previous step.
3. Step 3 - Clone and cd into pipeline

   git clone https://bitbucket.org/genomicepidemiology/flankophile/src/34f286e00088d12019dc30cb2c38c103f0efe506/ flankophile
   cd flankophile
   

   1. The first of the above commands clones the Flankophile repository into a directory named flankophile/.
      1. The full path to the job's current directory would be /lscratch/${USER}/${SLURM_JOB_ID}/flankophile.
   2. The second of the above commands changes the current working directory into the repository.
4. Step 4 - Load snakemake then build the required environments

   module load snakemake/6.9.1-Mamba-4.11.0-4
   snakemake --use-conda --conda-create-envs-only --cores $SLURM_CPUS_PER_TASK
   

   1. The first of the above commands loads the correct version of Snakemake that is required by Flankophile.
   2. The second of the above commands creates the Conda environments that Flankophile will use at runtime.
      1. The $SLURM_CPUS_PER_TASK variable is inhereted from the job's environment and is equal to the value set for the --cpus-per-task variable in the job submission script's header.
5. Step 5 - Copy config.yaml into /lscratch working directory

   cp ${SLURM_SUBMIT_DIR}/config.yaml ./
   

   1. The above command copies the config.yaml file into the job's current working directory, /lscratch/${USER}/${SLURM_JOB_ID}/flankophile.
   2. Because the command expects the config.yaml file to exist in the directory from which the job was submitted, No such file or directory errors can occur if the job is submitted from elsewhere.
      1. In other words, make sure to cd into /home/$USER/flankophile before executing sbatch flankophile-sub.sh.
6. Step 6 - Activate newly created environment

   source activate $(ls -d .snakemake/conda/*/)
   

   1. The above command activates the Conda environment created in Step 4.
      1. Because the Flankophile pipeline is created at the beginning of the job, and then deleted at the end, the subcommand, $(ls -d .snakemake/conda/*/), will return the correct name of the Conda environment.
7. Step 7 - Run the pipeline

   snakemake --use-conda --cores $SLURM_CPUS_PER_TASK
   

   1. The above command starts the Flankophile pipeline while specifying the number of cores to use with the $SLURM_CPUS_PER_TASK environment variable.
8. Step 8 - Copy the output directory into submit directory

   cp -r output/ ${SLURM_SUBMIT_DIR}/
   

   1. The above command copies the output/ directory and its contents recursively into the directory from which the job was submitted.
      1. This would be equal to cp -r /lscratch/${USER}/${SLURM_JOB_ID}/flankophile/output/ /home/$USER/flankophile.
9. Step 9 - Clean up by removing the /lscratch working directory

   rm -rf /lscratch/${USER}/${SLURM_JOB_ID}
   

   1. The above command prevents files no longer in use from consuming space on /lscratch by explicitly removing the job's working directory as a last step.

Submitting the /lscratch Job

Be sure to cd into the flankophile/ directory prior to submitting the job.

1. cd /home/$USER/flankophile
   

2. sbatch flankophile-sub.sh
   

Installation

• Version 0.2.10: Install in your user's scratch directory.

System

• 64-bit Linux