Flankophile-Sapelo2

Category

Pipeline

Program On

Bitbucket

Version

0.2.10

Author / Distributor

Genomic Epidemiology

Description

"Flankophile is a bioinformatics pipeline for gene synteny analysis." Bitbucket

Running Program

Please also refer to Running Jobs on Sapelo2.

Requirements

Flankophile requires the following:

1. The Flankophile pipeline
   1. Contains the pipeline's files and all required conda environments.
2. Input sequence data in FASTA (.fa) format
   1. One or more FASTA files that Flankophile should use as input.
3. A tab-separated value (.tsv) metadata file
   1. Describes the assembly names, their filepaths, and the genus for each input FASTA file.
4. An analysis configuration file in YAML (.yaml)
   1. Describes the analysis for Flankophile to perform and the parameters to use during analysis.
   2. Should be located at the root of the Flankophile repository directory.

Setup

The steps to setup Flankophile are as follows:

1. Enter into an interactive session The installation of Flankophile involves an amount of resource utilization that should be performed by a compute node and not a submit node.
   1. interact --cpus-per-task 16 --mem 32GB
2. Change the working directory to your user's /scratch directory Like other pipelines, the Flankophile installation directory should be in a location suited for large amounts of sustained I/O. Do not install Flankophile in the /home directory; the filesystem that hosts /home is not suited for this workload.
   1. cd /scratch/$USER
3. Clone the Flankophile repository at the desired commit The versions of Flankophile are not labeled with tags in the project's Bitbucket page. Instead, the version is stated in the commit message; this guide will use the commit corresponding to version 0.2.10 (34f286e).
   1. git clone https://bitbucket.org/genomicepidemiology/flankophile/src/34f286e00088d12019dc30cb2c38c103f0efe506/ flankophile
      1. The ...0efe506/ flankophile at the end instructs git to clone the repository into the directory, flankophile.
4. Change the working directory to the newly cloned flankophile/ repository
   1. cd flankophile
5. Load the Snakemake module.
   1. module load snakemake/6.9.1-Mamba-4.11.0-4
6. Create the conda environments that Flankophile will use at runtime In some tests, creating the conda environments at runtime causes the pipeline to fail due to missing or incompatible dependencies. To avoid this, create the conda environments ahead of time. This step will take ~15-25 minutes to complete.
   1. snakemake --use-conda --conda-create-envs-only --cores 16

Example Files

As listed above, Flankophile requires input files, a metadata file, and a config file to run.

Example Input Files

Unfortunately, the Flankophile pipeline does not come with any input files to use as examples.

Example .tsv Metadata File

The following text is an example of a metadata file's contents:

/scratch/$USER/flankophile/metadata.tsv

#assembly_name    path    metadata
assembly32    /scratch/$USER/e_coli-assembly32.fa    Escherichia
assembly25    /scratch/$USER/p_vulgaris-assembly25.fa    Proteus

When creating a metadata.tsv file for your own data, be sure to take note of its filepath. This will be listed after input_list: in the config.yaml file, shown below.

Example .yaml Configuration File

The following text is an example of a Flankophile configuration file's contents:

/scratch/$USER/flankophile/config.yaml

# FLANKOPHILE version 0.2.10
# Alix Vincent Thorn


## 1 #######################################################################################

database: "input/example_input_files/ResFinder_08_02_2022.fa"

input_list: "/scratch/$USER/flankophile/metadata.tsv"     # Edit this!

min_coverage_abricate: "98"                   # Minimum coverage in percentage compared to reference sequence.

min_identity_abricate: "98"                   # Minimum percentage identity compared to reference sequence.

## 2 ######################################################################################

flank_length_upstreams: "1500"

flank_length_downstreams: "1500"

## 3 #######################################################################################

cluster_identity_and_length_diff: "0.98"      # CD-HIT parameters = c and s.  0.98 = cluster at 98 % identity and a maximum length difference of 98 %

## 4  ######################################################################################

k-mer_size: "16"                              # Kmersize used by kma index. Try with "16" if in doubt.

distance_measure: "1"                         # Choose measurement for making distance matrix



# Distance calculation methods:
#
#        1      k-mer hamming distance
#       64      Jaccard distance
#      256      Cosine distance
#     4096      Chi-square distance

This configuration file contains the path to the metadata file, which contains the paths to the input data.

When creating a config.yaml file for your own data, be sure to include the location of the metadata.tsv file you created.

Starting the Pipeline

Once the required files are in place, using either the above examples or your own inputs, the pipeline can be started in either an interactive session or within a job submission script.

In an Interactive Session

The procedure to start the Flankophile pipeline in an interactive session is as follows:

1. Enter into an interactive session if not already inside one.
   1. interact --cpus-per-task 16 --mem 32GB
2. Change the working directory to the location of the flankophile/ directory. (The following command assumes it is in the base of your /scratch directory)
   1. cd /scratch/$USER/flankophile/
3. Load the snakemake module into your environment. (Flankophile requires snakemake 6.9.1 with Miniconda/Mamba 4.11.0)
   1. module load snakemake/6.9.1-Mamba-4.11.0-4
4. Identify the name of the Conda environment to activate.
   1. ls .snakemake/conda/
   2. This will be a random series of numbers and letters.
5. Activate the Conda environment that Flankophile will use.
   1. source activate /scratch/$USER/flankophile/.snakemake/conda/50e62e607e6a24bb70ce9a5a50888445
   2. Make sure to adjust the command according to the name of the environment, identified previously.
6. Run the pipeline.
   1. snakemake --use-conda --cores 16

In a Job Submission Script

The following is an example of a job submission script that runs the Flankophile pipeline:

#!/bin/bash
#SBATCH --job-name=Flankophile
#SBATCH --partition=batch
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=32gb
#SBATCH --time=01:00:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err

module load snakemake/6.9.1-Mamba-4.11.0-4

# The following assumes flankophile/ was copied into the base of your user's /scratch 
cd /scratch/$USER/flankophile/
source activate /source/$USER/flankophile/.snakemake/conda/50e62e607e6a24bb70ce9a5a50888445

snakemake --use-conda --cores 16

Please note that you need to identify the correct name of the Conda environment just as you would if running the pipeline from within an interactive session.

Assuming flankophile/ was cloned into your user's /scratch directory, use the following steps to identify the name of the Conda environment:

1. Identify the name of the Conda environment to activate.
   1. ls /scratch/$USER/flankophile/.snakemake/conda/

Make sure to adjust the job submission script so that the correct Conda environment is loaded.

Installation

• Version 0.2.10: Install in your user's scratch directory.

System

• 64-bit Linux