Cromwell-Sapelo2 - Revision history

Shtsai at 00:27, 5 April 2026

2026-04-05T00:27:13Z

Isaiah: Formatting

2024-02-27T15:55:06Z

Formatting

← Older revision		Revision as of 11:55, 27 February 2024
Line 153:		Line 153:
	"CromwellBowtie2.cpus_per_task": "8"		"CromwellBowtie2.cpus_per_task": "8"
	}		}
	</syntaxhighlight><code>--inputs input.json</code>		</syntaxhighlight>'''<u>Usage:</u>''' <code>--inputs input.json</code>

	In an input file, variables are referenced using the '''workflow name''' followed by a '''period''' and the '''variable name''' <code>CromwellBowtie2.input_fq</code>.		In an input file, variables are referenced using the '''workflow name''' followed by a '''period''' and the '''variable name''' <code>CromwellBowtie2.input_fq</code>.
Line 174:		Line 174:
	"use_relative_output_paths": true		"use_relative_output_paths": true
	}		}
	</syntaxhighlight><code>--options options.json</code>		</syntaxhighlight>'''<u>Usage:</u>''' <code>--options options.json</code>

	Without specifying an alternative output directory, the output would be in a location similar to the following:		Without specifying an alternative output directory, the output would be in a location similar to the following:

	<code>./cromwell-executions/CromwellBowtie2/04d44744-2a84-4b2f-bea6-492985543ace/call-Bowtie2/execution/</code>		* <code>./cromwell-executions/CromwellBowtie2/04d44744-2a84-4b2f-bea6-492985543ace/call-Bowtie2/execution/</code>

	Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory during the execution of the task, ''<code>Bowtie2</code>''.		Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory during the execution of the task, ''<code>Bowtie2</code>''.

Isaiah: Added "workflow CromwellBowtie2" to WDL segment in "inputs" section for added clarity

2024-02-27T15:50:44Z

Added "workflow CromwellBowtie2" to WDL segment in "inputs" section for added clarity

← Older revision		Revision as of 11:50, 27 February 2024
Line 138:		Line 138:
	In Cromwell, ''Workflow Input Files'' are written in JSON. They are specified with the <code>''--inputs''</code> flag when Cromwell is executed at the [https://cromwell.readthedocs.io/en/stable/CommandLine/#run command line]. These files define the requirements of the workflow, such as input files, or other input values. Specifying these input values in a separate file prevents the need to hardcode inputs in the original workflow file.		In Cromwell, ''Workflow Input Files'' are written in JSON. They are specified with the <code>''--inputs''</code> flag when Cromwell is executed at the [https://cromwell.readthedocs.io/en/stable/CommandLine/#run command line]. These files define the requirements of the workflow, such as input files, or other input values. Specifying these input values in a separate file prevents the need to hardcode inputs in the original workflow file.

	~~Continuing with~~ the above Example WDL File, the <code>CromwellBowtie2</code> workflow utilizes the following values:<syntaxhighlight>		From the above [[Cromwell-Sapelo2#Example WDL .28Workflow Description Language.29 File\|Example WDL File]], the <code>CromwellBowtie2</code> workflow utilizes the following values:<syntaxhighlight>
			workflow CromwellBowtie2 {
	File input_fq		File input_fq
	File index_dir		File index_dir
	String index_name		String index_name
	Int ~~threads~~		Int cpus_per_task
	</syntaxhighlight>The following JSON file provides definitions for each of these values, and can be found at <code>/usr/local/training/Cromwell/inputs.json</code>:		</syntaxhighlight>The following JSON file provides definitions for each of these values, and can be found at <code>/usr/local/training/Cromwell/inputs.json</code>:

Line 154:		Line 155:
	</syntaxhighlight><code>--inputs input.json</code>		</syntaxhighlight><code>--inputs input.json</code>

	In an input file, variables are referenced using the workflow name followed by a period and the variable name <code>CromwellBowtie2.input_fq</code>.		In an input file, variables are referenced using the '''workflow name''' followed by a '''period''' and the '''variable name''' <code>CromwellBowtie2.input_fq</code>.

	The example data referenced in this JSON file can be found at the following locations:		The example data referenced in this JSON file can be found at the following locations:

Isaiah: Some lines continue to overlap despite attempts to fix

2024-02-27T15:47:00Z

Some lines continue to overlap despite attempts to fix

← Older revision		Revision as of 11:47, 27 February 2024
Line 152:		Line 152:
	"CromwellBowtie2.cpus_per_task": "8"		"CromwellBowtie2.cpus_per_task": "8"
	}		}
	</syntaxhighlight><code>--inputs input.json</code>		</syntaxhighlight><code>--inputs input.json</code>

	In ~~the above~~ file, <code>input_fq</code>, <code>index_dir</code>, <code>index_name</code>, and <code>cpus_per_task</code> are all values from the <code>CromwellBowtie2</code> workflow, defined in the example WDL file:<syntaxhighlight>		In an input file, variables are referenced using the workflow name followed by a period and the variable name <code>CromwellBowtie2.input_fq</code>.
	~~workflow CromwellBowtie2 {~~
	~~File input_fq~~
	~~File index_dir~~
	~~String index_name~~
	~~Int cpus_per_task~~
	~~</syntaxhighlight>In input files~~, variables are referenced using the workflow name followed by a period and the variable name <code>CromwellBowtie2.input_fq</code>.

	The example data referenced in this JSON file can be found at the following locations:		The example data referenced in this JSON file can be found at the following locations:

Isaiah: Clarified how to refer to variable names in the inputs.json file

2024-02-27T15:43:39Z

Clarified how to refer to variable names in the inputs.json file

← Older revision		Revision as of 11:43, 27 February 2024
Line 153:		Line 153:
	}		}
	</syntaxhighlight><code>--inputs input.json</code>		</syntaxhighlight><code>--inputs input.json</code>

			In the above file, <code>input_fq</code>, <code>index_dir</code>, <code>index_name</code>, and <code>cpus_per_task</code> are all values from the <code>CromwellBowtie2</code> workflow, defined in the example WDL file:<syntaxhighlight>
			workflow CromwellBowtie2 {
			File input_fq
			File index_dir
			String index_name
			Int cpus_per_task
			</syntaxhighlight>In input files, variables are referenced using the workflow name followed by a period and the variable name <code>CromwellBowtie2.input_fq</code>.

	The example data referenced in this JSON file can be found at the following locations:		The example data referenced in this JSON file can be found at the following locations:

Isaiah: Changed instruction from copying examples into scratch into copying into a subdirectory of scratch

2024-02-27T15:20:05Z

Changed instruction from copying examples into scratch into copying into a subdirectory of scratch

← Older revision		Revision as of 11:20, 27 February 2024
Line 219:		Line 219:
	==== Running the example ====		==== Running the example ====
	To run the above example, navigate to scratch and copy the files into the working directory:<syntaxhighlight lang="bash">		To run the above example, navigate to scratch and copy the files into the working directory:<syntaxhighlight lang="bash">
	cd /scratch/$USER		mkdir /scratch/$USER/cromwell-example
			cd /scratch/$USER/cromwell-example
	cp -r /usr/local/training/Cromwell/* ./		cp -r /usr/local/training/Cromwell/* ./
	</syntaxhighlight>Once copied, the job can be submitted with <code>sbatch</code>:<syntaxhighlight lang="bash">		</syntaxhighlight>Once copied, the job can be submitted with <code>sbatch</code>:<syntaxhighlight lang="bash">

Isaiah: Grammar fix to Example Workflow Options File to provide added clarity

2024-02-27T15:15:04Z

Grammar fix to Example Workflow Options File to provide added clarity

← Older revision		Revision as of 11:15, 27 February 2024
Line 177:		Line 177:
	<code>./cromwell-executions/CromwellBowtie2/04d44744-2a84-4b2f-bea6-492985543ace/call-Bowtie2/execution/</code>		<code>./cromwell-executions/CromwellBowtie2/04d44744-2a84-4b2f-bea6-492985543ace/call-Bowtie2/execution/</code>

	Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory of the task, ''<code>Bowtie2</code>''.		Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory during the execution of the task, ''<code>Bowtie2</code>''.

	===== '''Example Job submission Script''' =====		===== '''Example Job submission Script''' =====

Isaiah: Added link to Job Submission Script to requirements section

2024-02-27T15:08:11Z

Added link to Job Submission Script to requirements section

← Older revision		Revision as of 11:08, 27 February 2024
Line 38:		Line 38:
	# '''<u>[[Cromwell-Sapelo2#Example Workflow Options File\|Options File]]</u>''' (optional)		# '''<u>[[Cromwell-Sapelo2#Example Workflow Options File\|Options File]]</u>''' (optional)
	## Defines any additional options.		## Defines any additional options.
	# '''<u>Job Submission Script</u>''' (required)		# '''<u>[[Cromwell-Sapelo2#Example Job submission Script\|Job Submission Script]]</u>''' (required)

	==== Example Requirements ====		==== Example Requirements ====

Isaiah: Fixed Job submission script subheader

2024-02-27T15:06:43Z

Fixed Job submission script subheader

← Older revision		Revision as of 11:06, 27 February 2024
Line 30:		Line 30:
	To execute Cromwell as a job on Sapelo2, the following are required:		To execute Cromwell as a job on Sapelo2, the following are required:

	# '''<u>Cromwell Configuration File</u>''' (required)		# '''<u>[[Cromwell-Sapelo2#Example Configuration File\|Cromwell Configuration File]]</u>''' (required)
	## Defines how each step in the workflow should be initialized.		## Defines how each step in the workflow should be initialized.
	# '''<u>WDL File</u>''' (required)		# '''<u>[[Cromwell-Sapelo2#Example WDL .28Workflow Description Language.29 File\|WDL File]]</u>''' (required)
	## Defines the workflow itself.		## Defines the workflow itself.
	# '''<u>Inputs File</u>''' (optional but recommended)		# '''<u>[[Cromwell-Sapelo2#Example Workflow Input File\|Inputs File]]</u>''' (optional but recommended)
	## Defines the inputs to the workflow.		## Defines the inputs to the workflow.
	# '''<u>Options File</u>''' (optional)		# '''<u>[[Cromwell-Sapelo2#Example Workflow Options File\|Options File]]</u>''' (optional)
	## Defines any additional options.		## Defines any additional options.
	# '''<u>Job Submission Script</u>''' (required)		# '''<u>Job Submission Script</u>''' (required)
Line 179:		Line 179:
	Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory of the task, ''<code>Bowtie2</code>''.		Where <code>cromwell-executions</code> is a subdirectory of the working directory, the <code>04d44744-2a84-4b2f-bea6-492985543ace</code> directory is named at runtime, and <code>execution</code> was the working directory of the task, ''<code>Bowtie2</code>''.

	'''Example Job submission Script'''		===== '''Example Job submission Script''' =====

	The following is an example job submission script that utilizes the files described above, and can be found at <code>/usr/local/training/Cromwell/cromwell-sub.sh</code>:<syntaxhighlight lang="bash">
			The following is an example job submission script that utilizes the files described above, and can be found at <code>/usr/local/training/Cromwell/cromwell-sub.sh</code>:

			====== <code>cromwell-sub.sh</code> ======
			<syntaxhighlight lang="bash">
	#!/bin/bash		#!/bin/bash
	#SBATCH --job-name=cromwell-bowtie2		#SBATCH --job-name=cromwell-bowtie2

Isaiah: Finished draft 1.0

2024-02-27T15:03:49Z

Finished draft 1.0

Show changes