DMRIharmonization-Sapelo2: Difference between revisions
No edit summary |
No edit summary |
||
(3 intermediate revisions by the same user not shown) | |||
Line 38: | Line 38: | ||
</pre> | </pre> | ||
Below is an example of a job submission script (sub.sh) to run '''harmonization.py''' with 24 parallel processes on a single compute node on the batch | Below is an example of a job submission script (sub.sh) to run '''harmonization.py''' with 24 parallel processes on a single compute node on the batch partition:<syntaxhighlight lang="shell"> | ||
#!/bin/bash | #!/bin/bash | ||
#SBATCH --job-name=test_dMRIharmonization | #SBATCH --job-name=test_dMRIharmonization | ||
Line 62: | Line 62: | ||
harmonization.py --nproc 24 <your other options and arguments> | harmonization.py --nproc 24 <your other options and arguments> | ||
deactivate | |||
</syntaxhighlight>In your actual submission script, please ensure that you request the appropriate computing resources for your job. For example, you can request CPU cores for running parallel processes using the Slurm headers, such as <code>--ntasks=24</code> and <code>--cpus-per-task=1</code>. | </syntaxhighlight>In your actual submission script, please ensure that you request the appropriate computing resources for your job. For example, you can request CPU cores for running parallel processes using the Slurm headers, such as <code>--ntasks=24</code> and <code>--cpus-per-task=1</code>. | ||
Line 69: | Line 71: | ||
* The value for the header <code>--ntasks</code>, e.g., <code>--ntasks=24</code>, should match the number specified for the <code>--nproc</code> option on your command line, i.e., <code>--nproc=24</code>. | * The value for the header <code>--ntasks</code>, e.g., <code>--ntasks=24</code>, should match the number specified for the <code>--nproc</code> option on your command line, i.e., <code>--nproc=24</code>. | ||
* We highly recommend setting <code>--cpus-per-task=1</code> and exporting <code>OMP_NUM_THREADS=1</code> by including <code>export OMP_NUM_THREADS=1</code>in your job submission script. | * We highly recommend setting <code>--cpus-per-task=1</code> and exporting <code>OMP_NUM_THREADS=1</code> by including <code>export OMP_NUM_THREADS=1</code>in your job submission script. | ||
Here is an example of job submission command: | Here is an example of job submission command: | ||
Line 130: | Line 133: | ||
=== Installation === | === Installation === | ||
Source code is download from https:// | Source code is download from https://github.com/pnlbwh/dMRIharmonization | ||
=== System === | === System === | ||
64-bit Linux | 64-bit Linux |
Latest revision as of 09:01, 17 September 2024
Category
Engineering
Program On
Sapelo2
Version
20240227
Author / Distributor
See https://github.com/pnlbwh/dMRIharmonization
Description
"dMRIharmonization repository is developed by Tashrif Billah, Sylvain Bouix, Suheyla Cetin Karayumak, and Yogesh Rathi, Brigham and Women's Hospital (Harvard Medical School)." More details are at https://github.com/pnlbwh/dMRIharmonization.
Running Program
- Version 20240227 is installed as a Python virtual environment on Sapelo2 at /apps/gb/dMRIharmonization/20240227
To use it, please load the module and activate its env with:
ml dMRIharmonization/20240227 source ${EBROOTDMRIHARMONIZATION}/harmonization/bin/activate source ${EBROOTDMRIHARMONIZATION}/../env.sh
To deactivate its env, please do:
deactivate
Below is an example of a job submission script (sub.sh) to run harmonization.py with 24 parallel processes on a single compute node on the batch partition:
#!/bin/bash
#SBATCH --job-name=test_dMRIharmonization
#SBATCH --partition=batch
#SBATCH --nodes=1
#SBATCH --ntasks=24
#SBATCH --cpus-per-task=1
#SBATCH --mem=100G
#SBATCH --time=7-00:00:00
#SBATCH --constraint="Genoa|Milan"
#SBATCH --mail-type=ALL
#SBATCH --mail-user=<yourMyID>@uga.edu
cd $SLURM_SUBMIT_DIR
ml purge
ml dMRIharmonization/20240227
source ${EBROOTDMRIHARMONIZATION}/harmonization/bin/activate
source ${EBROOTDMRIHARMONIZATION}/../env.sh
export OMP_NUM_THREADS=1
harmonization.py --nproc 24 <your other options and arguments>
deactivate
In your actual submission script, please ensure that you request the appropriate computing resources for your job. For example, you can request CPU cores for running parallel processes using the Slurm headers, such as --ntasks=24
and --cpus-per-task=1
.
Please note:
- Use the header
--constraint="Genoa|Milan"
in your job submission script for optimal job performance. - The value for the header
--ntasks
, e.g.,--ntasks=24
, should match the number specified for the--nproc
option on your command line, i.e.,--nproc=24
. - We highly recommend setting
--cpus-per-task=1
and exportingOMP_NUM_THREADS=1
by includingexport OMP_NUM_THREADS=1
in your job submission script.
Here is an example of job submission command:
sbatch ./sub.sh
Documentation
ml dMRIharmonization/20240227 source ${EBROOTDMRIHARMONIZATION}/harmonization/bin/activate source ${EBROOTDMRIHARMONIZATION}/../env.sh harmonization.py -h =============================================================================== dMRIharmonization (2018) pipeline is written by- TASHRIF BILLAH Brigham and Women's Hospital/Harvard Medical School tbillah@bwh.harvard.edu, tashrifbillah@gmail.com =============================================================================== See details at https://github.com/pnlbwh/dMRIharmonization Submit issues at https://github.com/pnlbwh/dMRIharmonization/issues View LICENSE at https://github.com/pnlbwh/dMRIharmonization/blob/master/LICENSE =============================================================================== Template creation, harmonization, and debugging Usage: harmonization.py [SWITCHES] Meta-switches: -h, --help Prints this help message and quits --help-all Prints help messages of all sub-commands and quits -v, --version Prints the program's version and quits Switches: --bvalMap VALUE:str specify a bmax to scale bvalues into --create turn on this flag to create template --debug turn on this flag to debug harmonized data (valid only with --process) --denoise turn on this flag to denoise voxel data --force turn on this flag to overwrite existing data --harm_list VALUE:ExistingFile harmonized csv/txt file with first column for dwi and 2nd column for mask: dwi1,mask1\n dwi2,mask2\n... --nproc VALUE:str number of processes/threads to use (-1 for all available, may slow down your system); the default is 4 --nshm VALUE:str spherical harmonic order; the default is -1 --nzero VALUE:str number of zero padding for denoising skull region during signal reconstruction; the default is 10 --process turn on this flag to harmonize --ref_list VALUE:ExistingFile reference csv/txt file with first column for dwi and 2nd column for mask: dwi1,mask1\n dwi2,mask2\n... --ref_name VALUE:str reference site name --resample VALUE:str voxel size MxNxO to resample into --stats print statistics of all sites, useful for recomputing --debug statistics separately --tar_list VALUE:ExistingFile target csv/txt file with first column for dwi and 2nd column for mask: dwi1,mask1\n dwi2,mask2\n...; required --tar_name VALUE:str target site name; required --template VALUE:str template directory; required --travelHeads travelling heads
Installation
Source code is download from https://github.com/pnlbwh/dMRIharmonization
System
64-bit Linux