ICommands-Sapelo2: Difference between revisions

From Research Computing Center Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
 
(11 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:Sapelo2old]][[Category:Software]][[Category:Tools]]   
[[Category:Sapelo2]][[Category:Software]][[Category:Tools]]   


=== Category ===
=== Category ===
Line 11: Line 11:
=== Version ===
=== Version ===
   
   
4.1.12
4.2.8
   
   
=== Author / Distributor ===
=== Author / Distributor ===
Line 19: Line 19:
=== Description ===
=== Description ===


from [https://cyverse.atlassian.net/wiki/spaces/DS/pages/241869855/Using+iCommands iCommands]: "iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store."
From [https://cyverse.atlassian.net/wiki/spaces/DS/pages/241869855/Using+iCommands iCommands]: "iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store."


=== Running Program ===
=== Running Program ===
   
   
Also refer to [[Running Jobs on Sapelo2]].
Also refer to [[Transferring Files]].


*Version 1.0.2, installed in /apps/eb/HpcGridRunner/1.0.2
* version 4.2.8 is installed on Sapelo2 IRODS '''transfer node''' (xfer-irods.gacrc.uga.edu). Please note that this command is not available on the other transfer nodes.
 
Please log on a Sapelo2 transfer node xfer-irods.gacrc.uga.edu to use iCommands, for example:


To use this version of HpcGridRunner, please first load the module with
<pre class="gscript">
<pre class="gscript">
ml HpcGridRunner/1.0.2
ssh username@xfer-irods.gacrc.uga.edu
</pre>  
</pre>


Running a blast job in parallel by splitting the fasta file:
Once you logged on this specific transfer node, you can initialize and start the connection to iRODS by using the iinit command, for example:
You will need to change the YOUR_QUERY_FASTA.fasta to the name of your fasta file. The cmd_template can be change to reflect your blast command. Leave the -query parameter set to __QUERY_FILE__ value and hpc_FASTA_Gridrunner will fill it dynamically.
You can also change the values for outfmt, evalue, max_target_seqs and db. Do not change the num_threads value unless you use a conf file that allocates more than one cpu.
The grid_conf file specified below will request one CPU core and 3GB of RAM.


<pre class="gscript">
<pre class="gscript">
#!/bin/bash
iinit
#SBATCH --partition batch
One or more fields in your iRODS environment file (irods_environment.json) are
#SBATCH --ntasks=1
missing; please enter them.
#SBATCH --time=48:00:00
Enter the host name (DNS) of the server to connect to: data.cyverse.org
#SBATCH --mem=3gb
Enter the port number: 1247
 
Enter your irods user name: [yourIRodsUserName]
ml hpcgridrunner/1.0.2
Enter your irods zone: [yourIRodsZone]
ml BLAST+/2.10.1-gompi-2019b
Enter your current iRODS password: [yourIRodsPassword]
 
hpc_FASTA_GridRunner.pl --grid_conf=$HPCGR_CONF_DIR/sapelo_1c_3G.conf --cmd_template "blastp -query __QUERY_FILE__ -db /db/uniprot/latest/uniprot_sprot  -max_target_seqs 1 -outfmt 6 -evalue 1e-5 -num_threads 1" --seqs_per_bin 250 --query_fasta YOUR_QUERY_FASTA.fasta --out_dir blast_result
 
</pre>
</pre>


The blast results for the each of the split group will be in a separate file under the specified output directory. The file name will end in fa.OUT. To gather the blast output you can generally concatenate all the individual outputs. You can use the following command to find and concatenate all the output files.
Please replace [yourIRodsUserName], [yourIRodsZone], and [yourIRodsPassword] with your own IRods username, zone name, and login password.  


<pre class="gcommand">
If the login succeeds and connection is built, you can use other iCommands, like iget/iput, etc., to download/upload the file/directory between iRODS and your current working directory on the cluster.
find OUTPUT_DIR -name '*.fa.OUT' | xargs cat > combined_blast_results.out
</pre>


=== Documentation ===
=== Documentation ===
   
   
Please see http://hpcgridrunner.github.io/
Please see [https://cyverse.atlassian.net/wiki/spaces/DS/pages/241869855/Using+iCommands iCommands]


[[#top|Back to Top]]
[[#top|Back to Top]]
Line 65: Line 58:
=== Installation ===
=== Installation ===


Installed in /apps/eb/HpcGridRunner/1.0.2
Installed on Sapelo2 transfer node xfer-irods.gacrc.uga.edu


=== System ===
=== System ===


64-bit Linux
64-bit Linux

Latest revision as of 10:08, 12 January 2021


Category

Tools

Program On

Sapelo2

Version

4.2.8

Author / Distributor

Please see iCommands

Description

From iCommands: "iCommands is a collection of commands for Linux and Mac OS operating systems that are used in the iRODS system to interact with the CyVerse Data Store."

Running Program

Also refer to Transferring Files.

  • version 4.2.8 is installed on Sapelo2 IRODS transfer node (xfer-irods.gacrc.uga.edu). Please note that this command is not available on the other transfer nodes.

Please log on a Sapelo2 transfer node xfer-irods.gacrc.uga.edu to use iCommands, for example: 

ssh username@xfer-irods.gacrc.uga.edu

Once you logged on this specific transfer node, you can initialize and start the connection to iRODS by using the iinit command, for example: 

iinit
One or more fields in your iRODS environment file (irods_environment.json) are
missing; please enter them.
Enter the host name (DNS) of the server to connect to: data.cyverse.org
Enter the port number: 1247
Enter your irods user name: [yourIRodsUserName]
Enter your irods zone: [yourIRodsZone]
Enter your current iRODS password: [yourIRodsPassword]

Please replace [yourIRodsUserName], [yourIRodsZone], and [yourIRodsPassword] with your own IRods username, zone name, and login password.

If the login succeeds and connection is built, you can use other iCommands, like iget/iput, etc., to download/upload the file/directory between iRODS and your current working directory on the cluster.

Documentation

Please see iCommands

Back to Top

Installation

Installed on Sapelo2 transfer node xfer-irods.gacrc.uga.edu

System

64-bit Linux

Retrieved from "https://wiki.gacrc.uga.edu/index.php?title=ICommands-Sapelo2&oldid=19776"