Transferring Files - Revision history

Shtsai at 18:17, 20 March 2026

2026-03-20T18:17:43Z

Shtsai: /* The File Transfer node for the teaching cluster (txfer.gacrc.uga.edu) */

2025-01-08T16:26:46Z

The File Transfer node for the teaching cluster (txfer.gacrc.uga.edu)

← Older revision		Revision as of 12:26, 8 January 2025
Line 282:		Line 282:

	Access to txfer.gacrc.uga.edu requires an active MyID and password, and two-factor authentication with Archpass Duo. For more details of Archpass Duo, please refer to		Access to txfer.gacrc.uga.edu requires an active MyID and password, and two-factor authentication with Archpass Duo. For more details of Archpass Duo, please refer to
	[https://eits.uga.edu/access_and_security/infosec/tools/~~archpass_duo~~ archpass_duo] .		[https://eits.uga.edu/access_and_security/infosec/tools/archpass archpass_duo] .

	===Transferring Files from your local storage device to the teaching cluster===		===Transferring Files from your local storage device to the teaching cluster===

Shtsai: /* The File Transfer nodes (xfer.gacrc.uga.edu) */

2025-01-08T16:26:09Z

The File Transfer nodes (xfer.gacrc.uga.edu)

← Older revision		Revision as of 12:26, 8 January 2025
Line 6:		Line 6:

	All Sapelo2 users, as well as all GACRC PIs, can access the xfer nodes using the hostname '''xfer.gacrc.uga.edu''' and using their UGA MyID (not the 810 or 811 number) and the MyID password to authenticate. Two-factor authentication using Archpass Duo is also necessary. For more details of Archpass Duo, please refer to		All Sapelo2 users, as well as all GACRC PIs, can access the xfer nodes using the hostname '''xfer.gacrc.uga.edu''' and using their UGA MyID (not the 810 or 811 number) and the MyID password to authenticate. Two-factor authentication using Archpass Duo is also necessary. For more details of Archpass Duo, please refer to
	[https://eits.uga.edu/access_and_security/infosec/tools/~~archpass_duo~~ archpass_duo].		[https://eits.uga.edu/access_and_security/infosec/tools/archpass archpass_duo].

	The GACRC file transfer nodes (xfer) are configured to facilitate file transfer. These nodes, which have a hostname of '''xfer.gacrc.uga.edu''', are connected via a higher-bandwidth network connection. Hence, maximum transfer speed between a user's local machine and the GACRC systems can be achieved by transferring files to/from the host xfer.gacrc.uga.edu (instead of the login nodes of the clusters).		The GACRC file transfer nodes (xfer) are configured to facilitate file transfer. These nodes, which have a hostname of '''xfer.gacrc.uga.edu''', are connected via a higher-bandwidth network connection. Hence, maximum transfer speed between a user's local machine and the GACRC systems can be achieved by transferring files to/from the host xfer.gacrc.uga.edu (instead of the login nodes of the clusters).
Line 23:		Line 23:

	4. The project file system for the lab: /project/groupname		4. The project file system for the lab: /project/groupname


	===Transferring Files from your local storage device to Sapelo2 or vice-versa===		===Transferring Files from your local storage device to Sapelo2 or vice-versa===

Keeko: added cyverse

2021-12-01T15:25:13Z

added cyverse

← Older revision		Revision as of 11:25, 1 December 2021
Line 435:		Line 435:
	Data transfers happen unattended and are faster than SCP/SFTP, data verification is on by default, and automatic restarts or continuation of transfers happen after a disruption.		Data transfers happen unattended and are faster than SCP/SFTP, data verification is on by default, and automatic restarts or continuation of transfers happen after a disruption.

	To use Globus you will need a Globus Identity. At a minimum you will need to setup your identity using the University of Georgia organizational login in order to access UGA systems. More information about how to get started using Globus can be found on the [[Globus\|'''Globus''']] page.		To use Globus you will need a Globus Identity. At a minimum you will need to setup your identity using the University of Georgia organizational login in order to access UGA systems. More information about how to get started using Globus can be found on the [[Globus\|'''Globus''']] page. You can also get more information about Globus from the [https://docs.globus.org/ Official Globus Documentation]

			=== Using iCommands to access the Cyverse Data Store ===
			[https://cyverse.org/data-store Cyverse] is a secure data storage and management service. iCommands is a collection of commands for Linux and Mac OS operating systems that are used to interact with the CyVerse Data Store. iCommands can be used by CyVerse account users to download files that have been shared by other users and to upload files to the Data Store, as well as add metadata, change permissions, and more. A CyVerse account is not required to download a public data file via iCommands.

			version 4.2.8 of iCommands is installed on Sapelo2 IRODS '''transfer node''' (xfer-irods.gacrc.uga.edu). Please note that this command is not available on the other transfer nodes.

			You will need to log on a Sapelo2 transfer node xfer-irods.gacrc.uga.edu and then initialize the connection to iRODS by using the iinit command to use iCommands

			More information about using iCommands can be found on the [[ICommands-Sapelo2\|'''ICommands-Sapelo2''']] page. You can also read [https://cyverse.atlassian.net/wiki/spaces/DS/pages/241869855/Using+iCommands Official iCommands Documentation]

Keeko at 15:25, 30 November 2021

2021-11-30T15:25:36Z

← Older revision		Revision as of 11:25, 30 November 2021
Line 432:		Line 432:

	=== Globus ===		=== Globus ===
	Globus is a high-performance data-transfer platform that allows you to perform and/or automate data transfers.		Globus is a high-performance data-transfer platform that allows you to perform and/or automate data transfers. These transfers can be to and from other institutions, your local computer, or your GACRC /project area. Collaborators can also share data with you by sharing a Collection or an "endpoint" with you. The shared Collection/Endpoint can be on another institution, on a desktop or laptop, or on their GACRC storage.
	Data transfers happen unattended and are faster than SCP/SFTP, data verification is on by default, and automatic restarts or continuation of transfers happen after a disruption.		Data transfers happen unattended and are faster than SCP/SFTP, data verification is on by default, and automatic restarts or continuation of transfers happen after a disruption.

	To use Globus you will need a Globus Identity. At a minimum you will need to setup your identity using the University of Georgia organizational login in order to access UGA systems. More information about how to get started using Globus can be found on the [[Globus\|'''Globus''']] page.		To use Globus you will need a Globus Identity. At a minimum you will need to setup your identity using the University of Georgia organizational login in order to access UGA systems. More information about how to get started using Globus can be found on the [[Globus\|'''Globus''']] page.

Keeko at 15:19, 30 November 2021

2021-11-30T15:19:23Z

← Older revision		Revision as of 11:19, 30 November 2021
Line 430:		Line 430:
	</pre>		</pre>
	where ''abclab'' needs to be changed to the name of your group.		where ''abclab'' needs to be changed to the name of your group.

			=== Globus ===
			Globus is a high-performance data-transfer platform that allows you to perform and/or automate data transfers.
			Data transfers happen unattended and are faster than SCP/SFTP, data verification is on by default, and automatic restarts or continuation of transfers happen after a disruption.

			To use Globus you will need a Globus Identity. At a minimum you will need to setup your identity using the University of Georgia organizational login in order to access UGA systems. More information about how to get started using Globus can be found on the [[Globus\|'''Globus''']] page.

Keeko: spelling

2021-09-09T14:08:37Z

spelling

← Older revision		Revision as of 10:08, 9 September 2021
Line 64:		Line 64:
	{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"		{\| width="100%" border="1" cellspacing="0" cellpadding="2" align="center" class="wikitable unsortable"
	\|-		\|-
	\| [[ ~~media~~:~~Xfer Filezilla 12032018~~.pdf]]		\| [[Media:Xfer_Filezilla_12032018.pdf]]
	\|-		\|-
	\|}		\|}
Line 88:		Line 88:
	</pre>		</pre>

	If the password authenticates successfully, then another pop-up window will prompt for Duo two-factor authentication. It says, Using Keyboard-interactive authentication and lists the Duo options you can choose from: Duo push, Phone Call, or SMS passcode, etc. Once the connection is established you can upload or download files from Local Machine to Cluster or vice-versa. Note: Under Advanced, ~~kept~~ the "Default" option for Transfer settings Rule. With this default option, you will be able to transfer multiple files or folders during the session.		If the password authenticates successfully, then another pop-up window will prompt for Duo two-factor authentication. It says, Using Keyboard-interactive authentication and lists the Duo options you can choose from: Duo push, Phone Call, or SMS passcode, etc. Once the connection is established you can upload or download files from Local Machine to Cluster or vice-versa. Note: Under Advanced, keep the "Default" option for Transfer settings Rule. With this default option, you will be able to transfer multiple files or folders during the session.

	<!--		<!--

Raj76 at 22:26, 13 July 2021

2021-07-13T22:26:08Z

← Older revision		Revision as of 18:26, 13 July 2021
Line 116:		Line 116:
	where ''filename'' is the name of the file to be transferred, ''yourUGAMyID'' is your UGA MyID, and ''subdirectory'' is the subdirectory in your Sapelo2 home directory to which files are being transferred. You will then be asked to enter your UGA MyID password. You will also be prompted for two-factor authentication with Archpass Duo.		where ''filename'' is the name of the file to be transferred, ''yourUGAMyID'' is your UGA MyID, and ''subdirectory'' is the subdirectory in your Sapelo2 home directory to which files are being transferred. You will then be asked to enter your UGA MyID password. You will also be prompted for two-factor authentication with Archpass Duo.

			To transfer files from Sapelo2 to your local machine, use the full path to your file on Sapelo2 followed by the location you would like to transfer the files locally. The following example transfers a file called "filename" to the local current working directory.

			<pre class="gcommand">
			scp yourUGAMyID@xfer.gacrc.uga.edu:full/path/to/filename ./
			</pre>

	(Note: yourUGAMyID@ can be omitted if your username on Sapelo2 is the same as on your local machine). Multiple files (e.g. file1, file2, and file3) can be transferred with a single command:		(Note: yourUGAMyID@ can be omitted if your username on Sapelo2 is the same as on your local machine). Multiple files (e.g. file1, file2, and file3) can be transferred with a single command:

Shtsai at 17:41, 12 January 2021

2021-01-12T17:41:13Z

← Older revision		Revision as of 13:41, 12 January 2021
Line 10:		Line 10:
	The GACRC file transfer nodes (xfer) are configured to facilitate file transfer. These nodes, which have a hostname of '''xfer.gacrc.uga.edu''', are connected via a higher-bandwidth network connection. Hence, maximum transfer speed between a user's local machine and the GACRC systems can be achieved by transferring files to/from the host xfer.gacrc.uga.edu (instead of the login nodes of the clusters).		The GACRC file transfer nodes (xfer) are configured to facilitate file transfer. These nodes, which have a hostname of '''xfer.gacrc.uga.edu''', are connected via a higher-bandwidth network connection. Hence, maximum transfer speed between a user's local machine and the GACRC systems can be achieved by transferring files to/from the host xfer.gacrc.uga.edu (instead of the login nodes of the clusters).

	It is not necessary to connect to the UGA VPN when connecting to a file transfer node from off-campus. If you are transferring data from/to a server outside of campus (such as ~~downloading data from NCBI into the cluster~~), you will get much faster transfer speeds if you are not connected to the VPN.		It is not necessary to connect to the UGA VPN when connecting to a file transfer node from off-campus. If you are transferring data from/to a server outside of campus (such as your local computer), you will get much faster transfer speeds if you are not connected to the VPN.

	Note that a user's home directory on the xfer nodes is the same as the user's Sapelo2 home directory, which is '''not''' the same as the user's home directory on the teaching cluster.		Note that a user's home directory on the xfer nodes is the same as the user's Sapelo2 home directory, which is '''not''' the same as the user's home directory on the teaching cluster.

Shtsai: /* Transferring Files between two file systems on Sapelo2 */

2021-01-08T22:06:18Z

Transferring Files between two file systems on Sapelo2

← Older revision		Revision as of 18:06, 8 January 2021
Line 232:		Line 232:
	Use ssh to login to xfer.gacrc.uga.edu using your UGA MyID username and password (and two-factor authentication via Archpass Duo) to transfer files between different file systems on Sapelo2, including the project area. To transfer data between two file systems that are available on the xfer node, you can use the '''cp''' or the '''rsync''' commands. If you have many files to copy, then a good option is to use the '''fpsync''' command, as that will allow the use of multiple cores.		Use ssh to login to xfer.gacrc.uga.edu using your UGA MyID username and password (and two-factor authentication via Archpass Duo) to transfer files between different file systems on Sapelo2, including the project area. To transfer data between two file systems that are available on the xfer node, you can use the '''cp''' or the '''rsync''' commands. If you have many files to copy, then a good option is to use the '''fpsync''' command, as that will allow the use of multiple cores.

	For example, if you want to use 4 cores to transfer all files from /scratch/myid/mydata to /project/mylab/myid/mydata, you can do the following on an xfer node:		For example, if you want to use 4 cores to transfer all files from /scratch/myid/mydata to /project/mylab/myid/mydata, you can use the following command on an xfer node shell prompt:

	<pre class="gcomman">		<pre class="gcomman">