Data transfer: Difference between revisions
Jump to navigation
Jump to search
(Wiki page to document the different data transfer methods available to GACRC users.) |
No edit summary |
||
(6 intermediate revisions by the same user not shown) | |||
Line 7: | Line 7: | ||
* You have to sign on to the UGA archpass VPN (The registration is only allowed when you are connected to the VPN, you can still send and receive data when you are not connected ot the VPN) | * You have to sign on to the UGA archpass VPN (The registration is only allowed when you are connected to the VPN, you can still send and receive data when you are not connected ot the VPN) | ||
* Go to [https://fex.gacrc.uga.edu/fur FEX Registration] and sign up for an account. Use your uga.edu email address | * Go to [https://fex.gacrc.uga.edu/fur FEX Registration] and sign up for an account. Use your uga.edu email address | ||
* | * You will get an email from the service with a link to confirm your request. | ||
* Visit the confirmation link sent in the email to confirm your request. | |||
* After confirmation is successful, you will see the following info '''''Save the info that you see in on the webpage''''': | |||
** '''FEX Account direct URL''' for your account | |||
** FEX Server '''URL''' (Always https://fex.gacrc.uga.edu/fup/) | |||
** '''Sender''' (Your email address) | |||
** '''auth-ID''' (Keep this to yourself) | |||
***If you decide to change this auth-ID at a later data, '''''PLEASE, DO NOT USE YOUR MYID PASSWORD. USE A DIFFERENT PASSWORD UNIQUE TO THIS SERVICE.''''' | |||
=== Accessing the FEX service from a web browser === | |||
* You can visit your unique '''FEX Account direct URL''' that you saved from the request confirmation page when you registered. | |||
* Or, you can go to [https://fex.gacrc.uga.edu/fup F*EX Service] and enter your email and auth-ID. | |||
=== Accessing the FEX service from GACRC (Xfer nodes) === | |||
The FEX command line clients are installed on all GACRC xfer nodes. | |||
Make sure that you use your email and auth-ID instead of mine at the highlighted spots below. | |||
<nowiki>raj76@xfer4:~$ fexsend -I | |||
F*EX server URL: https://fex.gacrc.uga.edu | |||
proxy address (hostname:port or empty if none): | |||
Your e-mail address as registered at https://fex.gacrc.uga.edu: YOUR_EMAIL_ADDRESS | |||
Your auth-ID for raj76@uga.edu at https://fex.gacrc.uga.edu: MY_PRIVATE_AUTH_ID | |||
data written to /home/raj76/.fex/id | |||
</nowiki> | |||
You can then send any file or directory using the fexsend command. | |||
To send a file you use the command: | |||
fexsend file_to_send email_of_recipients | |||
''email_of_recipients - Can be a comma separated list for multiple recipients.'' | |||
To send a directory as a archive, use the following command: | |||
fexsend -a archive.tgz Directory_to_send email_of_recipients | |||
<nowiki> | |||
>$ fexsend -h | |||
usage: | |||
fexsend [options] FILE[...] [@] RECIPIENT[,...] | |||
fexsend [special option] | |||
fexsend -l ['RECIPIENT-REGEXP'] | |||
fexsend -f NUMBER RECIPIENT[,...] | |||
fexsend -x NUMBER [-C -k -D -K] | |||
options: | |||
-v verbose mode | |||
-a ARCHIVE put files in archive (.zip .7z .tar .tgz) without .* files | |||
-A ARCHIVE put files in archive (.zip .7z .tar .tgz) with .* files | |||
-# FILELIST exclude files (# is list separator) from archive | |||
-0 do not compress files for .zip and .7z archives | |||
-d delete file on fex server | |||
-o overwrite mode, do not try to resume | |||
-m KBS limit throughput (KBS kB/s) | |||
-p SIZE:WAIT send file in chunks (SIZE MB, WAIT seconds) | |||
-i ACCOUNT use ID data [ACCOUNT] from ID file | |||
-C 'COMMENT' add COMMENT to notification email | |||
-k MAX keep file MAX days on fex server | |||
-D delay auto-delete after download | |||
-K no auto-delete after download | |||
-M MIME-file (to be displayed in recipient's webbrowser) | |||
-s STREAM read data from pipe and upload it with STREAM name | |||
-r ADDRESS use Reply-To ADDRESS in notification email | |||
-n send no notification email, just show the download URL | |||
-/ do not upload the file, but tell the server to link it | |||
-= ALIAS use ALIAS as filename | |||
-^ add date_time to filename | |||
-q quiet mode | |||
-+ undocumented feature - test it :-) | |||
special options: | |||
-I initialize ID file or show ID | |||
-I ACCOUNT add alternate ID data (secondary logins) to ID file | |||
-l list sent files (numbers needed for -f -x -d -N) | |||
-u list download URLs of sent files | |||
-f NUMBER forward already uploaded file to another recipient | |||
-x NUMBER use -C -k -D -K for already uploaded file | |||
-d NUMBER delete file on fex server | |||
-N NUMBER resend notification email | |||
-Q check quotas | |||
-U generate upload URL with bash function (-U+ for archives) | |||
-T UP:DOWN test internet speed with upload and download MBs | |||
-@ edit server address book (aliases) | |||
-S show server/user settings and auth-ID | |||
-V show version and installation/upgrade hint | |||
-H show hints and examples and more options | |||
examples: | |||
fexsend visualization.mp4 framstag@rus.uni-stuttgart.de | |||
fexsend -a images.zip *.jpg webmaster@belwue.de,metoo | |||
lshw | fexsend -s hardware.list admin@belwue.de | |||
</nowiki> |
Latest revision as of 12:22, 7 May 2021
F*EX Service
The F*EX service (Referred to as FEX) allows you to share large datasets with collaborators. You can automatically register for this service as long as you have a uga.edu active email address and can sign-on to the UGA VPN.
Registration
- You have to have an active uga.edu email address
- You have to sign on to the UGA archpass VPN (The registration is only allowed when you are connected to the VPN, you can still send and receive data when you are not connected ot the VPN)
- Go to FEX Registration and sign up for an account. Use your uga.edu email address
- You will get an email from the service with a link to confirm your request.
- Visit the confirmation link sent in the email to confirm your request.
- After confirmation is successful, you will see the following info Save the info that you see in on the webpage:
- FEX Account direct URL for your account
- FEX Server URL (Always https://fex.gacrc.uga.edu/fup/)
- Sender (Your email address)
- auth-ID (Keep this to yourself)
- If you decide to change this auth-ID at a later data, PLEASE, DO NOT USE YOUR MYID PASSWORD. USE A DIFFERENT PASSWORD UNIQUE TO THIS SERVICE.
Accessing the FEX service from a web browser
- You can visit your unique FEX Account direct URL that you saved from the request confirmation page when you registered.
- Or, you can go to F*EX Service and enter your email and auth-ID.
Accessing the FEX service from GACRC (Xfer nodes)
The FEX command line clients are installed on all GACRC xfer nodes. Make sure that you use your email and auth-ID instead of mine at the highlighted spots below.
raj76@xfer4:~$ fexsend -I F*EX server URL: https://fex.gacrc.uga.edu proxy address (hostname:port or empty if none): Your e-mail address as registered at https://fex.gacrc.uga.edu: YOUR_EMAIL_ADDRESS Your auth-ID for raj76@uga.edu at https://fex.gacrc.uga.edu: MY_PRIVATE_AUTH_ID data written to /home/raj76/.fex/id
You can then send any file or directory using the fexsend command. To send a file you use the command:
fexsend file_to_send email_of_recipients
email_of_recipients - Can be a comma separated list for multiple recipients.
To send a directory as a archive, use the following command:
fexsend -a archive.tgz Directory_to_send email_of_recipients
>$ fexsend -h usage: fexsend [options] FILE[...] [@] RECIPIENT[,...] fexsend [special option] fexsend -l ['RECIPIENT-REGEXP'] fexsend -f NUMBER RECIPIENT[,...] fexsend -x NUMBER [-C -k -D -K] options: -v verbose mode -a ARCHIVE put files in archive (.zip .7z .tar .tgz) without .* files -A ARCHIVE put files in archive (.zip .7z .tar .tgz) with .* files -# FILELIST exclude files (# is list separator) from archive -0 do not compress files for .zip and .7z archives -d delete file on fex server -o overwrite mode, do not try to resume -m KBS limit throughput (KBS kB/s) -p SIZE:WAIT send file in chunks (SIZE MB, WAIT seconds) -i ACCOUNT use ID data [ACCOUNT] from ID file -C 'COMMENT' add COMMENT to notification email -k MAX keep file MAX days on fex server -D delay auto-delete after download -K no auto-delete after download -M MIME-file (to be displayed in recipient's webbrowser) -s STREAM read data from pipe and upload it with STREAM name -r ADDRESS use Reply-To ADDRESS in notification email -n send no notification email, just show the download URL -/ do not upload the file, but tell the server to link it -= ALIAS use ALIAS as filename -^ add date_time to filename -q quiet mode -+ undocumented feature - test it :-) special options: -I initialize ID file or show ID -I ACCOUNT add alternate ID data (secondary logins) to ID file -l list sent files (numbers needed for -f -x -d -N) -u list download URLs of sent files -f NUMBER forward already uploaded file to another recipient -x NUMBER use -C -k -D -K for already uploaded file -d NUMBER delete file on fex server -N NUMBER resend notification email -Q check quotas -U generate upload URL with bash function (-U+ for archives) -T UP:DOWN test internet speed with upload and download MBs -@ edit server address book (aliases) -S show server/user settings and auth-ID -V show version and installation/upgrade hint -H show hints and examples and more options examples: fexsend visualization.mp4 framstag@rus.uni-stuttgart.de fexsend -a images.zip *.jpg webmaster@belwue.de,metoo lshw | fexsend -s hardware.list admin@belwue.de