Data transfer

From Research Computing Center Wiki
Jump to navigation Jump to search

F*EX Service

The F*EX service (Referred to as FEX) allows you to share large datasets with collaborators. You can automatically register for this service as long as you have a uga.edu active email address and can sign-on to the UGA VPN.

Registration

  • You have to have an active uga.edu email address
  • You have to sign on to the UGA archpass VPN (The registration is only allowed when you are connected to the VPN, you can still send and receive data when you are not connected ot the VPN)
  • Go to FEX Registration and sign up for an account. Use your uga.edu email address
  • You will get an email from the service with a link to confirm your request.
  • Visit the confirmation link sent in the email to confirm your request.
  • After confirmation is successful, you will see the following info Save the info that you see in on the webpage:
    • FEX Account direct URL for your account
    • FEX Server URL (Always https://fex.gacrc.uga.edu/fup/)
    • Sender (Your email address)
    • auth-ID (Keep this to yourself)
      • If you decide to change this auth-ID at a later data, PLEASE, DO NOT USE YOUR MYID PASSWORD. USE A DIFFERENT PASSWORD UNIQUE TO THIS SERVICE.

Accessing the FEX service from a web browser

  • You can visit your unique FEX Account direct URL that you saved from the request confirmation page when you registered.
  • Or, you can go to F*EX Service and enter your email and auth-ID.

Accessing the FEX service from GACRC (Xfer nodes)

The FEX command line clients are installed on all GACRC xfer nodes. Make sure that you use your email and auth-ID instead of mine at the highlighted spots below.

raj76@xfer4:~$ fexsend -I
F*EX server URL: https://fex.gacrc.uga.edu
proxy address (hostname:port or empty if none): 
Your e-mail address as registered at https://fex.gacrc.uga.edu: YOUR_EMAIL_ADDRESS
Your auth-ID for raj76@uga.edu at https://fex.gacrc.uga.edu: MY_PRIVATE_AUTH_ID
data written to /home/raj76/.fex/id

You can then send any file or directory using the fexsend command. To send a file you use the command:

fexsend file_to_send email_of_recipients

email_of_recipients - Can be a comma separated list for multiple recipients.

To send a directory as a archive, use the following command:

 fexsend -a archive.tgz Directory_to_send email_of_recipients
>$ fexsend -h
usage:
  fexsend [options] FILE[...] [@] RECIPIENT[,...]
  fexsend [special option]
  fexsend -l ['RECIPIENT-REGEXP']
  fexsend -f NUMBER RECIPIENT[,...]
  fexsend -x NUMBER [-C -k -D -K]
options:
  -v            verbose mode
  -a ARCHIVE    put files in archive (.zip .7z .tar .tgz) without .* files
  -A ARCHIVE    put files in archive (.zip .7z .tar .tgz) with .* files
  -# FILELIST   exclude files (# is list separator) from archive
  -0            do not compress files for .zip and .7z archives
  -d            delete file on fex server
  -o            overwrite mode, do not try to resume
  -m KBS        limit throughput (KBS kB/s)
  -p SIZE:WAIT  send file in chunks (SIZE MB, WAIT seconds)
  -i ACCOUNT    use ID data [ACCOUNT] from ID file
  -C 'COMMENT'  add COMMENT to notification email
  -k MAX        keep file MAX days on fex server
  -D            delay auto-delete after download
  -K            no auto-delete after download
  -M            MIME-file (to be displayed in recipient's webbrowser)
  -s STREAM     read data from pipe and upload it with STREAM name
  -r ADDRESS    use Reply-To ADDRESS in notification email
  -n            send no notification email, just show the download URL
  -/            do not upload the file, but tell the server to link it
  -= ALIAS      use ALIAS as filename
  -^            add date_time to filename
  -q            quiet mode
  -+            undocumented feature - test it :-)
special options:
  -I          initialize ID file or show ID
  -I ACCOUNT  add alternate ID data (secondary logins) to ID file
  -l          list sent files (numbers needed for -f -x -d -N)
  -u          list download URLs of sent files
  -f NUMBER   forward already uploaded file to another recipient
  -x NUMBER   use -C -k -D -K for already uploaded file
  -d NUMBER   delete file on fex server
  -N NUMBER   resend notification email
  -Q          check quotas
  -U          generate upload URL with bash function (-U+ for archives)
  -T UP:DOWN  test internet speed with upload and download MBs
  -@          edit server address book (aliases)
  -S          show server/user settings and auth-ID
  -V          show version and installation/upgrade hint
  -H          show hints and examples and more options
examples:
  fexsend visualization.mp4 framstag@rus.uni-stuttgart.de
  fexsend -a images.zip *.jpg webmaster@belwue.de,metoo
  lshw | fexsend -s hardware.list admin@belwue.de