Globus-CLI

From Research Computing Center Wiki
Revision as of 14:02, 7 October 2021 by Shtsai (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

The Globus Command Line Interface (CLI) provides an interface to Globus services from the shell, and is suited to both interactive and simple scripting use cases. The CLI can be used to integrate Globus actions into your scripts to automate your data flows.

To use it on Sapelo2, you need to load the module first, and authenticate a session. This can be done on an interactive node, before you use Globus CLI commands in your scripts.

Example steps to use:

1. Start an interactive session with the command 

interact

2. On the interactive node, load the Globus-CLI module with 

ml Globus-CLI/2.1.0-GCCcore-8.3.0

3. Authenticate with Globus using the command globus login. This command will present you a URL, which you should copy and paste into a browser on your local machine. Once you visit the link in the browser, you will be prompted to login. Please choose University of Georgia in the "Use your existing organizational login" box. Choose continue and you will be forwarded to a UGA Single Sign-On (SSO) login page. You will also need to authenticate with Duo (two-factor authentication). Once the authentication is complete, your browser will display an Authorization Code, which you need to enter into your Sapelo2 session and press the return key. 

[shtsai@ss-sub3 ~]$ interact

srun --pty  --cpus-per-task=1 --job-name=interact --ntasks=1 --nodes=1 --partition=inter_p --time=12:00:00 --mem=2GB /bin/bash -l

[shtsai@ra3-22 ~]$ ml Globus-CLI/2.1.0-GCCcore-8.3.0 
[shtsai@ra3-22 ~]$ globus login
Please authenticate with Globus here:
------------------------------------
https://auth.globus.org/v2/oauth2/authorize?client_id=c39024f239054858-d791-4e0c-bf78-b193sdf94506af640f&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&scope=openid+profile+email+urn%3Aglobus%3Aauth%3Ascope%3Aauth.globus.org%3Aview_identity_set+urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall&state=_default&response_type=code&access_type=offline&prompt=login
------------------------------------

Enter the resulting Authorization Code here: EpaIO2Xdt94kgfkash6pzftQUM

You have successfully logged in to the Globus CLI!

You can check your primary identity with
  globus whoami

For information on which of your identities are in session use
  globus session show

Logout of the Globus CLI with
  globus logout

[shtsai@ra3-22 ~]$

The first time you authenticate, a (hidden) file called .globus.cfg will be created in your Sapelo2 home directory. This file stores the configuration for your globus authentication token.

4. You can use the CLI to search for endpoints, for example, to search for endpoints that have GACRC in the name: 

[shtsai@ra3-22 ~]$ globus endpoint search "GACRC"
ID                                   | Owner                                                        | Display Name                
------------------------------------ | ------------------------------------------------------------ | ----------------------------
75b676f4-e35a-4318-a3cf-48357165393d | 75b676f4-e35a-4318-a3cf-48357165393d@clients.auth.globus.org | UGA GACRC Endpoint          
a640b990-6d04-11e5-ba46-22000b92c6ec | gacrc@globusid.org                                           | gacrc#copy3                 
006db804-1b14-4dcf-a93a-53812be01598 | 75b676f4-e35a-4318-a3cf-48357165393d@clients.auth.globus.org | UGA GACRC Collection        
255e7108-f173-4ab7-b5c1-09101d32fac4 | 75b676f4-e35a-4318-a3cf-48357165393d@clients.auth.globus.org | UGA GACRC Project Sharing   
d7b7190e-a649-4b48-b0a0-55dfae52e639 | 75b676f4-e35a-4318-a3cf-48357165393d@clients.auth.globus.org | UGA GACRC read-write Sharing

5. To list the files in your Sapelo2 home dir, use the command globus ls followed by the ID of the UGA GACRC Collection. This command will prompt you to run a globus session consent command. This command will present an URL that you need to open with a browser on your local machine. Login to your globus account via the browser and paste the authentication token. 

[shtsai@ra3-22 ~]$ globus ls 006db804-1b14-4dcf-a93a-53812be01598
The collection you are trying to access data on requires you to grant consent for the Globus CLI to access it.
message: Missing required data_access consent

Please run

  globus session consent 'urn:globus:auth:scope:transfer.api.globus.org:all[*https://auth.globus.org/scopes/006db804-1b14-4dcf-a93a-53812be01598/data_access]'

to login with the required scopes
[shtsai@ra3-22 ~]$ globus session consent 'urn:globus:auth:scope:transfer.api.globus.org:all[*https://auth.globus.org/scopes/006db804-1b14-4dcf-a93a-53812be01598/data_access]'
Please authenticate with Globus here:
------------------------------------
https://auth.globus.org/v2/oauth2/authorize?client_id=c39024f2-d791-4e0c-bf78-b19306af640f&redirect_uri=https%3A%2F%2Fauth.globus.org%2Fv2%2Fweb%2Fauth-code&scope=urn%3Aglobus%3Aauth%3Ascope%3Atransfer.api.globus.org%3Aall%5B%2Ahttps%3A%2F%2Fauth.globus.org%2Fscopes%2F006db804-1b14-4dcf-a93a-53812be01598%2Fdata_access%5D&state=_default&response_type=code&access_type=offline&prompt=login
------------------------------------

Enter the resulting Authorization Code here: eN08ZMhoiZlqmxZMQXahHNB5l5eHDV

You have successfully updated your CLI session.

[shtsai@ra3-22 ~]$

Once this is done, you can list your files with the command globus ls 006db804-1b14-4dcf-a93a-53812be01598 .

6. You can transfer files between Sapelo2 file systems, or to/from another endpoint, including endpoints on your local machine.

Help page: 

[shtsai@ra3-22 ~]$ globus --help
Usage: globus [OPTIONS] COMMAND [ARGS]...

  Interact with Globus from the command line

  All `globus` subcommands support `--help` documentation.

  Use `globus login` to get started!

  The documentation is also online at https://docs.globus.org/cli/

Options:
  -v, --verbose                  Control level of output
  -h, --help                     Show this message and exit.
  -F, --format [unix|json|text]  Output format for stdout. Defaults to text
  --jmespath, --jq TEXT          A JMESPath expression to apply to json
                                 output. Takes precedence over any specified '
                                 --format' and forces the format to be json
                                 processed by this expression

  --map-http-status TEXT         Map HTTP statuses to any of these exit codes:
                                 0,1,50-99. e.g. "404=50,403=51"


Commands:
  bookmark        Manage endpoint bookmarks
  config          Manage your Globus config file. (Advanced Users)
  delete          Submit a delete task (asynchronous)
  endpoint        Manage Globus endpoint definitions
  get-identities  Lookup Globus Auth Identities
  list-commands   List all CLI Commands
  login           Log into Globus to get credentials for the Globus CLI
  logout          Logout of the Globus CLI
  ls              List endpoint directory contents
  mkdir           Create a directory on an endpoint
  rename          Rename a file or directory on an endpoint
  rm              Delete a single path; wait for it to complete
  session         Manage your CLI auth session
  task            Manage asynchronous tasks
  transfer        Submit a transfer task (asynchronous)
  update          Update the Globus CLI to its  latest version
  version         Show the version and exit
  whoami          Show the currently logged-in identity

Please refer to https://docs.globus.org/cli/

Retrieved from "https://wiki.gacrc.uga.edu/index.php?title=Globus-CLI&oldid=20583"