“Instead of maximizing usability at the expense of utility, we should focus on making powerful tools and fostering a culture of mentorship".

The iCommands are a powerful set of tools that can be installed on any Unix environment and used both for administrative tasks and (big) data handling. The high performance of iCommands comes from its use of the full ability of iRODS to transfer files in parallel. It can securely send a large file completely saturating the available network.

Note that the iCommands can be solely operated from within the terminal. The following information assumes basic proficiency of command line tools.

(Most of this information can also be found on docs.irods.org.)

Credentials

To access Yoda via the command line, a Data Access Password is required (How to get a Data Access Password).

The Data Access Password must be entered when a password is requested.

Installation

For Ubuntu 18.04:

wget -qO - https://packages.irods.org/irods-signing-key.asc | sudo apt-key add -
echo "deb [arch=amd64] https://packages.irods.org/apt/ xenial main" | sudo tee /etc/apt/sources.list.d/renci-irods.list
sudo apt-get update
sudo apt -y install irods-icommands

For CentOS:

sudo yum -y install wget epel-release
sudo rpm --import https://packages.irods.org/irods-signing-key.asc
wget -qO - https://packages.irods.org/renci-irods.yum.repo | sudo tee /etc/yum.repos.d/renci-irods.yum.repo
sudo yum -y install irods-icommands

For MacOS:

There are no up-to-date packages for MacOS at time of writing this document.
The iCommands can be built from source (https://github.com/irods/irods_client_icommands), however it is not trivial.

We have also worked successfully with iCommands using Docker containers. Please let us know by opening a request on https://servicedesk.surf.nl if you would like to explore this option.

For Windows:

The iCommands work in Linux.
They should be installable via the Ubuntu method using the linux subsystem of Windows 10.

Connecting to Yoda/iRODS

To connect to Yoda/iRODS you have to create the `~/.irods/irods_environment.json` file and edit it as follows:

{
"irods_authentication_scheme": "pam",
"irods_client_server_negotiation": "request_server_negotiation",
"irods_encryption_algorithm": "AES-256-CBC",
"irods_encryption_key_size": 32,
"irods_encryption_num_hash_rounds": 16,
"irods_encryption_salt_size": 8,
"irods_host": "YODA_HOST_ADDRESS",
"irods_home": "/YODA_ZONE/home",
"irods_port": 1247,
"irods_user_name": "YODA_USERNAME",
"irods_zone_name": "YODA_ZONE"
}

Note to change the specific details:

• Check Yoda Hosting (URLs) for choosing the correct YODA_HOST_ADDRESS
• Your username, YODA_USERNAME, is either the identity from your institute or the e-mail address you were invited through
• The zone, YODA_ZONE, is usually the acronym of your institute (for example: tue or aumc), but sometimes it is just "yoda" or "tempZone". These are the zones for the institutes that use Yoda:

You can then run the command:

iinit

Enter your password, and a token will be stored locally which will be used when running subsequent iCommands.

You can find help for the iCommands with:

ihelp

which will show you all the possible commands.

Basic data/collection handling

Note that in iRODS files are called data objects, and folders are called collections.

The iCommands have basic file handling functionality that have their (almost) equivalent in the Linux bash commands but with an 'i' in front of the name, *e.g.* 'ls' and 'ils', 'cp' and 'icp', 'mv' and 'imv', 'pwd' and 'ipwd', 'mkdir' and 'imkdir', 'cd' and 'icd'. 'rm' and 'irm'. Try running 'icommand -h' or 'ihelp' if you want to know more about the available options.

However, please note that iRODS is not a filesystem like you have on your personal computer - data handling is done differently.

Create collections and navigate

Use 'ils' to see what collections are available to you and navigate to that collection with 'icd':

ils

icd <collection name>

Note that you can't use bash tab completion with paths in iRODS.

You can create a collection with 'imkdir'.

If you get an error that you are not allowed to create a collection, be sure to have navigated to a collection where you do have the permission to write.
You can use:

ils -A 

to check the permissions ('-A' stands for Access Control Lists) of data objects and collections.

Sharing data objects and/or collections with other users

With 'ils -A' you can view what the current Access Control Lists are, *i.e.* a list who has what type of access to the data objects and collections in the current working directory of iRODS.
The ACLs can be changed by using 'ichmod'. With this command you modify the access to data objects and collections for specific user or groups.

When you store a file, you are the owner and have full control - you can read, write or delete it and, by default, no one else can. With chmod you can give access to other users or groups, either just read access, or read and write, or full ownership. You can only give (or remove) access to others if you own the file yourself, but if you give 'own' to someone else, they can also give (and remove) access to others.

For example, if you like to give user or group 'alice' recursive read permissions on folder 'fruit' which is located in your iRODS current working directory (you could also use the full iRODS path), you should do the following:

ichmod -r read alice fruit

For more information, see 'ichmod -h'.

Tickets are another way to provide access to iRODS data objects (files) or collections (directories or folders).
The 'iticket' command allows you to create, modify, list, and delete tickets. When you create a ticket, its 15 character string is given to you which you can share with others.

For example, if you would like to give read permission for a collection 'fruit':

iticket create read fruit

This command will print out the ticket string to screen.

Note that you can view all active tickets with 'iticket ls'.

You can use 'iticket mod ticket-string-or-id'` to modify the restrictions on the ticket. Restrictions can be set on who is able to use the ticket, how many times the ticket can be used, which host or IP is able to use the ticket or the expiration date of the ticket.

Uploading a file with 'iput'

To upload a file, use the 'iput' command:

iput <source_file> <dest_do>

When no destination data object is given, the local filename will be used.

iRODS will issue a warning if the file already exists (which you can force to do with the `-f` option).

The same command is used to upload a folder (and its contents):

iput -r <source_folder> <dest_coll>

Note that you need to add the '-r' option for recursively putting data into Yoda/iRODS.

Downloading a file with 'iget'

iget <source_do> <dest_file>

Again, if the local destination is not specified, the name of the data object will be used.

To download a collection you have to specify the '-r' option (for 'recursive'):

iget -r <source_coll> <dest_folder>

Metadata and querying for data

In Yoda/iRODS a data object is not only the bitstream and the filename, but also includes user defined metadata. This makes it very powerful for the findability of data. Data objects can be found by querying for metadata.

Querying based on metadata

Metadata attached to data objects and collections becomes very useful when you want to query for data. Queries are mainly be performed using the 'iquest' command, but simple queries can also be done using the 'imeta' command:

#find data objects where distance is larger than 12
imeta qu -d distance '<=' 12

#find collections where key a is value b and key c smaller than value 10
imeta qu -C a = b and c '<' 10

#find data objects where r is not between 5 and 7
imeta qu -d r '<' 5 or '>' 7

#find data-objects with attribute 'a' with a value that starts with 'b'
imeta qu -d a like b%

#find data-objects with attribute 'a' defined (with any value)
imeta qu -d a like %

You can enter the 'imeta' prompt by just typing 'imeta' and hitting Enter. You can then get more helpful tips by executing 'help qu'.

To see all the available keys you can query for:

iquest attrs

General 'iquest' queries look like:

iquest "select COLL_NAME, DATA_NAME, META_DATA_ATTR_VALUE where \
META_DATA_ATTR_NAME like 'author'" 

In the above command, we looked for everything with the metadata attribute name (metadata key) that resembles 'author' and retrieved the collection name, data name and metadata attribute value (metadata value).
iRODS will respond with something similar to:

COLL_NAME = /yoda/home/research-dms
DATA_NAME = testfile
META_DATA_ATTR_VALUE = val1
------------------------------------------------------------

You can change the format of the iRODS response by using a C-like format string after 'iquest' like so:

iquest "User %-6.6s has %-5.5s access to file %s" "SELECT USER_NAME, DATA_ACCESS_NAME, DATA_NAME WHERE COLL_NAME = '/yoda/home/research-dms'"

where after 'iquest' there is the format string and the second string is again the SQL-like query. This can be convenient for making reports, but also for retrieving absolute filepaths:

iquest "%s/%s" "select COLL_NAME, DATA_NAME where \
META_DATA_ATTR_NAME like 'author' and META_DATA_ATTR_VALUE = 'Lewis Carroll'"