Hands-On for Introduction to SURF Research Cloud

0. Onboarding

For logging in ("authentication and authorization") SURF Research Cloud relies on another SURF-service: SURF Research Access Management (SRAM).

To be able to log in to the Research Cloud portal ( https://portal.live.surfresearchcloud.nl ) you have to me a member of an SRAM collaborative organization (CO).

There are different ways to create a CO and to become a member. Preferably, your institution can create a CO and invite you to it. SURF can create one for you, too.

If you are using a training account, the CO and your membership are set up, already. In that case, you will also have a training budget/wallet.

Otherwise, your wallet is funded by an NWO grant or a paid contract between your institution and SURF ("RCCS-contract").

0.1 Log in using SRAM

At the start of the online webinar you will be provided with a training account. You have received an invitation e-mail to a training-CO. When accepting this inviation you couple it to your identity at your institute. That way you will be able to log in with your usual institute credentials. The training account also has a wallet already linked so you can start using it directly.  You will be able to use this account for a week after this training, after which all workspaces and data will be deleted. If you will become a Research Cloud user in the future you will either use your own institute account or create an EduId.nl account for logging in, after being invited to a CO in SRAM.

0.2 Log in to the Research Cloud dashboard

1. Open the Research Cloud portal in your browser: https://portal.live.surfresearchcloud.nl
2. Click on the Log In button. A list of identity providers will show.
3. Select the same identity provider from the list that you registered with in SRAM in the previous section.
4. Follow your identity provider's steps to log in (either your institute or EduID)
5. Click Continue or Next until you reach the Research Cloud portal, where you are welcomed with the title: "Welcome to your SURF research cloud dashboard"

Exercise 1. First steps in an Ubuntu workspace

Once you are on board (i.e.: set up in the external services), we will be creating a first workspace.

1.1 Create a workspace

You can create your first Ubuntu-based workspace by following our step-by-step guide from our general user documentation.

This is the manual to create your workspace: https://servicedesk.surfsara.nl/wiki/x/kIKV .

For completeness sake, that guide includes steps to delete the workspace. For the time being, make sure that you just follow the steps to create a workspace from that guide now. Or put differently: do NOT delete it yet.

1.2 Log in to your workspace

Once you have an Ubuntu-based workspace, you can log in by following our step-by-step guide here: https://servicedesk.surfsara.nl/wiki/x/koKV

Depending on your earlier choice for "Desktop" or "Server" you follow the steps for SSH or for TOTP.

1.3 Working in your workspace

After logging in into your workspace you can start working with it. You can take a look around and try some shell commands for example, or try to install some of the packages you would normally use. Most of our catalog items give you 'sudo' rights in the workspace, this should be in the catalog item description or name.

1.4 Delete your workspace

On the Dashboard section of the Research Cloud portal, you can see all workspaces in your Collaborative Organisations. You can also see logs and delete workspaces that belong to you. When you are running a workspace, your quota is ticking from your wallet.

Once you no longer need a workspace, you can best delete it to release resources that the workspace may be keeping busy. Let us do that now for the workspace that you created so far.

1. In your dashboard, locate your Ubuntu-based workspace
2. Click on the arrow to the right of your workspace line, so that you display the workspace's details
3. Click on the Delete button

Exercise 2. Running a first scientific workflow in Jupyter

2.1 Create a workspace

You can create your first Jupyter-based workspace by following steps similar to the ones that you followed in the previous section (remember our step-by-step guide here: https://servicedesk.surfsara.nl/wiki/x/kIKV ).

2.2 Log in to your workspace

Once you have a Jupyter-based workspace, you can log in by following our step-by-step guide here: https://servicedesk.surfsara.nl/wiki/x/koKV

2.3 Delete your workspace

Once you no longer need a workspace, you can best delete it to release resources that the workspace may be keeping busy. Let us do that now for the workspace that you created so far.

1. In your dashboard, locate your Ubuntu-based workspace
2. Click on the arrow to the right of your workspace line, so that you display the workspace's details
3. Click on the Delete button

Exercise 3. Running a first scientific workflow in R-studio

Repeat a similar scenario as in Exercise 2. This time we leave it up to you to find some nice exercise to carry out in R-studio. If you find nice tips, please share with the room!

Think of releasing resources when you are done.

Exercise 4. Working with persistent data

4.1 Create a volume

As you can see, once you delete a workspace, all data that may have existed in the workspace is deleted as well. To allow to store data more persistently, you can create a storage volume. Let us do that now:

1. On the dashboard, look for the "Create new storage" card, and click the Create New button.
2. Follow the wizard analogously to how you created a workspace in the past. Make sure to choose the small size.

Once you finish the wizard, back on the dashboard, under the Storage tab you can now see a new volume there that you are the owner of. Wait until it has the state "Available".

4.2 Attach a volume to a workspace

If you now follow the wizard to launch a workspace, then in step 5 you will be able to choose the volume that you created a moment ago.

Choose any Ubuntu-based application to try this.

Can you see now that the volume's state is now In-use?

Wait until the workspace shows to be running. Then you can connect to it (see Exercise 1).

Check that you can see the volume, by running the following command:

df -hT /data/volume_2

That should deliver a result similar to this:

Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/sdb1      xfs    50G   84M   50G   1% /data/volume_2


See that you have the volume mounted under /data/volume_2. Let us see if you can write there:

date > /data/volume_2/first.txt

Then verify that you have written the time into that first.txt file:

cat /data/volume_2/first.txt

Do you see the date and time of a few seconds ago?

4.3 Upload data to a volume

Imagine you now want to upload some files from your laptop to this workspace into the persistent volume, so that you can process them later. Let us do that now.

1. On your laptop, open a new terminal
2. Create a new file called second.txt in your local home directory, which will contain the word "hello", like this:
   • echo hello > ~/second.txt
3. Now upload the file to your workspace, like this:
   • (note: pay attention to replacing your_ssh_username and workspace_ip with the right values, which you can get from your workspace's details! They are the same as the ones for your ssh connection)
     

     scp ~/second.txt your_ssh_username@workspace_ip:/data/volume_2

Now that you have uploaded a file (pretend that this was a large dataset), you can verify in the workspace that the file is actually there.

1. Go back to the original terminal, where you were connected to the workspace via SSH.
2. On the workspace, run the same command as before, but this time to show the contents of the second file:
   • cat /data/volume_2/second.txt

Can you see that the command returns the word "hello"? Congratulations! That proves that you have uploaded the right file!

4.4 Delete the workspace

You are now ready to delete the workspace. Do it as you did in previous exercises.

See that the volume is still there! Can you see that it is now back in state Available?

You can now launch a new workspace and attach the volume to it again (make sure you choose the app "Ubuntu 18.04 with storage"). Can you still see the data there?

Make sure to delete the workspace once you no longer need it.

4.5 Delete the volume

Following the same steps as for deleting workspaces, you can delete your data volumes. Delete now the volume you were playing with.