To migrate data between two instances (for example, between researchdrive.surfsara.nl and a branded instance), we advice to use rclone, which you can download here: https://rclone.org/downloads/
Define data source and destination
First, log in to the source environment, go to 'settings' in the top right menu, and then to 'security'. Scroll down and create a new app password. Don't click 'done' yet! You need the information that it shows there:
You need from this:
- Username
- Password
- URL
Configure rclone
After installing rclone, begin by opening a command line terminal
Start the rclone cofigurator:
rclone config
First, give a clear name to your remote:
n) New remote s) Set configuration password q) Quit config n/s/q> n name> source-rdrive
Now rclone will give you a (long) list of possible storages. Select 'Webdav'
Type of storage to configure. Choose a number from below, or type in your own value 1 / Alias for a existing remote \ "alias" 2 / Amazon Drive \ "amazon cloud drive" (...) 19 / SSH/SFTP Connection \ "sftp" 20 / Webdav \ "webdav" (...) Storage> 20
Now copy/paste the full URL from the webdav settings (that you got earlier)
URL of http host to connect to Choose a number from below, or type in your own value 1 / Connect to example.com \ "https://example.com" url> https://researchdrive.surfsara.nl/remote.php/nonshib-webdav/
Select owncloud
Name of the Webdav site/service/software you are using Choose a number from below, or type in your own value 1 / Nextcloud \ "nextcloud" 2 / Owncloud \ "owncloud" 3 / Other site/service or software \ "other" vendor> 2
Copy/paste the username from the webdav settings (that you got earlier)
User name user> example_user
Copy/paste the password from the webdav settings (that you got earlier)
Password. y) Yes type in my own password g) Generate random password n) No leave this optional password blank y/g/n> y Enter the password: password: Confirm the password: ******** password: ********
Depending on your rclone version, it may ask for advanced config or bearer token here. You do not need those:
Edit advanced config? (y/n) y) Yes n) No (default) y/n> n
Check the configuration and confirm if it is correct
Remote config -------------------- [source-rdrive] type = webdav url = https://researchdrive.surfsara.nl/remote.php/nonshib-webdav/ vendor = owncloud user = example_user pass = *** ENCRYPTED *** -------------------- y) Yes this is OK e) Edit this remote d) Delete this remote y/e/d> y
Quit the configurator
Current remotes: Name Type ==== ==== source-rdrive webdav e) Edit existing remote n) New remote d) Delete remote r) Rename remote c) Copy remote s) Set configuration password q) Quit config e/n/d/r/c/s/q> q
repeat steps
You have to repeat the above steps for the destination environment
Testing your configuration
If all went well, you have a source and a destination defined:
rclone listremotes source-rdrive: target-example:
You should be able to list your files:
rclone ls "source-rdrive:/source_project (Projectfolder)/" rclone ls "target-example:/destination_project (Projectfolder)/"
Migrating the data
See rclone performance considerations for some important performance tips!
It is important to set the '--timeout
' option high enough. As a rule of thumb, set it to 10 minutes for every GB of the biggest file in a collection. So if the biggest file you want to upload in a collection is 10GB, set --timeout 100m
This may look ridiculously large, but it provides a safe margin to avoid problems with timeout issues.
And you should be able to transfer your files:
rclone -vvv copy --timeout 360m --use-cookies --transfers 16 "source-rdrive:/source_project (Projectfolder)/" "target-example:/destination_project (Projectfolder)/"