To connect to a bucket from the DigitalArchive with an s3 client you need

• Endpoint: https://digitalarchiv.s3.fds.rwth-aachen.de:9021
• Access Key
• Secret Key
• Bucket Name

You will get this from the admins of the corresponding data project in SeviRe to which your s3-bucket belongs.

---

1. Download

Download the client software rclone: https://rclone.org/downloads/
Rclone is a single executable file (rclone or rclone.exe on Windows) that you can download as a zip archive and extract to a location of your choice.

2. Installation and Configuration

Call the command line. To do this, either enter "cmd" in the Windows search bar and confirm with Enter or use the Windows key + "R" and then type "cmd" and confirm with Enter.

Now use the command line to call up the directory in which you have stored the rclone file.

Note: 

• You change the directory with the command cd<path> for example: cd C:\FolderA\Folder1\rclone\. 
• Note here that CMD does not support UNC paths (examples: "Downloads", "Documents",...). Therefore, please make sure that the downloaded rclone folder is located locally on the computer and the correct path is specified to it.

Run the rclone config command.

For example: C:\Program Files\rclone-v1.57.0-windows-amd64>rclone config

Now you've started the configuration of rclone. Create a remote access to the object storage by following the configuration steps.

Note: The commands may differ slightly depending on the rclone version, so pay less attention to the number you enter in the following steps and more attention to what you have to select.

# New remote

-> n

# Name (=RemoteName)

-> free selectable (Here it is advisable to choose a name that corresponds to the name chosen for your data project in Sevire)

# Storage type

-> (Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS)

# Provider

-> any other S3 compatible provider

# credentials

-> 1 (Enter AWS credentials in the next step.)

# access_key_id

-> [Access Key]  (zugehörig zu Ihrem Bucket auf SeviRe)

# secret_access_key

-> [Secret Access Key] (zugehörig zu Ihrem Bucket auf SeviRe)

# region

-> 2 (other-v4-signature)

# endpoint

-> https://digitalarchiv.s3.fds.rwth-aachen.de:9021

# location_constraint

-> Do not enter anything and press Enter

# ACL

-> 1 (Owner gets FULL_CONTROL.)

# advanced config

-> n (No)

# server_side_encryption 

-> Do not enter anything and press Enter

# sse_kmskey_id

-> Do not enter anything and press Enter

# okay?

-> y (Yes, this is ok)

# quit

-> q

 

This completes the configuration and you have now created a connection to your bucket.

3. Upload files

Starting point: From the command line, call the directory where you placed the software file and run the rclone copy command.

• Generic

rclone copy --s3-no-check-bucket [file(-path)] [Remotename]:[Bucketname]\

• Concrete

rclone copy --s3-no-check-bucket testfile MyDigiatlarchiv:ABC-1234567\

• With more spending

rclone copy --s3-no-check-bucket testfile MyDigiatlarchiv:ABC-1234567\ -v

 

4. Multithread

Rclone offers the possibility of a multithread upload/download.

Note: When dividing, the size of the sections depends on the load and equipment of the server. In the following examples we have chosen a size of 4 MB. The specification of the --s3-upload-concurrency  also depends on this. 

• Generic

rclone copy --s3-no-check-bucket [file] [RemoteName]:[Bucketname] \ --multi-thread-cutoff [section size] --multi-thread-streams [size] --s3-upload-concurrency [size]

• Concrete

rclone copy --s3-no-check-bucket testfile MyDigiatlarchiv:ABC-1234567 \ --multi-thread-cutoff 4M --multi-thread-streams 4 --s3-upload-concurrency 4

• With more spending

rclone copy --s3-no-check-bucket testfile MyDigiatlarchiv:ABC-1234567 \ --multi-thread-cutoff 4M --multi-thread-streams 4 --s3-upload-concurrency 4 -v

 

5. Download files

To download objects, first specify their bucket name and then the location where you want to save their objects.

 

• Generic

rclone copy [RemoteName]:[Bucketname]\[Object, you want to download] [storage location]\

• Concrete

rclone copy MyDigiatlarchiv:ABC-1234567\testfile C:\Downloads\

• With more spending

rclone copy MyDigiatlarchiv:ABC-1234567\testfile C:\Downloads\ -v

 

Note: Run rclone ls [RemoteName]:[Bucketname] to get a listing of the objects that are in your bucket.