CLEAR CLoud Storage with Rclone Guide

Using rclone on Personal Computers

Rclone is a command-line program for managing files on cloud storage. It supports many familiar file-management operations, including copying, synchronizing, moving, listing, and organizing files.

This section applies primarily to personal computers and Rice-owned computers where you have persistent local user storage. Lab computers may not retain the rclone configuration files required between sessions.

Supported Systems

Rclone can be used on systems where you have persistent user storage, including:

• Personal Windows, macOS, and Linux computers
• Rice-owned computers where you have persistent local user storage
• CLEAR systems for classroom and course work

Cloud Storage Providers

This guide focuses on the following cloud storage providers:

• Box: Rice Box Login
• Google Drive: Google Drive
• Microsoft OneDrive: Microsoft OneDrive

Rclone Documentation

• Rclone Home
• Rclone Commands

Rclone Help Commands

rclone --help
rclone <command> --help
rclone help flags
rclone help backends

What Can You Do with rclone?

Common rclone operations include:

• Copying files
• Synchronizing files and directories
• Moving files
• Checking and comparing files
• Backing up and restoring files
• Migrating files between storage providers
• Listing and viewing storage contents

Additional functionality is available depending on the system, cloud provider, and permissions being used.

Install rclone

Follow the official installation instructions for your operating system:

Rclone Installation Instructions

Configure rclone

Start the interactive rclone configuration process with:

rclone config

Configuration options vary depending on the cloud storage provider and the version of rclone installed. Follow the prompts presented by rclone config and refer to the provider-specific documentation when necessary.

During configuration, you will typically:

1. Create a new remote.
2. Choose a short name for the remote, such as MyBox, MyGDrv, or MyOneDrv.
3. Select the appropriate cloud storage provider.
4. Configure the requested provider options.
5. Complete authentication in your web browser when prompted.
6. Review and save the completed configuration.

NOTE: The numbered storage-provider options displayed by rclone config may change between rclone versions. Select the provider by its displayed name rather than relying on a specific option number.

If prompted for a Client ID or Client Secret and you do not have custom application credentials, follow the current rclone instructions for the selected provider.

On a personal computer with a graphical web browser, automatic configuration can normally be used to complete the provider authentication process.

Testing Your Configuration

The following example assumes that a Google Drive remote has been configured with the name MyGDrv:

rclone lsd MyGDrv:/
rclone mkdir MyGDrv:/testdir
rclone copy <file> MyGDrv:/testdir
rclone lsd MyGDrv:/
rclone ls MyGDrv:/
rclone tree MyGDrv:/

The first command may return no results if the remote storage location is empty.

Provider-Specific Documentation

Box

• Rclone Box Auto Config Screenshots
• Rclone Box Documentation
• Rclone Box Advanced Options
• Rclone Box Limitations

Google Drive

• Rclone Drive Auto Config Screenshots
• Rclone Google Drive Documentation
• Rclone Drive Standard Options
• Rclone Drive Advanced Options
• Rclone Drive Backend Commands
• Google Docs Limitations

Microsoft OneDrive

• Rclone OneDrive Auto Config Screenshots
• Rclone OneDrive Documentation
• Rclone OneDrive Standard Options
• Rclone OneDrive Advanced Options
• Rclone OneDrive Limitations

Using rclone on CLEAR

Rclone can also be used directly from the CLEAR Linux systems to transfer and manage files stored with supported cloud storage providers.

Your rclone configuration is stored in your user environment. Therefore, you should have a working CLEAR account and home directory before configuring rclone.

Connect to CLEAR

Connect to CLEAR using SSH:

ssh <NetID>@ssh.clear.rice.edu

Replace <NetID> with your Rice NetID and complete authentication when prompted.

Configure rclone

After logging in to CLEAR, start the configuration process:

rclone config

Create a new remote, select your cloud storage provider, and follow the configuration prompts.

Configuration options vary by provider and rclone version. Do not rely on a specific numerical option for Box, Google Drive, or OneDrive. Select the appropriate provider by name from the list displayed by rclone.

Browser-Based Authentication from CLEAR

Cloud providers generally require browser-based authentication. When configuring rclone on a remote CLEAR system, follow the authentication instructions displayed by the version of rclone installed on CLEAR.

Depending on the authentication method presented, you may need to use a web browser on your local computer or use an SSH session with X11 forwarding.

Using X11 Forwarding

If the authentication process requires a graphical application running from CLEAR, establish an SSH connection with X11 forwarding enabled.

Windows

MobaXterm can provide both SSH access and an X server. When creating the SSH session, enable X11 forwarding and connect to:

ssh.clear.rice.edu

macOS

Make sure an X11 server such as XQuartz is installed and running. Connect using:

ssh -Y <NetID>@ssh.clear.rice.edu

Linux

On a graphical Linux desktop with an X server available, connect using:

ssh -Y <NetID>@ssh.clear.rice.edu

Test X11 Forwarding

If you are using X11 forwarding, you can test the connection by running:

xterm

If X11 forwarding is working correctly, a new terminal window should appear on your local computer. Close the test window before continuing.

Complete the rclone Configuration

Run:

rclone config

Follow the prompts to configure your cloud storage provider. When rclone requests authentication, follow the browser or authorization instructions displayed by rclone.

After authentication is complete, return to your SSH terminal and finish the configuration process. Review the configuration summary before accepting and saving the remote.

Test the CLEAR Configuration

The following example uses a Google Drive remote named MyGDrv:

rclone lsd MyGDrv:/
rclone mkdir MyGDrv:/testdir
rclone copy <file> MyGDrv:/testdir
rclone lsd MyGDrv:/
rclone ls MyGDrv:/
rclone tree MyGDrv:/

Use the remote name that you created during rclone config.

Microsoft OneDrive Authentication

The following screenshots illustrate the browser-based authentication process used when configuring rclone with Microsoft OneDrive. The exact appearance of authentication screens may change over time.

Initial Login

Enter your Rice email address when prompted:

<Rice_NetID>@rice.edu

Confirm Your Account

Select Next.

Enter Your Password

Enter your Rice NetID password when prompted.

Select Sign in.

Complete Duo Authentication

Complete the required Duo authentication using your registered device.

Authentication Complete

After authentication succeeds, return to your rclone config session and complete the remaining configuration steps.

Google Drive Authentication

The following screenshots illustrate the browser-based authentication process used when configuring rclone with Google Drive. The exact appearance of authentication screens may change over time.

Initial Login

Enter your Rice email address when prompted:

<Rice_NetID>@rice.edu

Select Next.

Rice Authentication

Enter your Rice NetID and Rice NetID password when prompted by the Rice authentication system.

Complete Duo Authentication

Complete the required Duo authentication using your registered device.

Google Account Verification

Verify that the displayed account is your Rice account, then continue.

Authorize rclone

Review the requested access and authorize rclone if you want it to access your Google Drive.

Authentication Complete

After authorization succeeds, return to your rclone config session and complete the configuration.

Box Authentication

The following screenshots illustrate the browser-based authentication process used when configuring rclone with Box. The exact appearance of authentication screens may change over time.

Initial Login

Select Single Sign On.

Single Sign-On

Enter your Rice email address:

<Rice_NetID>@rice.edu

Continue with the authentication process.

Rice Authentication

Enter your Rice NetID and Rice NetID password when prompted by the Rice authentication system.

Complete Duo Authentication

Complete the required Duo authentication using your registered device.

Grant rclone Access

Review the requested access and grant rclone access to your Box account if you want to continue.

Authentication Complete

After authentication succeeds, return to your rclone config terminal and complete the remaining configuration steps.