Rclone as a helper for external backups
Rsync for the Cloud
Backups for critical data have to withstand major disasters. Rclone, a descendant of Rsync, enables easy backups to the cloud and supports many services.
We all live in the hope of never getting into a situation where our belongings are destroyed by theft, fire, water, or natural disaster, but there are no guarantees. If you are serious about preserving your important digital data, you should secure it outside your home, preferably in several places at the same time. The Rsync-based Rclone [1] tool helps with off-site backup scenarios.
Rclone is a command-line program that performs one-way synchronization between locally defined datasets and a cloud. If you run it a second time, it searches the local filesystem for changes and then incrementally uploads only the changes to the cloud of your choice. Rclone implements most of Rsync's options and syntax. In addition, it offers additional commands to optimally support the individual services. Rclone offers a graphical front end for people who prefer not to work at the command line.
Rsync's two-way synchronization is not part of Rclone's feature set. This means that it deletes any locally-deleted files from the cloud, but not vice versa. In addition, the program does not save older versions when changes are made to files. It overwrites changed files, unless the selected service has its own versioning system, such as Google Drive or Dropbox. One way to avoid this is to use the backup-dir
function, which moves changed or deleted files to a separate directory [2].
The tool, written in Go, which can be found in the archives of most distributions, supports numerous cloud service providers and protocols, including the open source clouds Nextcloud, ownCloud, and OpenStack Swift, as well as the commercial offerings Amazon S3, Box, Dropbox, Google Drive, pCloud, and Microsoft OneDrive [3].
Rclone also supports the HTTP, FTP, SFTP, and WebDAV protocols, so web and file servers, as well as NAS devices, can be integrated as back ends using standard protocols. The program is available in variants for x86, ARM, and MIPS architectures. Not only for Linux, Rclone also is available for macOS, Windows, BSD, Plan 9, and Solaris, as well as NAS servers from QNAP and Synology.
Rclone v1.48 is the current version. Your distribution may use an earlier version (e.g., Debian "Stable" uses version 1.35). However, you can install the latest version of Rclone directly with:
curl https://rclone.org/install.sh | sudo bash
You may have to install cURL first by typing:
sudo apt install curl
Creating Remotes
You control Rclone with the command of the same name (rclone
) and an attached command line including various options (see Table 1). To get started, call the configuration with rclone config
. The setup wizard offers a number of options: Use S
to set a password and Q
to quit the program.
Table 1
Important Rclone Commands
Command | Function |
---|---|
rclone config |
Opens the configuration |
rclone copy |
Copies data from the local system to cloud storage |
rclone sync |
Synchronizes source and target |
rclone mount |
Mount the drive locally into the filesystem |
rclone move |
Moves data from source to destination |
rclone delete |
Removes the contents of a path |
rclone purge |
Removes the path and its contents |
rclone ls |
Lists all objects in the path with a size specification |
rclone md5sum |
Creates md5sum checksums for all objects in the path |
rclone size |
Number and total size of all objects in <Remote>:<Path> |
rclone cryptcheck |
Checks the integrity of an encrypted remote |
These and other commands show their options after entering -h. |
Then use N
to set up the first remote connection ("remote" here refers to the connection to a specific folder on a specific cloud service) – select this option first. Since Rsync lets you configure multiple remotes, make sure you use a meaningful name in the first step. Then select the desired service or protocol using a number (Figure 1).
Google Drive Tested
In my hands-on test, I set up Rclone with Google Drive as a proprietary provider on a laptop. Of course, I do not want to store private data on Google's servers without encryption. Therefore, after an initial open test, I encrypted the data to be backed up before uploading with Rclone.
In the test, the remote repository is named gdrv
. You then need to select the corresponding number for the desired service (Google Drive here) from the list by entering 12
. For each provider and each protocol, the documentation offers a detailed manual [3], including Google Drive. The following two queries after client_id
and client_secret
can be skipped by pressing Enter.
Then decide whether Rclone should have full access to all files of the online repository or only access its own uploaded data. In addition, access can be restricted so that Rclone can only read. I selected full access for the test with the 1
option. Then skip the question about the root_folder_id
, as well as the question about the service_account_file
– Rclone can determine the root itself.
Can It Do That?
Finally, it doesn't hurt to look at the advanced configuration for Google Drive to learn about the additional options detailed in the documentation [4].
You are then taken to the point where Rclone can access your Google Drive account. To do this, say Y
to the auto config
prompt, and open your Google login in a browser window. After logging in, confirm that Rclone is allowed to access your drive (Figure 2). In the terminal, a Got Code message will appear; you can now close the browser again.
Having said this, the automatic method did not work for me. It was possible to register with Google, but the Rclone connection did not happen. A second attempt where I manually entered the URL, http://127.0.0.1:53682/auth, was then successful and let me complete this remote's configuration. To run Rclone in headless mode (i.e., without the option of opening a browser), create an API key in JSON format via a Google Service Account [5] and enter it in the advanced configuration.
This completes the setup wizard. You now have the choice of configuring additional remotes or protecting the configuration with a password. To do this, restart the configuration and select S
for "set configuration password". Instead of a password, you can also make the configuration file readable for only your user with:
chmod 600 ~/.config/rclone/rclone.conf
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Systemd Fixes Bug While Facing New Challenger in GNU Shepherd
The systemd developers have fixed a really nasty bug amid the release of the new GNU Shepherd init system.
-
AlmaLinux 10.0 Beta Released
The AlmaLinux OS Foundation has announced the availability of AlmaLinux 10.0 Beta ("Purple Lion") for all supported devices with significant changes.
-
Gnome 47.2 Now Available
Gnome 47.2 is now available for general use but don't expect much in the way of newness, as this is all about improvements and bug fixes.
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.
-
Armbian 24.11 Released with Expanded Hardware Support
If you've been waiting for Armbian to support OrangePi 5 Max and Radxa ROCK 5B+, the wait is over.
-
SUSE Renames Several Products for Better Name Recognition
SUSE has been a very powerful player in the European market, but it knows it must branch out to gain serious traction. Will a name change do the trick?
-
ESET Discovers New Linux Malware
WolfsBane is an all-in-one malware that has hit the Linux operating system and includes a dropper, a launcher, and a backdoor.
-
New Linux Kernel Patch Allows Forcing a CPU Mitigation
Even when CPU mitigations can consume precious CPU cycles, it might not be a bad idea to allow users to enable them, even if your machine isn't vulnerable.
-
Red Hat Enterprise Linux 9.5 Released
Notify your friends, loved ones, and colleagues that the latest version of RHEL is available with plenty of enhancements.
-
Linux Sees Massive Performance Increase from a Single Line of Code
With one line of code, Intel was able to increase the performance of the Linux kernel by 4,000 percent.