Process to rsync only new files in Linux

Remote Linux Server Administrator
Server Management Service
381

Are you trying to use rsync to transfer only new files?

This guide is for you.

Rsync, which stands for “remote sync”, is a remote and local file synchronization tool. It uses an algorithm that minimizes the amount of data copied by only moving the portions of files that have changed.

Here at Ibmi Media, as part of our Server Management Services, we regularly help our Customers to perform rsync related tasks.

In this context, you shall learn how to transfer files using rsync.

More Information about rsync ?

Rsync (Remote Sync) tool helps in transferring and synchronize files locally and remotely. It is a great tool popularly used for backup operations and mirroring.

It is highly flexible that it allows users to specify any number of files to copy.

Also, it permits copying of links, devices, file or directory owners, groups, and permissions. It also supports users without root privileges.

The general syntax of rsync is as below:

# rsync options source destination

How we use rsync to transfer only new files Locally ?

Now you will see how we can transfer the files using rsync.

Here is the command to copy files from the Documents directory to /tmp/documents directory locally.

$ rsync -av Documents/* /tmp/documents

In the above command;

-a – means archive mode

-v – means verbose, showing details of ongoing operations

By default, rsync will only copy new or changed files from a source to a destination.

The –update or -u option is used by rsync to skip files that are still new in the destination directory.

Also, –dry-run or -n enables us to execute a test operation without making any changes. It shows us what files we need to copy.

$ rsync -aunv Documents/* /tmp/documents

After executing a test run, we can then do away with the -n and perform a real operation:

$ rsync -auv Documents/* /tmp/documents

How we use rsync to transfer only new files From Local to Remote Linux ?

Here is the command to copy files from local machine to a remote server with the IP address 1x.xx.1.x. Also, this command will only sync new files on the local machine, that do not exist on the remote machine, we can include the –ignore-existing option.

$ rsync -av –ignore-existing Documents/* user@1x.xx.1.x:~/all/

Moreover, to sync only updated or modified files on the remote machine that have changed on the local machine, we can perform a dry run before copying files as below.

$ rsync -av –dry-run –update Documents/* user@1x.xx.1.x:~/all/
$ rsync -av –update Documents/* user@1x.xx.1.x:~/all/

Here, the –update will force rsync to skip any files that exist on the destination file and have a modified time that is newer than the source file. If an existing destination file has a modification time equal to the source file, it will be updated if the sizes are different.

Also, to update the existing files and prevent the creation of new files in the destination, we utilize the –existing option.