How to use rsync with NTFS and FAT32 on Linux: Problems and solutions

Introduction

When using rsync on NTFS or FAT32 partitions, it is necessary to consider the particularities of these file systems in relation to those used in Unix-based systems. In Unix environments, it's common to use the -a as a parameter for the rsync command, which is equivalent to the set of options -rlptgoD:

The mentioned parameters have the following functions:

-r Copies subdirectories recursively.
-l Copies symbolic links as links.
-p Preserves permissions.
-t Preserves timestamps.
-g Preserves the group.
-o Preserves the owner.
-D Transfers special and block device files (e.g., sda, tty).

For detailed information about using the rsync command on Linux, see:

Of the listed parameters, the file systems used in Windows fully support only recursive copying -r. NTFS and FAT32 do not support the options -p, -g, -o, and -D. Although recent versions of NTFS allow the use of symbolic links, they are not compatible with the Unix style, which may cause the -l option to fail.

Preservation of timestamps -t is supported, but in some cases, differences in precision may occur. NTFS uses a precision of 100 nanoseconds, while in Unix-based systems, this precision can vary from 1 second to microseconds.

Between more modern Unix file systems like ext4 and XFS, and NTFS, there should be no issues with timestamp verification. However, when dealing with FAT32, the discrepancy becomes more significant. This can lead rsync to interpret identical files as different due to variations in the modification date.

There are options to work around these limitations, as will be demonstrated below.

Environment

The commands used in this article were tested on Slackware Linux 15, with kernel 5.15.161, and should work on any Linux or Unix-based system. The root file system was formatted with ext4, while NTFS and FAT32 partitions were used on other disks within the same system.

In the tests, NTFS partitions were mounted both with ntfs-3g, a widely used user-space driver for years, and with the native NTFS kernel driver, introduced in version 5.15. The behavior of rsync was the same in both cases.

Using rsync with NTFS

In more recent versions of the file systems used in Linux and NTFS, rsync should work normally both between Linux and NTFS partitions and between NTFS partitions, even when using the -a parameter, which includes options incompatible with NTFS, as discussed earlier.

This happens because, when mounting an NTFS partition, Linux simulates file permissions. Thus, unsupported parameters are ignored, and the files receive owner, group, and permissions as defined during mounting.

It is essential that the partitions are mounted with appropriate permissions for the user who will execute the rsync command. If the drive is mounted with ownership by the root user, even if other users have read and write permissions, rsync will present an error when trying to set the timestamp:

failed to set times on "/destination/folder/": Operation not permitted (1)

This error will prevent rsync from correctly verifying whether the files at the destination are identical to those at the source, resulting in redundant copying of files each time the command is executed.

To bypass this error, the partition must be mounted with appropriate permissions. An example configuration in the /etc/fstab file is:

/etc/fstab

/dev/sdXn   /mnt/partition   ntfs3   fmask=022,dmask=022,uid=1000,gid=100

Where:

uid=1000 Replace 1000 with the ID of the user.
gid=100 Replace 100 with the ID of the user's group.
fmask=022 Grants full permission to the file owner and read and execute permissions for group and other users.
dmask=022 Grants full permission to the directory owner and read and execute permissions for group and other users.

With the correct permissions, rsync can be used with conventional options:

$ bash

rsync -av /source/ /destination/

After the first execution and copying the files to the destination, run the command again to check if the files are copied on the second attempt. If they are, rsync failed to compare timestamps. If everything is correct, no files should be transferred on the second execution.

If the command fails due to permissions or the use of older drivers, you can run it using only the parameters compatible with NTFS:

$ bash

rsync -rtv /source/ /destination/

In this example, rsync is executed with the following options:

-r Enables recursive mode.
-t Preserves the timestamp.
-v Verbose.

Use the --modify-window parameter

NTFS may show small differences in the accuracy of timestamps compared to Unix-based systems. To work around this limitation, use the --modify-window parameter, which allows a margin of error when comparing timestamps. For example:

$ bash

rsync -rtv --modify-window=1 /source/ /destination/

This tells rsync to consider timestamps equal if the difference between them is up to 1 second.

Using rsync with FAT32

FAT32 is limited in terms of timestamps, with low precision, does not store access times, and has timezone issues. These characteristics make it unsuitable for scenarios where temporal precision is crucial, such as backups or file synchronization.

To run rsync with FAT32 preserving timestamps, use the command:

$ bash

rsync -rtv --modify-window=1 /source/ /destination/

The --modify-window=1 option instructs rsync to consider timestamps equal if the difference between them is up to 1 second. If the command doesn't work with this value, try using 2 seconds. Always run the command again to check if the files will be transferred on the second execution; they will only be copied if rsync fails to verify the timestamp.

Use the --size-only parameter

If you don't need to preserve timestamps and want to avoid problems related to precision, use the --size-only option. This makes rsync consider only the file size when deciding if a file should be copied.

$ bash

rsync -rv --size-only /source/ /destination/

Note that the -t and --modify-window parameters have been removed from the command since they are used to handle timestamps and are incompatible with the --size-only option, which disables timestamp checking.

Conclusion

When using rsync with NTFS or FAT32 file systems, it is crucial to pay attention to partition mounting options, as permissions and file properties are limitations of these systems. Parameters like --modify-window and --size-only help work around timestamp precision issues and ensure more efficient synchronization.

References

The manual (man page) for rsync was consulted in preparing this article.

Site Notes

This is not my original language and I don't speak it very well. I used my little knowledge and translators tools to compose the text of this article. Sorry for possible spelling or grammatical errors, suggestions for corrections are appreciated and can be sent to the contact email in the footer of the site. My intention is to share some knowledge and I hope this translation is good enough.

Some of the content on this website, including text, images, graphics and other materials, may be generated or enhanced by artificial intelligence (AI) tools. For more details on the use of AI, please see our Term of Use.