rdiff-backup

NAME

rdiff-backup - local/remote mirror and incremental backup

SYNOPSIS

rdiff-backup [options...] action [sub-options...] [locations...] rdiff-backup [--new] [-h|--help|-V|--version]

DESCRIPTION

rdiff-backup is a script, written in python(1) that backs up one directory to another. The target directory ends up a copy (mirror) of the source directory, but extra reverse diffs are stored in a special sub-directory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves symlinks, special files, hardlinks, permissions, uid/gid ownership, and modification times. rdiff-backup can also operate in a bandwidth efficient manner over a pipe, like rsync(1). Thus you can use ssh and rdiff-backup to securely back a hard drive up to a remote location, and only the differences will be transmitted. Using the default settings, rdiff-backup requires that the remote system accept ssh connections, and that rdiff-backup is installed in the user’s PATH on the remote system. See the REMOTE OPERATION section for details. Note that you should not write to the mirror directory except with rdiff-backup. Many of the increments are stored as reverse diffs, so if you delete or modify a file, you may lose the ability to restore previous versions of that file. Finally, this man page is intended more as a precise description of the behavior and syntax of rdiff-backup. New users may want to check out the examples file included in the rdiff-backup distribution. The rdiff-backup commands knows four types of parameters Note that this documents the new command line interface of rdiff-backup since 2.1+; for the traditional one, check rdiff-backup-old(1) but consider that it is deprecated and will disappear.

Options

-h, --help -V, --version --api-version apiversion --chars-to-quote, --override-chars-to-quote chars --current-time currenttime --force --fsync, --no-fsync --new, --no-new --null-separator --parsable-output --remote-schema remoteschema --remote-tempdir dirpath --ssh-compression, --no-ssh-compression --tempdir dirpath --terminal-verbosity { 0,1,2,3,4,5,6, 7,8, 9} --use-compatible-timestamps -v, --verbosity { 0,1,2,3,4,5,6, 7,8, 9}

Actions

backup [CREATION OPTIONS] [COMPRESSION OPTIONS] [SELECTION OPTIONS] [FILESYSTEM OPTIONS] [USER GROUP OPTIONS] [STATISTICS OPTIONS] sourcedir targetdir calculate [--method average] statfile1 statfile2 [...] complete [--cword index] [--unique|--no-unique] -- words [...] compare [SELECTION OPTIONS] [--method method] [--at time] sourcedir targetdir info list files [--changed-since time|--at time] repository list increments [--no-size|--size] repository regress [COMPRESSION OPTIONS] [USER GROUP OPTIONS] [TIMESTAMP OPTIONS] repository remove increments --older-than time [--size] repository restore [CREATION OPTIONS] [COMPRESSION OPTIONS] [SELECTION OPTIONS] [FILESYSTEM OPTIONS] [USER GROUP OPTIONS] [ --at time|--increment] source targetdir server [RESTRICT OPTIONS] [ --debug] test remote_location_1 [remote_location_2 ...] verify [ --at time] location

COMPRESSION OPTIONS

--compression, --no-compression --not-compressed-regexp regexp

CREATION OPTIONS

--create-full-path

FILESYSTEM OPTIONS

--acls, --no-acls --carbonfile, --no-carbonfile --eas, --no-eas --resource-forks, --no-resource-forks --hard-links, --no-hard-links --compare-inode, --no-compare-inode --never-drop-acls

RESTRICT OPTIONS

--restrict-path dirpath --restrict-mode { read-write,read-only,update-only}

SELECTION OPTIONS

This section only quickly lists the existing options, the section FILE SELECTION explains those more in details.

Globs, Regex, File lists selection

--include,--exclude glob --include-globbing-filelist,--exclude-globbing-filelist globsfile --include-globbing-filelist-stdin,--exclude-globbing-filelist-stdin --include-regexp,--exclude-regexp regexp --include-filelist,--exclude-filelist listfile --include-filelist-stdin,--exclude-filelist-stdin

Special files selection

--include-device-files,--exclude-device-files --include-fifos,--exclude-fifos --include-sockets,--exclude-sockets --include-symbolic-links,--exclude-symbolic-links --include-special-files,--exclude-special-files

Other selections

--include-other-filesystems,--exclude-other-filesystems --include-if-present,--exclude-if-present filename --max-file-size sizeinbytes --min-file-size sizeinbytes

STATISTICS OPTIONS

--file-statistics, --no-file-statistics --no-print-statistics, --print-statistics

TIMESTAMP OPTIONS

--allow-duplicate-timestamps

USER GROUP OPTIONS

See the USERS AND GROUPS section for more information. --group-mapping-file mapfile --user-mapping-file mapfile --preserve-numerical-ids

RESTORING

There are two ways to tell rdiff-backup to restore a file or directory: For example, suppose in the past you have run: to back up the '/usr' directory into the '/usr.backup' directory, and now want a copy of the '/usr/local' directory the way it was 3 days ago placed at '/usr/local.old'. One way to do this is to run: here above the '3D' means 3 days (for other ways to specify the time, see the TIME FORMATS section). The '/usr.backup/local' directory was selected, because that is the directory containing the current version of 'usr/local'. Note that the parameter of --at always specifies an exact time. (So '3D' refers to the moment 72 hours before the present). If there was no backup made at that time, rdiff-backup restores the state recorded for the previous backup. For instance, in the above case, if '3D' is used, and there are only backups from 2 days and 4 days ago, '/usr/local' as it was 4 days ago will be restored. The second way to restore files involves finding the corresponding increment file. It would be in the '/backup/rdiff-backup-data/increments/usr' directory, and its name would be something like 'local.2002-11-09T12:43:53-04:00.dir' where the time indicates it is from 3 days ago. Note that the increment files all end in '.diff', '.snapshot', '.dir', or '.missing', where '.missing' just means that the file didn’t exist at that time (finally, some of these may be gzip-compressed, and have an extra '.gz' to indicate this). Then running: would also restore the file as desired. If you are not sure exactly which version of a file you need, it is probably easiest to either restore from the increments files as described immediately above, or to see which increments are available with 'list increments', and then specify an exact time with --at.

TIME FORMATS

rdiff-backup uses time strings in two places. Firstly, all of the increment files rdiff-backup creates will have the time in their filenames in the w3 datetime format as described in a w3 note at <https://www.w3.org/TR/NOTE-datetime>. Basically they look like '2001-07-15T04:09:38-07:00', which is basically "{Year}-{Month}-{Day}T{Hours}:{Minutes}:{Seconds}{Timezone}", the time zone being 7 hours behind UTC in this example (hence the minus). Secondly, the --at, --changed-since, --older-than options take a time string, which can be given in any of several formats:

REMOTE OPERATION

In order to access remote files, rdiff-backup opens up a pipe to a copy of rdiff-backup running on the remote machine. Thus rdiff-backup must be installed on both ends. To open this pipe, rdiff-backup first splits the location into 'host_info::pathname'. It then substitutes 'host_info' into the remote schema, and runs the resulting command, reading its input and output. The 'host_info' can be anything understood as a destination by your version of SSH. Assuming it is the standard OpenSSH, it can be: The default remote schema is 'ssh -C {h} rdiff-backup --server' where 'host_info' is substituted for '{h}'. So if the 'host_info' is '[email protected]', then rdiff-backup runs 'ssh <[email protected]> rdiff-backup --server '. Using --remote-schema, rdiff-backup can invoke an arbitrary command in order to open up a remote pipe. For instance, is basically equivalent to (but slower than) Concerning quoting, if for some reason you need to put two consecutive colons in the 'host_info' section of a 'host_info::pathname' argument, or in the pathname of a local file, you can quote one of them by prepending a backslash. So in 'a\::b::c', 'host_info' is 'a::b' and the pathname is 'c'. Similarly, if you want to refer to a local file whose filename contains two consecutive colons, like 'strange::file', you’ll have to quote one of the colons as in 'strange\::file'. Because the backslash is a quote character in these circumstances, it too must be quoted to get a literal backslash, so 'foo\::\\bar' evaluates to 'foo::\bar'. To make things more complicated, because the backslash is also a common shell quoting character, you may need to type in '\\\\' at the shell prompt to get a literal backslash. You may also use the placehoders '{Vx}', '{Vy}' and '{Vz}' for the 'x.y.z' version of rdiff-backup, so that you can have multiple versions of rdiff-backup installed on the server, and automatically targeted from the client. For example, if you have rdiff-backup 2.1.5 and 2.2.1 installed in virtual environments on the server, respectively under '/usr/local/lib/rdiff-backup-2.0' and '/usr/local/lib/rdiff-backup-2.1' (we assume that the z-Version isn’t relevant to any kind of compatibility), then the client may be called with the following remote schema: The client will then use the correct version of rdiff-backup based on its own version 'x.y.z'. You’ll find more explanations in the migration file in the documentation. And finally, to include a literal '%' in the string specified by --remote-schema, quote it with another '%', as in '%%' (this is due to the compatibility with the deprecated host placeholder '%s', which you shouldn’t use anymore). Although ssh itself may be secure, using rdiff-backup in the default way presents some security risks. For instance if the server is run as root, then an attacker who compromised the client could then use rdiff-backup to overwrite arbitrary server files by "backing up" over them. Such a setup can be made more secure by using the sshd configuration option 'command="rdiff-backup server"' possibly along with the --restrict-path and --restrict-mode options to rdiff-backup. For more information, see the web page, the wiki, and the entries for those options on this man page.

FILE SELECTION

rdiff-backup has a number of file selection options. When rdiff-backup is run, it searches through the given source directory and backs up all the files matching the specified options. This selection system may appear complicated, but it is supposed to be flexible and easy-to-use. If you just want to learn the basics, first look at the selection examples in the examples file included in the package, or on the web at <https://rdiff-backup.net/docs/examples.html>. rdiff-backup’s selection system was originally inspired by rsync(1), but there are many differences. For instance, trailing backslashes have no special significance. All the available file selection conditions are listed under SELECTION OPTIONS. Two principles need to be understood before really starting: For example, the pattern 'bar' matches the path 'bar', but doesn’t match the path 'foo/bar' and neither the path './bar'. Both are matched by the pattern '*/bar', as well as by '**/bar'. This last pattern would match any path containing the file 'bar', e.g. 'foo/boz/bar'. Each file selection condition either matches or doesn’t match a given file. A given file is excluded by the file selection system exactly when the first matching file selection condition specifies that the file be excluded; otherwise the file is included. When backing up, if a file is excluded, rdiff-backup acts as if that file does not exist in the source directory. When restoring, an excluded file is considered not to exist in either the source or target directories. For instance, is exactly the same as because the include and exclude directives match exactly the same files, and the --include comes first, giving it precedence. Similarly, would backup the '/usr/local/bin' directory (and its contents), but not '/usr/local/doc'. The include, exclude, include-globbing-filelist, and exclude-globbing-filelist options accept extended shell globbing patterns. These patterns can contain the special patterns '*', '**', '?', and '[...]'. As in a normal shell, '*' can be expanded to any string of characters not containing '/', '?' expands to any character except '/', and '[...]' expands to a single character of those characters specified (ranges are acceptable). The new special pattern, '**', expands to any string of characters whether or not it contains '/'. Furthermore, if the pattern starts with 'ignorecase:' (case insensitive), then this prefix will be removed and any character in the string can be replaced with an upper or lowercase version of itself. If you need to match filenames which contain the above globbing characters, they may be escaped using a backslash '\'. The backslash will only escape the character following it so for '**' you will need to use '\*\*' to avoid escaping it to the '*' globbing character. Remember that you may need to quote these characters when typing them into a shell, so the shell does not interpret the globbing patterns before rdiff-backup sees them. The --exclude pattern option matches a file if and only if: Conversely, --include pattern matches a file if and only if: For example, matches '/usr/local', '/usr/local/lib', and '/usr/local/lib/netscape'. It is the same as And similarly: specifies that '/usr', '/usr/local', '/usr/local/lib', and '/usr/local/lib/netscape' (but not '/usr/doc') all be backed up. Thus you don’t have to worry about including parent directories to make sure that included subdirectories have somewhere to go. Finally, would match a file like '/usr/5fOO/hello/there/world.py'. If it did match anything, it would also match '/usr'. If there is no existing file that the given pattern can be expanded into, the option will not match '/usr'. The --include-filelist, --exclude-filelist, --include-filelist-stdin, and --exclude-filelist-stdin options also introduce file selection conditions. They direct rdiff-backup to read in a file, each line of which is a file specification, and to include or exclude the matching files. Lines are separated by newlines or nulls, depending on whether the --null-separator switch was given. Each line in a filelist is interpreted similarly to the way extended shell patterns are, with a few exceptions: For example, if the file 'list.txt' contains the lines: then '--include-filelist list.txt' would include '/usr', '/usr/local', and '/usr/local/bin'. It would exclude '/usr/local/doc', '/usr/local/doc/python', etc. It neither excludes nor includes '/usr/local/man', leaving the fate of this directory to the next specification condition. Finally, it is undefined what happens with '/var'. A single file list should not contain conflicting file specifications. The --include-globbing-filelist and --exclude-globbing-filelist options also specify filelists, but each line in the filelist will be interpreted as a globbing pattern the way --include and --exclude options are interpreted (although '+ ' and '- ' prefixing is still allowed). For instance, if the file 'globbing-list.txt' contains the lines: Then '--include-globbing-filelist globbing-list.txt' would be exactly the same as specifying on the command line: Finally, the --include-regexp and --exclude-regexp allow files to be included and excluded if their filenames match a python regular expression. Regular expression syntax is too complicated to explain here, but is covered in Python’s library reference. Unlike the --include and --exclude options, the regular expression options don’t match files containing or contained in matched files. So for instance matches any files whose full pathnames contain 7 consecutive digits which aren’t followed by 'foo'. However, it wouldn’t match '/home' even if '/home/ben/1234567' existed.

USERS AND GROUPS

There can be complications preserving ownership across systems. For instance the username that owns a file on the source system may not exist on the destination. Here is how rdiff-backup maps ownership on the source to the destination (or vice-versa, in the case of restoring): The user and group mapping files both have the same format: Each line should contain a name or id, followed by a colon ':', followed by another name or id. If a name or id is not listed, they are treated in the default way described above. When restoring, the above behavior is also followed, but note that the original source user/group information will be the input, not the already mapped user/group information present in the backup repository. For instance, suppose you have mapped all the files owned by alice in the source so that they are owned by ben in the repository, and now you want to restore, making sure the files owned originally by alice are still owned by alice. In this case there is no need to use any of the mapping options. However, if you wanted to restore the files so that the files originally owned by alice on the source are now owned by ben, you would have to use the mapping options, even though you just want the unames of the repository’s files preserved in the restored files. See USER GROUP OPTIONS for a list and description of related options.

FILES

any-config-file session_statistics, file_statistics backup.log, restore.log, error_log

ENVIRONMENT

RDIFF_BACKUP_VERBOSITY= [0-9] RDIFF_BACKUP_DEBUG=[ address][:port]

RETURN CODES

The following return codes have not been fully implemented so test before you rely on them. Also note that they can be combined, so that for example a return code 3 might be returned if a warning was found, then an error. 0 - OK 1 - ERROR 2 - WARNING 4 - FILE ERROR 8 - FILE WARNING

BUGS

See GitHub issues In doubt subscribe to and ask the mailing list

AUTHORS

SEE ALSO

python(1), rdiff(1), rsync(1), ssh(1). The main rdiff-backup web page is at <https://rdiff-backup.net/>. It has more documentation, links to the mailing list and source code.