rsync: sync and backup examples
rsync is a powerful tool for syncing or backup folders. This post describes the parameters I usually use.
rsync -a ~/MyStuff michael@devdeb2:/home/michael/MyStuff
-a => means -rlptgoD -r, --recursive recurse into directories -l, --links copy symlinks as symlinks -p, --perms preserve permissions -t, --times preserve modification times -g, --group preserve group --devices preserve device files (super-user only) --specials preserve special files -o, --owner preserve owner (super-user only) -D same as --devices --specials --devices preserve device files (super-user only) --specials preserve special files
other very usefull options:
-P same as --partial --progress --partial keep partially transferred files --progress show progress during transfer -u, --update skip files that are newer on the receiver -z, --compress compress file data during the transfer -v, --verbose increase verbosity --stats give some file-transfer stats # Full sync delete files in destination which not exist in source --delete delete extraneous files from destination dirs # On Fullsync just rename a file instead of delete them when it not exist in source -b --suffix=.bak --numeric-ids don't map uid/gid values by user/group name of source and target. Just copy the numerical user/group ids
And as a full command line. This example uses ssh as transport protocol and requires that rsync is installed an both, the source and the target machine.
michael@debdev ~# rsync -a -P -u -z --stats root@devdeb2:/srv /mnt/sdb2
rsync can also be used with a User/Password respectively a shared secret authentification without ssh.
On the target machine install and start rsync as a daemon.
root@debdev ~# apt-get install rsync root@debdev ~# systemctl enable rsync
Open respectively create file /etc/rsyncd.conf and add a “Module”. In this example customized monit files. Explanation: A module exports a directory by a symblic name. uid and gid are the credentials under which the rsync process runs. Means the user needs access to the path defined in the module.
path = /etc/monit/conf.d
comment = Monit
uid = root
gid = root
auth users = monit-replication
secrets file = /etc/rsyncd.secrets
On your source machine. Create also a /etc/rsyncd.secrets file with just the password as content.
root@debdev2 ~# chown root:root /etc/rsyncd.secrets root@debdev2 ~# chmod 400 /etc/rsyncd.secrets
To run rsync without prompting for the password specify the –password-file=/etc/rsyncd.secrets parameter or set the Environment variable RSYNC_PASSWORD
root@debdev2 ~# export RSYNC_PASSWORD=MySecretPassword
Check folder access
root@debdev2 ~# rsync --password-file=/etc/rsyncd.secrets rsync://monit-replication@debdev/monit
And get the folder
root@debdev2 ~# rsync -a -P -u -z --stats --password-file=/etc/rsyncd.secrets rsync://monit-replication@debdev/monit /tmp
Filters. Note: Default is to includes all. This example copies all pdf files. Importent is to set the include before the exclude filter because the first match determines if a file is copied or not.
root@debdev2 ~# rsync -a -P -u -z --stats --include '*.pdf' --exclude '*' --password-file=/etc/rsyncd.secrets rsync://monit-replication@debdev/monit /tmp --include '*.pdf' --exclude '*'