backup_with_rsync
This is an old revision of the document!
Backup with rsync
So, this is what I've been doing to make backups of machines for years, it stores permissions, and you can export the /Backup folder via NFS, then remap it to another folder as read-only so users can do their own restores if you want (not going into that here).
This is what I see in my /Backup folder with;
ls -Al /Backup.
total 64 drwxr-xr-x 12 root root 4096 2026-02-04+15:20:02 00-EFI -rw------- 1 root root 613 2024-08-27+11:29:00 00-EFI.conf drwxr-xr-x 12 root root 4096 2026-02-04+15:23:10 01-Root -rw------- 1 root root 605 2024-08-27+11:29:15 01-Root.conf drwx------ 2 root root 16384 2024-06-27+11:37:59 lost+found -rwx------ 1 root root 4489 2024-06-27+13:18:01 Snapshot -rwx------ 1 root root 333 2024-06-27+13:17:20 SystemBackup
Below is a short script that just runs the actual backup script with different sets of options for what goes where;
sudo cat /Backup/SystemBackup
- /Backup/SystemBackup
#!/bin/bash # make sure we're running as root if (( 0 != 0 )); then { echo "Sorry, must be root. Exiting..."; exit; } fi; # Run a backup for every .conf file in the /Backup directory cd /Backup for Script in *.conf do ./Snapshot ${Script} done # And send an e-mail with the list of Summaries; cat /Backup/*/Summary
The .conf script below that gets sourced by the backup script below that;
sudo cat /Backup/01-Root.conf
- /Backup/01-Root.conf
# =-=-=-=-=-=- File locations and variables. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # Where to write the backups SNAPSHOT_RW=/Backup/01-Root; # What directory to back up, begins and ends in / SNAP_DIR="/"; # Where to store the list of files and directories that we backed up. FILELIST=$SNAPSHOT_RW/Filelist # Summary file, a short list of actions taken by rsync to complete the backup. SUMMARY=$SNAPSHOT_RW/Summary # Stuff not to back up, there is always stuff to not back up! EXCLUDES=$SNAPSHOT_RW/Excludes; # How many snapshot backups do we want to keep? Must be at least 2! NUM_SNAP=10;
Below is the actual script that calls rsync to do the backup, into a unique folder-by-date.
sudo cat /Backup/Snapshot
- /Backup/Snapshot
#!/bin/bash # =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # 2017-01-06-SRJ # For information on how this script works, and what it does, read the # file README in this folder # =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # # suggestion from H. Milz: avoid accidental use of $PATH unset PATH # # =-=-=-=- System commands used by this script. -=-=-=-=-=-=-=-= ID=/usr/bin/id; ECHO=/bin/echo; MOUNT=/bin/mount; RM=/bin/rm; MV=/bin/mv; CP=/bin/cp; TOUCH=/bin/touch; RSYNC=/usr/bin/rsync; DATE=/bin/date; WC=/usr/bin/wc; CAT=/bin/cat; TAIL=/usr/bin/tail; HEAD=/usr/bin/head; GREP=/bin/grep; CHMOD=/bin/chmod # =-=-=-=-=-=-=- Make sure we're running as root -=-=-=-=-=-=-=-=-= if (( `$ID -u` != 0 )); then { $ECHO "Sorry, must be root. Exiting..."; exit; } fi; # =-=-=-=-=-=-=-=-=-=- Read in the Variables -=-=-=-=-=-=-=-=-=-=-= if [ ! -e $1 ] ; then { $ECHO "No viable config file given."; exit 0 } fi; # If we're here, there must be a config file to read, the line below # reads the variables from that file into this bash environment. . $1 # # # Update 2017-01-01 SRJ The line above sources the variables from # # # the config file into this script - see /Backup/00-System.conf # # # for more information. # Make sure that the folder list exists, create it if needed; if [ ! -f $SNAPSHOT_RW/Backups ] ; then { $ECHO Nothing >> $SNAPSHOT_RW/Backups } fi; # The full name of the oldest snapshot, first line in Backups [ $($CAT $SNAPSHOT_RW/Backups | $WC --lines) -ge $NUM_SNAP ] && \ OLDEST=$($HEAD -1 $SNAPSHOT_RW/Backups) || OLDEST="Nothing" # The full name of the last snapshot, last line in Backups LAST=$($TAIL -1 $SNAPSHOT_RW/Backups) # The full name of the snapshot for this itteration. (2006-06-22-09-47) CURRENT=$SNAPSHOT_RW/$($DATE +%F-%H-%M) # What time is it, in seconds past the epoch? StartTime=$($DATE +%s) # Hour, minute, second function; _hms() { local S=${1} # this S is used only in this sub-routine ((h=S/3600)) # fills h with the integer of seconds / 3600 ((m=S%3600/60)) # fills m with the integer of the modula from above / 60 ((s=S%60)) # fills s with the integer of seconds / 60 printf "%02d:%02d:%02d\n" $h $m $s # print 2 digit zero padded numbers } # =-=-=-=-=-=-=-=-=- The script itself. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= # I stopped mounting and unmounting the filesystem, I now do this by # hand on an as-needed basis - these backup scripts are now on the # backup drives themselves. # Attempt to remount the RO mount point as RW; else abort # $MOUNT -o remount,rw $MOUNT_DEVICE $MOUNT_DIR ; # if (( $? )); then { # $ECHO "Could not remount $MOUNT_DIR read-write"; exit; # } fi; # Creating the snapshot. # Delete the oldest snapshot, if it exists: if [ -e ${OLDEST} ] ; then { $RM -rf ${OLDEST} ;} fi ; # Delete the last filelist and create a new one with proper permissions. if [ -f $FILELIST ] ; then { $RM -f $FILELIST ;} fi ; $TOUCH $FILELIST ; $CHMOD 600 $FILELIST # Rsync from the system into the latest snapshot (notice that rsync # behaves like cp --remove-destination by default, so the destination is # unlinked first. If it were not so, this would copy over the other # snapshot(s) too! RSARGS=" --archive --verbose --delete --delete-excluded --numeric-ids \ --modify-window=1 --hard-links --one-file-system --acls --xattrs \ --exclude-from="$EXCLUDES" --link-dest=$LAST/ " # This is the line that does everything we're doing here... $RSYNC $RSARGS $SNAP_DIR $CURRENT >> $FILELIST; # Add today's snapshot to the list, and remove the oldest; $ECHO $CURRENT >> $SNAPSHOT_RW/Backups $GREP -v $OLDEST $SNAPSHOT_RW/Backups >> $SNAPSHOT_RW/$$.tmp $MV -f $SNAPSHOT_RW/$$.tmp $SNAPSHOT_RW/Backups EndTime=$($DATE +%s) ; ElapsedTime=$(( $EndTime - $StartTime )) TimeWellSpent=$( _hms ${ElapsedTime}) # Write out the summary file. $HEAD -2 $FILELIST > $SUMMARY; $ECHO $(( $($GREP -ve /$ $FILELIST | $WC --lines )-5)) \ New files backed up. >> $SUMMARY $ECHO $OLDEST was removed and $CURRENT was created >> $SUMMARY $ECHO Elapsed time: $TimeWellSpent >> $SUMMARY $TAIL -3 $FILELIST >> $SUMMARY $MV $FILELIST $CURRENT # And thats it for SNAP_DIR. Now remount the RW snapshot mountpoint as RO # $MOUNT -o remount,ro $MOUNT_DEVICE $MOUNT_DIR ; # if (( $? )); then { # $ECHO "Could not remount $MOUNT_DIR readonly"; exit; # } fi;
The /Backup/01-Root/Excludes file is important, there's always stuff you don't want backed up (some things can't be as they recursively call themselves) and this is a list that has been created over years of 'what the heck happened' moments;
sudo cat /Backup/01-Root/Excludes
- /Backup/01-Root/Excludes
/Backup/* /home/Shared/Backup */Cache/* /dev/* /flows/* /initrd* /home/*/.gvfs */lost+found* /media/* /mnt/* /run/* /tmp/* /proc/* /var/virus* /var/lib/vmware* /var/run/cups/certs* /var/lib/docker/*
And, what's in the partition backup folder;
ls -Al /Backup/01-Root/
total 52 drwxr-xr-x 44 root root 4096 2026-01-26+15:21:57 2026-01-26-20-20 drwxr-xr-x 44 root root 4096 2026-01-27+15:22:44 2026-01-27-20-20 drwxr-xr-x 44 root root 4096 2026-01-28+15:22:39 2026-01-28-20-20 drwxr-xr-x 44 root root 4096 2026-01-29+15:22:20 2026-01-29-20-20 drwxr-xr-x 44 root root 4096 2026-01-30+15:21:58 2026-01-30-20-20 drwxr-xr-x 44 root root 4096 2026-01-31+15:22:42 2026-01-31-20-20 drwxr-xr-x 44 root root 4096 2026-02-01+15:23:43 2026-02-01-20-20 drwxr-xr-x 44 root root 4096 2026-02-02+15:22:12 2026-02-02-20-20 drwxr-xr-x 44 root root 4096 2026-02-03+15:25:32 2026-02-03-20-20 drwxr-xr-x 44 root root 4096 2026-02-04+15:23:10 2026-02-04-20-20 -rw-r--r-- 1 root root 330 2026-02-04+15:23:10 Backups -rw------- 1 root root 199 2024-09-02+19:06:02 Excludes -rw-r--r-- 1 root root 349 2026-02-04+15:23:10 Summary
The Summary file is a quick recap of what happened during the last backup;
sudo cat /Backup/01-Root/Summary
sending incremental file list created directory /Backup/01-Root/2026-02-04-20-20 13145 New files backed up. /Backup/01-Root/2026-01-25-20-20 was removed and /Backup/01-Root/2026-02-04-20-20 was created Elapsed time: 00:03:08 sent 2,314,795,346 bytes received 461,750 bytes 28,063,722.38 bytes/sec total size is 264,243,317,875 speedup is 114.13
backup_with_rsync.1770309023.txt.gz · Last modified: by steve