Protecting TimeMachine backups from itself

Going down the time machine rabbit hole…

I love the fact that MacOS comes with TimeMachine built-in, and I also really appreciate its simplicity. It makes backups easy and accessible even for non-technical people. It gets messy though if you also want to have real offsite backups however.

TimeMachine works great with a USB external HD, but things get tricky over the network.

I own a small Synology NAS, and I managed to mount a TimeMachine volume and get it to backup to that volume. The problem started when the volume size started to grow. I could set a quota on the volume, but for some strange reason, when the quota is reached, TimeMachine just started failing without a clear reason. There’s no way to tell TimeMachine to only keep X versions, or keep disk storage below a certain threshold. It’s supposed to prune backups automatically, but seems to fail with my network volume.

tmutil to the rescue

After a bit of digging, I discovered a useful command-line utility that helps managing TimeMachine. tmutil lets you list your backups and also delete specific backups. I could manually easily prune the oldest backup of TimeMachine using tmutil listbackups | head -1 | xargs -I {} sudo tmutil delete "{}".

How difficult could it be to automate this script so it runs regularly and keeps X copies?

The script

The script itself was pretty simple. Please don’t bash my bash skills, but I hope the code is clear and seems to do the job

#!/bin/bash

# keeps backups for up to 7 days
KEEP=7

function timestamp() {
    date -jf '%F-%H%M%S' "$1" '+%s'
}

LAST_BACKUP=$(/usr/bin/tmutil listbackups | /usr/bin/tail -n1)
LAST_BACKUP_DATE=$(basename "$LAST_BACKUP")
LAST_TIMESTAMP=$(timestamp $LAST_BACKUP_DATE)

OLDEST_BACKUP=$(/usr/bin/tmutil listbackups | /usr/bin/head -n1)
OLDEST_BACKUP_DATE=$(basename "$OLDEST_BACKUP")
OLDEST_TIMESTAMP=$(timestamp $OLDEST_BACKUP_DATE)

DIFF=$(( ($LAST_TIMESTAMP - $OLDEST_TIMESTAMP) / (24*3600) ))

while [[ $DIFF -gt $KEEP ]]; do
    echo "cleaning"
    sudo tmutil delete "$OLDEST_BACKUP"
    OLDEST_BACKUP=$(/usr/bin/tmutil listbackups | /usr/bin/head -n1)    
    OLDEST_BACKUP_DATE=$(basename "$OLDEST_BACKUP")
    OLDEST_TIMESTAMP=$(timestamp $OLDEST_BACKUP_DATE)
    DIFF=$(( ($LAST_TIMESTAMP - $OLDEST_TIMESTAMP) / (24*3600) ))
done

Easy-peasy, right? There were two problems left to solve:

1. run this script on schedule, let’s say once a day or once a week
2. make sure we can run this as root, so we don’t need to use sudo (or avoid the password prompt if we do run sudo)

How difficult can it be?

Running scheduled jobs on MacOS is quite a bit different from Linux. You could still somehow use a cron job, but it’s not recommended, might be deprecated, and generally not the “right way” to do things. You are supposed to create a launchd script… Ok, so without digging too deep, here’s a simple launchd file for running a command every 24 hours. You need to place this file under ~/Library/LaunchAgents/ and give it a name like org.whatever.script-name.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <!-- The label should be the same as the filename without the extension -->
    <string>org.whatever.script-name</string>
    <!-- Specify how to run your program here -->
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/code/timemachine-cleanup.sh</string>
    </array>
    <!-- Run every 24 hours -->
    <key>StartInterval</key>
    <integer>86400</integer><!-- seconds -->
</dict>
</plist>

Then you would need to run this command, so the launchd script gets scheduled

launchctl load org.whatever.script-name.plist

note: if you need to stop your script from running, you can use the same command, but with unload instead to unload it.

First problem: Run as root

The first problem is that this job now runs as your own user id. This is fixable if you move the file to the global /Library/LaunchAgents, make the owner of the file root and use sudo to load it. It runs as root, but tmutil now fails. Why? because some tools/operations like tmutil require something called Full Disk Access (FDA).

Second problem: Full Disk Access

Here’s an interesting post explaining why some privileged commands are not allowed on MacOS, and a few problems/workarounds being suggested on StackOverflow’s Ask Different.

You can get those commands to run from your terminal, if you add your terminal to the list of allowed apps with Full Disk Access

The problem however, is that for some strange reason, you cannot simply add your script to the list. Or you could add it, but it will still be blocked. I have no clue why.

Workaround: wrap your script as an app

You could use Automator.app or Script Editor to do that. You can then add this wrapper app to give it Full Disk Access permissions. That works, but then there’s no way to make it run as root. So we’re back to the first problem again… :-/

Workaround #2: sudoers with NOPASSWD

So we won’t run it as root, but use sudo. Then we’re prompted for a password, which we tried to avoid for a scheduled job.

We can however add any specific command to be allowed to run with sudo without a password. We’ll run sudo visudo and then edit the file to add this line:

myuser ALL=(ALL) NOPASSWD: /usr/bin/tmutil

If you want to be even more secure, you can include the sha256 hash of tmutil. First get the hash by running sha256sum /usr/bin/tmutil and then adding this sha to your sudoers file. Mine looks like this

myuser ALL=(ALL) NOPASSWD: sha256:57a753bd2bef425205684630a676765913e1adca7ec0a9d73c205e4da32488c6 /usr/bin/tmutil

Alternatives to app wrapper

One thing I noticed with the wrapper app is that when it launches, it’s visible on my desktop. It could even grab my input for a moment, which is slightly annoying. It also felt weird to have a whole app just to wrap a script to give it Full Disk Access.

One alternative mentioned on the linked StackOverflow page above is to compile a binary app to wrap your shell script. I didn’t try it, since I am not using any compiled languages regularly.

Another thing I did try and will probably end up using is launching the script via the terminal. How? I already have passwordless (public key) SSH remote login access set up on my Mac. So I simply modified the launchd script slightly so it launches my script via ssh

<key>ProgramArguments</key>
<array>
    <string>ssh</string>
    <string>myuser@myhost.local</string>
    <string>/path/to/code/timemachine-cleanup.sh</string>
</array>

Since terminal is already set with FDA, this seems to work :)

It’s a bit of an awkward workaround to ssh to your own host with your own user in order to run a script, but it’s not that much different from compiling a binary to wrap your script, or wrapping it in an “app”.

Why TimeMachine?

I guess some people might rightfully question why I’m using TimeMachine and not a more flexible backup tool for Mac. I did consider it and tested a couple of options like CarbonCopyCloner and others. As far as I could tell, none of the commercial backup tools can create a full machine image on a remote network storage like TimeMachine does. If you know of something that does this, please let me know!

… and for those backup freaks out there: No. the Synology volume isn’t my only backup copy. I then also run restic to clone the TimeMachine volume to Backblaze B2 for offsite safekeeping.