How to delete all but last [n] ZFS snapshots?

Question

I'm currently snapshotting my ZFS-based NAS nightly and weekly, a process that has saved my ass a few times. However, while the creation of the snapshot is automatic (from cron), the deletion of old snapshots is still a manual task. Obviously there's a risk that if I get hit by a bus, or the manual task isn't carried out, the NAS will run out of disk space.

Does anyone have any good ways / scripts they use to manage the number of snapshots stored on their ZFS systems? Ideally, I'd like a script that iterates through all the snapshots for a given ZFS filesystem and deletes all but the last n snapshots for that filesystem.

E.g. I've got two filesystems, one called tank and another called sastank. Snapshots are named with the date on which they were created: sastank@AutoD-2011-12-13 so a simple sort command should list them in order. I'm looking to keep the last 2 week's worth of daily snapshots on tank, but only the last two days worth of snapshots on sastank.

Answer 1

You may find something like this a little simpler

zfs list -t snapshot -o name | grep ^tank@Auto | tac | tail -n +16 | xargs -n 1 zfs destroy -r

• Output the list of the snapshot (names only) with zfs list -t snapshot -o name
• Filter to keep only the ones that match tank@Auto with grep ^tank@Auto
• Reverse the list (previously sorted from oldest to newest) with tac
• Limit output to the 16th oldest result and following with tail -n +16
• Then destroy with xargs -n 1 zfs destroy -vr

Deleting snapshots in reverse order is supposedly more efficient or sort in reverse order of creation.

zfs list -t snapshot -o name -S creation | grep ^tank@Auto | tail -n +16 | xargs -n 1 zfs destroy -vr

Test it with ...|xargs -n 1 echo.

Answer 2

This totally doesn't answer the question itself, but don't forget you can delete ranges of snapshots.

zfs destroy zpool1/dataset@20160918%20161107

Would destroy all snapshots from "20160918" to "20161107" inclusive. Either end may be left blank, to mean "oldest" or "newest". So you could cook something up that figures out the "n" then destroy "...%n"..

Sorry to resurrect an old question.

Answer 3

More general case of getting most recent snapshot based on creation date, not by name.

zfs list -H -t snapshot -o name -S creation | head -1

Scoped to a specific filesystem name TestOne

zfs list -H -t snapshot -o name -S creation -d1 TestOne | head -1

-H:No header so that first line is a snapshot name

-t snapshot: List snapshots (list can list other things like pools and volumes)

-o name: Display the snapshot name property.

-S creation: Capital S denotes descending sort, based on creation time. This puts most recent snapshot as the first line.

-d1 TestOne: Says include children, which seems confusing but its because as far as this command is concerned, snapshots of TestOne are children. This will NOT list snapshots of volumes within TestOne such as TestOne/SubVol@someSnapshot.

| head -1: Pipe to head and only return first line.

Answer 4

You might also want to check out zfs-prune-snapshots.

Remove snapshots from one or more zpools that match given criteria

It has a fairly robust time based mechanism for deleting snapshots, an example from the docs:

Remove snapshots older than two months on the tank pool that end with the string "_frequent"

zfs-prune-snapshots -s '_frequent' 2M tank

Answer 5

growse's didn't work on OpenIndiana for me. It didn't understand -0 for xargs.

If using sort, be aware that it sorts alphabetically which may not be desired as you are probably wanting to find the most recent.

Here is code that will delete all but the last snapshots.

Remove the 'echo' to go live.

RETENTION=5
FS=tank1/test
SNAPNAME=daily-

zfs list -t snapshot -o name | grep ^$FS@${SNAPNAME} |  sed -n -e :a -e '1,${RETENTION}!{P;N;D;};N;ba' | xargs -n 1 echo zfs destroy -r

Sources: http://sed.sourceforge.net/sed1line.txt

Answer 6

I may have solved this with some bash-fu.

 zfs list -t snapshot -o name | grep ^tank@AutoD- | sort -r | wc -l | xargs -n 1 expr -$NUM_TO_KEEP + | tr -d '\n' | xargs -0 -i bash -c "zfs list -t snapshot -o name | grep ^tank@AutoD- | sort -r | tail -n{} | sort |xargs -t -n 1 zfs destroy -r"

Wow. It feels so wrong.

Answer 7

Just wanted to chime in on how I'm doing this on FreeBSD and OmniOS:

Get number of snapshots:

zfs list -t snapshot -o name | grep ^tank@Auto | wc -l
141

Subtract number you want to leave for n (e.g. 30 for a month of latest daily snapshots):

zfs list -t snapshot -o name | grep ^tank@Auto |  head -n +111 | xargs -n 1 zfs destroy -vr

Note how I replaced tail with head to delete in order from oldest to newest, since there's no tac command on FreeBSD

That's it! Works great for me...

Answer 8

The path for head is needed on Solaris, but should work without the path on other distros.

retention=14
dataset=vmstorage-17/824

zfs list -rt snap -H -o name ${dataset} | \
/usr/gnu/bin/head -n -${retention} | xargs -n 1 zfs destroy -r

Answer 9

Identify the latest two snapshots for a given dataset (creation, newest to oldest)

zfs_latest=`zfs list -H -t snapshot -o name -S creation | grep ^tank/example_dataset@ | head -2`

Identify ALL snapshots for a given dataset (creation, newest to oldest)

zfs_delete=`zfs list -H -t snapshot -o name -S creation | grep ^tanks/example_dataset@`

Remove latest snapshots from all set; creating a list of snapshots to delete.

for keep_snap in ${zfs_latest[@]}; do
  zfs_delete=( "${zfs_delete[@]/$keep_snap}" );
done

Remove the old snapshots

for snap in ${zfs_delete[@]}; do
  zfs destroy ${snap}
done

Answer 10

If someone wants to do recursive snapshots of the entire pool's datasets, I wrote this script to do just that:

#!/bin/bash

#Method to purge old snapshots. $1 is the dataset that snapshots are being purged for. $2 is the number of snapshots to retain.
purge_snapshots() {
  echo -e "\n  Purging snapshots for: $1"

  #We search for snapshots and return jsut the name sorted by creation time. We filter with grep just for snapshots created with this script with prefix "$1@auto"
  snapshots=$(zfs list -t snapshot -H -o name -S creation "$1" | grep ^"$1"@auto)
  if [[ -z "$snapshots" ]]; then
    echo -e "\n    No snapshots found for dataset"
    return 0
  fi
  echo -e "\n    Existing snapshots for $1:\n"
  echo "$snapshots"

  #We use "tail -n +" to skip the first $2 snapshots. We need + 1 to begin after that many snapshots. This gives us the list of snapshots to purge
  old_snapshots=$(echo "$snapshots" | tail -n +"$(($2 + 1))")
  if [[ -z "$old_snapshots" ]]; then
    echo -e "\n    Not enough old snapshots too purge"
    return 0
  fi
  echo -e "\n    Deleting the following old snapshots:\n"
  echo "$old_snapshots"
  echo "$old_snapshots" | xargs -n 1 zfs destroy -r
}

#Function to create snapshots. it receives the list of snapshots in $1 and retention count in $2.
create_snapshots() {
  formatted_date="$(date +"%Y-%m-%d_%H-%M-%S")"

  #We loop the list of datasets and create and purge snapshots for each of them with the same timestamp. this way all child datasets will have the same timestamp and retention count as the root dataset.
  while read -r dataset; do
    echo -e "\nCreating snapshot for: $dataset"
    snapshot_name="${dataset}@auto-${formatted_date}"
    zfs snapshot "$snapshot_name"
    echo -e "\n  Snapshot created: $snapshot_name"
    purge_snapshots "$dataset" "$2"
  done <<< "$1"
}

#We validate the input arguments. $1 is the pool name and $2 is the retention count.
if [ "$#" -ne 2 ]; then
  echo "Usage: $0 poolname retentioncount"
  exit 1
fi

if [[ -z "$1" ]]; then
  echo "poolname cannot be empty"
  exit 2
fi

if [[ "$2" -le 0 ]]; then
  echo "retentioncount must be a positive number"
  exit 3
fi

#We get the list of data sets in the pool recursively so that we include every single child dataset in the pool.
datasets=$(zfs list -H -o name -r "$1")
echo -e "Datasets in pool $1:\n"
echo "$datasets"
create_snapshots "$datasets" "$2"

Answer 11

Here is my version of Dan Buhler's script:

\# Ref: https://manpages.ubuntu.com/manpages/kinetic/man8/zfs-destroy.8.html

\# To get a list of datasets:
\# zfs list -t snapshot -o name -S creation

\# To get a list of memory usage:
\# zfs list -r -o space

\# The number of the most recent snapshots to be retained:
RETENTION=5

\# The datasets to prune with filesystem and snapshot name:
\# Format: <filesystem>@<snapshot name> 
declare -a SNAPS=(
\# Example array elements:
bpool/BOOT/ubuntu_7pdn8o@autozsys_
rpool/ROOT/ubuntu_7pdn8o/usr@autozsys_
rpool/ROOT/ubuntu_7pdn8o/var/cache@autozsys_
rpool/USERDATA/root_kpsn24@autozsys_
)

echo Deleting all but the last $RETENTION system snapshots.
for SNAP in ${SNAPS[@]}
do
echo SNAP=${SNAP}
\# The dry run (comment out when satisfied all is well):
zfs list -t snapshot -o name | grep ^${SNAP} | sed -n -e :a -e "1,${RETENTION}!{P;N;D;};N;ba" | xargs -n 1 zfs destroy -nprd
\# For real (uncomment when all is well):
\# zfs list -t snapshot -o name | grep ^${SNAP} | sed -n -e :a -e "1,${RETENTION}!{P;N;D;};N;ba" | xargs -n 1 zfs destroy -rd
done

After running this it may be necessary to reboot before all is finally deleted.

I configured my Ubuntu 22.04 OS to run this script whenever it boots.