I recently installed a cheap 2tb hard drive on a server for backing up files which are also backed up elsewhere. It's basically an overflow drive. The other drives on the server are configured as 1TB in a Raid 6 array. This single drive I configured as Raid 0 just for convenience.
In essence I was moving about 700GB of data from the Raid 6 drive to the Raid 0 drive because the Raid 6 drive was almost full. So ... 2 TB should be way more than enough, right?
The data is in the form of data rsynced from a remote server, with 6 days of incremental backups handled in a standard 'hard link' manner to ensure I am only storing/transferring changes, and not backing up the entire data every day.
However the behaviour I am seeing is that data that was stored at around 700GB on the Raid 6 drives quickly balloons to almost fill the 2TB drive, as if I were not using hard links.
Yesterday I deleted about 300GB of data which is no longer needed, and overnight the storage was back to 97% full.
Does anybody know what's going on? Is the drive really 'full', or is it just bad calculation of hard linking?
All drives are formatted as Ext4.
** Edit **
Details of backup process:
Each day a cronjob copies backup0 to backup1 using
cp -al backup0 backup1. Previous backups are moved by
mv backup1 backup2, etc, prior to an rsync taking place.
backup5 is deleted each day. After that happens, a remote server rsyncs to backup0 (thus updating only changed files). Thus, 5 days of incremental backups. This is basically how software like 'backintime' works too.
** Second Edit **
I just deleted backup3 to backup 5 and it freed up about 2 thirds of the drive. So, the problem seems to be how storage is being calculated. (I use
df -h to monitor storage).
The question remains ... will the drive be considered 'full' even though there should be ample space, when it reaches "100%".