1 NAS at our main office (Synology Cubestation CS407)
20 remote servers in various physical retail stores (1 server per store)
Network between HQ and stores are in an MPLS/VPN.
I have created password-less access from each store server into our NAS.
I have created a bash script that backup specific set of folders varying from 2,5 to 14 gigs of data depending of the store.
The bash script is run on the individual store servers, and uses Rsync to copy folders into the NAS
rsync -azv --delete-after --stats /u/archive/ root@nas01:/volume1/bak/$HOSTNAME/u/archive$(date +%u)/
Can I safely execute this script simultaneaus on multiple servers, each night at i.e. 2AM - Can Rsync on the NAS handle incoming data/connections from multiple servers at the same time?
So i ran the backup script last night at 2AM, running it only on the oldest/biggest store (store1) which has around 9.5G of data that is backed up. This fresh/full Rsync took 1 hour 6 minutes.
The store that I made my test runs on earlier yesterday (store8) was also backed up simultaneous to store1. Store8 has 2.4G of data and took 29 minutes.
My tests with store8 earlier yesterday fresh/full Rsync, took 9 minutes to complete. And 3 minutes for a immediate consecutive Rsync, meaning next to no filechanges.
And as already suggested, I plan to stagger/schedule the Rsyncs to even out the load.
What is backed up:
3 Directories, bash script runs 3 Rsync commands
- Dir 1: Archived data / sales data, no updates to data, 1 folder with ~30 files added per day.
- Dir 2: Main data directory, settings for the POS system, todays sales data.
- Dir 3: PostgreSql database directory. Plan to also run pg_dump just before Rsync.
How do the data is split:
- Dir 1: This is the incrementing directory, ranging from 8.3G (store1, 15 years) to 70M (store17, 3 months)
- Dir 2: 0.5G to 1G, Average 90K files and 4000 dirs
- Dir 3: 0.1G to 0.4G, database files, average 450 files and 13 dirs
HQ: 100/100Mbit fiber.
Store: ADSL, depending on copper at hand, ranging between 1 to 2 Mbit upload.
Synology Cubestation CS407 (DSM 3.1)
4x STM3250310AS, Maxtor DiamondMax 21
- SATA 3.0Gb/s
- 250GB Capacity
- 7200 RPM
- 8MB Cache
- 4.16ms Average Latency
Configured in RAID 5
684GB Total Capacity
130GB Used (19%)
Once I have the procedure running I will most likely replace this ageing NAS and drives