0

Scenario:

1 NAS at our main office (Synology Cubestation CS407)
20 remote servers in various physical retail stores (1 server per store)

Network between HQ and stores are in an MPLS/VPN.

I have created password-less access from each store server into our NAS.

I have created a bash script that backup specific set of folders varying from 2,5 to 14 gigs of data depending of the store.

The bash script is run on the individual store servers, and uses Rsync to copy folders into the NAS

rsync -azv --delete-after --stats /u/archive/ root@nas01:/volume1/bak/$HOSTNAME/u/archive$(date +%u)/

Question:

Can I safely execute this script simultaneaus on multiple servers, each night at i.e. 2AM - Can Rsync on the NAS handle incoming data/connections from multiple servers at the same time?

Update:

So i ran the backup script last night at 2AM, running it only on the oldest/biggest store (store1) which has around 9.5G of data that is backed up. This fresh/full Rsync took 1 hour 6 minutes.

The store that I made my test runs on earlier yesterday (store8) was also backed up simultaneous to store1. Store8 has 2.4G of data and took 29 minutes.

My tests with store8 earlier yesterday fresh/full Rsync, took 9 minutes to complete. And 3 minutes for a immediate consecutive Rsync, meaning next to no filechanges.

And as already suggested, I plan to stagger/schedule the Rsyncs to even out the load.

Additional info:
What is backed up:
3 Directories, bash script runs 3 Rsync commands
- Dir 1: Archived data / sales data, no updates to data, 1 folder with ~30 files added per day.
- Dir 2: Main data directory, settings for the POS system, todays sales data.
- Dir 3: PostgreSql database directory. Plan to also run pg_dump just before Rsync.

How do the data is split:
- Dir 1: This is the incrementing directory, ranging from 8.3G (store1, 15 years) to 70M (store17, 3 months)
- Dir 2: 0.5G to 1G, Average 90K files and 4000 dirs
- Dir 3: 0.1G to 0.4G, database files, average 450 files and 13 dirs

Bandwith:
HQ: 100/100Mbit fiber.
Store: ADSL, depending on copper at hand, ranging between 1 to 2 Mbit upload.

The NAS:
Synology Cubestation CS407 (DSM 3.1)
4x STM3250310AS, Maxtor DiamondMax 21
- SATA 3.0Gb/s
- 250GB Capacity
- 7200 RPM
- 8MB Cache
- 4.16ms Average Latency
Configured in RAID 5
684GB Total Capacity
130GB Used (19%)

Once I have the procedure running I will most likely replace this ageing NAS and drives

1

You can run this script, but it depends on what you're looking for in safety.

-azv means archive, compress, verbose

--delete-after means you're deleting anything that gets rsync'd off the source server after it reaches the destination server.

20 servers to 1 server... is doable-ish.

Assume the following:

  • 20 x 2.5GB = 50GB (total amount of data that has to be rsync'd every rsync session)
  • 10Mbit = 1.25MB/s (bandwidth available at the NAS site for traffic from internet)
  • (4) 7200RPM SATA drives in RAID1 = 190MB/s or RAID5 = 200MB/s (number of drives, their speeds, and their throughput capability)

Your limiting factor would be your network bandwidth

  • 50GB = 51200MB
  • 51200MB / 1.25MB/s = 40960s = 682min = 11hrs

This doesn't take into account whether or not your NAS can handle the performance necessary for multiple streams at the same time or whether or not you have any other processes going. In my experience, I've seen 10GB rsync's that take forever, because there are hundreds of thousands of files; the more files you have, the longer it takes. The more simultaneous rsync processes you run at the same time, the more distributed your processing gets, which can delay your rsync; if what your backing up is time-sensitive, you'll want to be wary of how quickly you get your rsync process to finish.

Lastly, you'll probably want to stagger your rsync schedule or invest in a better NAS/SAN as well if you need better performance.

Note: if your bandwidth allotment allows for 20Mbit incoming, then you'd 1/2 your rsync time, but you'd still have a huge performance hit on your NAS.

  • Thanks for your input. I have updated my post with additional info. – Phliplip Jul 16 '15 at 7:31
  • Your limiting factor is going to be your branch stores' upload bandwidth. At 1-2Mbit/s, you're going to see maybe, 125KB/s uploads. Based on math, 51200MB * 1024 = 52428800KB / 125KB/s = 419430s / 60s = 6990m / 60m = 116hrs / 24hrs = ~5days. It'd almost be faster to put a copy of all your data onto hard drives and have it shipped to your HQ, then have the data copied to your NAS. You need to increase your branch sites bandwidth or reduce the amount of data you need to transmit. – CIA Jul 16 '15 at 13:01
  • As i wrote in the update, I ran a fresh/full Rsync of store1 which took 1 hour 6 minutes. I then "cp -rfp archive4 archive5" , and i just checked the log from the bashscript, and it ran the refresh in 25 minutes and had updated 47 , 413 and 24 number of files in the pgsql, data and archive directories. This fits the glove where i have planned to run the larger/older stores in 30 minute interval simultaneaus with younger stores in 10-15 minute intervals. – Phliplip Jul 17 '15 at 1:38
0

Adding to the other answers (they all have valid points - bandwidth is important here, too!). A quick websearch revealt that the NAS you mentioned seems to have a Marvell 5281 CPU running at 500MHz and 128 MB of RAM.

While technically rsync can handle multiple connections, I would suggest you don't try to run all the jobs at the same time, but instead leave a gap between the jobs that is long enough for each job to complete before the next one starts, so that ideally no jobs run simultaneously. This hardware doesn't look strong enough for me to feature 20 simultanous connections.

You'll of course need to calculate this offset based on your available bandwidth at that location and the overall write performance of your NAS.

0

Yes, because you put files on different paths on the NAS it is safe. Yes, rsync is capable of multiple connections. so that will be ok.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.