2

I have two storage servers at home - one is my workstation, which has a pair of 3TB disks in RAID1.

At first, I stored all of my data there. Recently, I received a small SBC with four 1TB disks. I set them up with RAID0, with the idea being that the SBC remains on at all times for everyone to use (and is faster), and when the workstation is up they sync the differences and the data stays safe there.

Right now, I have some unique data on each. What I'm looking for is a way to do one large transfer of the unique data, and then have something more incremental.

Both servers are formatted with BtrFS.

I had a few options in mind:

  • tar the directories I'm transferring, compress them (with zstd) and then scp or rsync them.
  • send the tar to standard output and pipe it into the compressor and over the network
  • use BtrFS send-receive over the network, either with or without compression
  • use rsync, with or without compression
  • mount one of the server's filesystem on the other with SMB or NFS and do the transfer.

Which of these options do you think would perform best?

Thanks!

Improve this question
4
  • 1
    Try what works best for you. Better yet, worry less about performance. Having to restore from other backups (you do have other backups, right?) is more expensive, so chose a robust method (both robust in that you will not run a bad command by mistake, and robust in that it can handle interruptions gracefully).
    – anx
    Sep 26, 2020 at 15:34
  • That is good advice - but in my case, everything important is already on the workstation - the other server is just for convenience (I can access it from anywhere, always on etc). I've been using each of these methods at some point for other purposes - so I was just wondering how to maximize the transfer over our (slower) network.
    – dkd6
    Sep 26, 2020 at 17:22
  • Another option that you may want to consider is unison. Depending on your use case, you may find it very useful - I have.
    – rickhg12hs
    Sep 27, 2020 at 10:08
  • Looks interesting. I'll give it a spin. Thanks!
    – dkd6
    Sep 27, 2020 at 12:18

1 Answer 1

Reset to default
0

Raid-0 always scares me as there is no fail-over. I supposed it's ok in this situation since you will have a backup and you can use the other server in an emergency.

My suggestion: Resilio Sync (formerly bit-torrent sync) It is extremely fast and will keep the two up-to-date. There's a free version for linux home use.

https://www.resilio.com/individuals/

Other than that for the destination server, I would suggest

rsync -avz [email protected]:/path/to/data/ /path/to/data/
Improve this answer
1
  • 1
    That one is new to me. I'll give it a look. Thanks for the suggestion!
    – dkd6
    Sep 26, 2020 at 17:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .