0

I hate to admit it, but I'm feeling stuck and overwhelmed at the moment. I have a colocated server running Debian that really, really, desperately needs to be properly backed up. I have much of the truly irreplaceable stuff "sort of" copied, but I really want to get a complete copy of everything.

This isn't mission-critical enterprise data that has to be protected and available at all costs at all times. It's a hobby server for an Android app I wrote that's kind of grown over the years to the point where backing it up in its entirety has become a formidable task. I want to start a backup process now, have it finish at some point in the reasonable future, and end up with a decent copy of everything on the server that can hopefully be kept reasonably up to date going forward.

My general goals

One-way backup of Linux server to Windows desktop PC. Some files, like logfiles I can identify by path and wildcard, can be deleted from the remote server once the successful copy has been confirmed.

The server's hard drive is 250 gigs, with about 25 gigs free at the moment. Roughly 1/4 of that is logfiles, about 1/3 of it is image files, and another 1/4 or so are daily backup copies of the database.

It's OK to skip files that are huge (> 1 gig) or locked (Mysql database), as long as it's logged in some obvious manner so I can deal with them individually later.

Limited scratch space on server, so it has to be done in bite-sized chunks.

Live server, so the backup process can't bring it to its knees or lock files. Just skip the file(s) and log it somewhere.

Prefer to encrypt in transit across Internet, but don't want to encrypt the actual backup files.

Plenty of local hard drive space for the backup, but would prefer to compress data when it will meaningfully reduce the transfer time more by more than the time it takes to compress in the first place. For the sake of argument, let's say my link rate is 2mbps-5mbps sustained (nominally 10mbps at the server end, 50+mbps alleged at the Comcast end).

I'm root on the server, and it's running Debian (I'm embarrassed to admit I don't know which specific release... it's the Debian release that was considered stable in May 2010, but a newer version had just come out right around that time. I think it might be Etch, and Lenny was the one that was just released). My home computer is running win7/64.

A good start would probably be suggestions for specific Linux and Windows implementations to try, and maybe a howto or two somewhere with examples that approximate my own use case. I've found hundreds of online howtos about using rsync to backup windows PCs to Linux, backup media files to "the cloud", and backup Linux to Linux, but nothing that really involves "backing up my (now-huge) hobby server running Linux to my desktop PC running windows"

Thanks! :-)

1 Answer 1

0

This is pretty easy to do so I'll keep the answer a little high-level.

You can use Cygwin and curl http://ifconfig.me/ip to get the current IP of your Windows box.

Forward rsync(ssh presumably) to your Windows machine.

Simple script on the Windows/Cygwin box to then rsync the tar'd file(you're taring it, right?) that needs to be backed up. I.e., rsync -avz -e ssh [email protected]:/home/user/mybackedupfiles.tar.gz /path/to/save/to

That should do it.

4
  • Actually, nothing is tar'ed yet. I was under the impression that rsync (or at least an app derived from it) can compress and/or split files on the fly before sending, then transparently reassemble them at the other end. From what I've read, once you have the remote copy, incremental updates are quick and painless... I just don't know how to GET to the point where I've successfully copied up to 250 gigs over the internet from my server to my desktop PC, recovering from network hiccups and chugging along for a week or two if necessary, w/o bringing the server to its knees along the way. :)
    – Bitbang3r
    Sep 27, 2012 at 4:44
  • That's part of why I recommended tar, if only for the first major transfer. 250 gigs is significant, even for a professional setup. Rsync in of itself shouldn't bring your server to its knees. Just watch top and be patient.
    – Publiccert
    Sep 27, 2012 at 12:26
  • Hmmm. Crazy question... how does rsync deal with filenames that are valid for ext3, but invalid or ambiguous under NTFS? Can it do something insane, like create multiple 1-gig tarballs one at a time by recursively adding files from a directory, 7zip it, send the temp tarball, delete the source tarball, and keep going until the whole directory has been sent one tarball at a time (deferring files too big for the current tarball until a later round, and sending them individually or splitting them if necessary)? Ultimately, treating files in the pile of tarballs on my PC like directories of files?
    – Bitbang3r
    Sep 27, 2012 at 15:58
  • That can all be scripted but you're probably better off asking another question specifically about that.
    – Publiccert
    Sep 27, 2012 at 16:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .