2

It's my first message here, so bear with me...

(I have already checked quite a few of the "Related Questions" suggested by the editor)

Here's the setup,

  • a brand new dedicated server (8GB RAM, some 140+ GB disk, Raid 1 via HW controller, 15000 RPM)
  • it's a production web server (with MySQL in it, too, not just serving web requests); not a personal desktop computer or similar.
  • Ubuntu Server 64bit 10.04 LTS

We have an Amazon EC2+EBS setup with the EBS volume formatted as XFS for easily taking snapshots to S3, via AWS' console.

We are now migrating to the dedicated server and I want to be able to backup our data to Amazon's S3. The main reason being the possibility of using the latest snapshot from an EC2 instance in case of hardware failure on the dedicated server.

There are two approaches I am thinking of:

  1. do a "simple" file-based backup with rsync, dumping the database' and other files, and uploading to amazon via S3 API commands, or to an EC2 instance, or something.
  2. do a file-system "freeze" (using XFS) with the usual ebs/ec2 snapshot tool to take part of the file system, take a snapshot, and upload it to Amazon.

Here's my question (or series of questions):

  1. Can I safely use XFS for the whole system as the main and only format on the dedicated server?
  2. If not, is it safe to use EXT4? Or should I use something else?
  3. would then be possible to make snapshots of the system to upload to Amazon?
  4. Is it possible/feasible/practical to do what I want to do, anyway?
  5. any recommendations?

When searching around for S3/EBS/XFS, anything relevant to my problem is usually focused on taking snapshots of a XFS system that is already an EBS volume. My intention is to do it in a "real"/metal dedicated server.


Update: I just saw this on Wikipedia:

XFS does not provide direct support for snapshots, as it expects the snapshot process to be implemented by the volume manager.

I had always assumed that I could choose 2 ways of doing snapshots: via LVM or via XFS (without LVM). After reading this, I realize these 2 options are more like it:

  1. With XFS: 1) do xfs_freeze; 2) copy the frozen files via, eg, rsync; 3) unfreeze xfs
  2. With LVM and XFS: 1) do xfs_freeze; 2) make a binary copy of the frozen fs via lvcreate and related commands; 3) unfreeze xfs; 4) somehow backup the LVM snapshot.

Thanks a lot in advance,

Let me know if I need to clarify something.

1 Answer 1

1

Every linux filesystem (ext2, ext3, ext4, xfs, jfs, raiserfs) in current kernel can be frozen, but it must be placed on LVM first.

If you have LVM, making a snapshot automatically freezes the FS for the time it takes to make the snapshot -- it's better than doing only a freeze (your data is still available for writing and will not break the backup) and much better than simple rsync (as it will copy files in a consistent state).

Other questions:

XFS is safe, but can be problematic if you don't disable write cache or don't have battery backed cache (only ext3 is quite resilient to that)

Yes, ext4 is considered safe now. Choosing a FS mainly depends on the kind of workload you'll experience. XFS is slow with small files, very fast with large files.

4
  • Thanks. LVM is not set up. One reason is, I'm not familiar with it and I want to mess with the least amount of things as possible. Another reason is that, based on my vague understanding, if I used XFS I would be able to do the freezing without needing LVM. Also, the hosting company warned of bottlenecks when combining XFS with LVM. If I understand you correctly, you seem to imply that LVM+EXT4 would be better than XFS ("making a [LVM] snapshot automatically freezes the FS... it's better than doing only a freeze"). I'll continue looking into this a bit more.
    – Rafa
    Oct 19, 2010 at 9:01
  • yes, both ext and xfs are better at snapshotting when on LVM -- you can still process requests while taking backup, the FS is frozen only for few seconds at worst, not half an hour at best. You have to remember to delete the snapshot after taking the backup (that's probably the bottleneck they mantioned, LVM snapshotting is slow when you modify the source volume). Oct 19, 2010 at 10:05
  • "making a snapshot automatically freezes the FS for the time it takes to make the snapshot" : do you have any documentation from the linux kernel to support this statement ? Is there a way to generically freeze a filesystem ?
    – Benoît
    Dec 4, 2011 at 16:42
  • OK, it's seems to be fairly new kernelnewbies.org/…
    – Benoît
    Dec 4, 2011 at 17:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .