So I'm tinkering around with ZFS on linux and zrep. I've got 2 VMs on my laptop and I'm running zrep and synchronizing the contents of one filesystem to another.

One unexpected situation is this: If I'm on the "slave" -- the box that's receiving the data -- and I'm continuously reading the contents of a file (such as with sum), if the file is rapidly changing on the master I will get an Input/output error on the slave as the snapshot is getting applied. This does not happen if I'm continuously reading a file that isn't changing in the snapshot.

To be clear -- the "sum" program or any other standard userland program that is reading the changing file on the target filesystem will periodically get an Input/output error and crash.

The ZFS replication itself works correctly -- zrep is just a nice tool for managing the replication process.

I'm a little confused at this behavior -- will reads of files that get updated when a ZFS snapshot is applied cause read errors, or is this a bug in ZFS on linux?

  • is the input/output error reported by the zfs receive or by your sum run? I do not know how exactly zrep is implementing "locking and other sanity checks" but this sounds like either a typical locking issue or a race condition. – the-wabbit Apr 24 '14 at 10:11
  • The error is seen by the program reading the file, not the programs involved in the snapshot replication. – chris Apr 24 '14 at 11:43

You're not supposed to use the replicated filesystem that way. The files are changing as the replication stream updates the target.

What were you expecting to happen?

  • I wasn't sure what would happen. Unix has a pretty convenient way of dealing with files as they change; I wasn't sure if the same conventions would apply in this scenario. If I'm reading a file and someone else changes it, I'll either see the changes or not, depending on my current offset in the file and where the change took place. Snapshots certainly add another dimension to this problem. – chris Apr 24 '14 at 14:36
  • And -- I could see lots of situations where I'd want to use a filessytem this way -- let's say I had a bunch of web files I wanted to atomically update on a bunch of remote servers. I could use rsync and change a symbolic link or I could send a new snapshot to all of them. – chris Apr 24 '14 at 15:05
  • @chris a snapshot application changes the entire dataset. I would expect the entire dataset to become unavailable for a period of time, not just the single file you've changed. – the-wabbit Apr 25 '14 at 5:57
  • @syneticon-dj -- that's not actually the behavior I observed -- I saw quite clearly that a static (in the source pool) file would not cause IO errors in the destination pool; files changing frequently on the source pool, however, would show this behavior on the target pool. – chris Apr 28 '14 at 15:14
  • @chris The point is that you're doing this the wrong way. You shouldn't be using the filesystem of a replication target when there's replication activity occurring. – ewwhite Apr 28 '14 at 15:16

In my opinion the official docs are pretty clear, at least now, some years after your question. :-) The problem you are seeing with changed files seems as expected to me, while the success you have with unchanged files might simply because of things like those are cached in memory?

The file system and all descendent file systems are unmounted.

The file systems are inaccessible while they are being received.

In the linked docs I don't see any documented difference in the behaviour between incremental and full sends of snapshots. Instead the wording that file systems are inaccessible and get unmounted reads to me that things are the same and file systems might already exist, like is the case for incremental sending of snapshots.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.