Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

On my 8 core NAS I see only one core maxed out when ZFS replication over SSH is running.

I would expect that transferring a snapshot would scale perfectly with the number or cores, as the snapshot could be split up in N independent pieces.

Are there any technical reasons why it is single threaded?

share|improve this question

1 Answer 1

up vote 4 down vote accepted

Not sure about this, but an (educated) guess:

Replication locally on the machine can easily be implemented by a multi-threading approach, but the question is if this would give you much benefit as the data-transfer to/from the HD's is very much a sequential process. Of course you can do multiple HD's in parallel, but synchronization between the parallel processes would be very important and complicate matters a lot.

When you throw a LAN connection into the mix matters are again different.
The SSH connection is a single threaded thing. You can obviously run multiple connections in parallel but again the nature of a LAN connection is highly sequential (and LAN speed usually the most limiting factor) , so it is debatable if this would gain much performance and whether the effort is worth the gain.

And of course... Is the other side able to handle this in the same way ?
If the receiving NAS has less cores (worse case: just 1) than the sender you can easily saddle it with an enormous amount of task-switching-overhead.
And if the physical disk-structures between the NAS-boxes don't match there is also an (non trivial) amount of data re-organisation that needs to be done on the fly.
Again: Would it be worth the complexity and effort ?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.