I have a weird issue regarding glusterfs,

So ive setup a very simple replication volume accross two nodes using this command

sudo gluster  volume create cloudStorage replica 2 transport tcp  machine1:/data  machine2:/data force

Ive then started the volume and mounted it on my two hosts (machine3, machine4) in /etc/fstab, this also works fine.

If I then create a folder on machine3 it will appear instantly on machine4 as available and is in the filesystem of machine2

If I then create a folder on machine4 it will appear on machine3's mount. But if i check machine1 the newly created folder doesn't appear. It eventually will appear on machine1's filesystem but is already available to machine3 to use and browse ? (machine1 can't browse to the newly created folder)

I plan to implement CTDB and im worried about a recovery file lock issue if gluster isn't replicating as it should be.

Im aware that this question may be abit vague so what I think im asking is

A) Why is replicating a single folder creation taking so long ?
B) Why does it take so long in one direction only ?
C) How can I improve the performance

A little background on the setup

The servers are

  • 2 ESXi
  • 2 Ubuntu 14.04 (glusterfs nodes)
  • 2 Ubuntu 16.04 (gluster client nodes)

They are linked between two ports in a draytek 2860N they are in the same building and the must be about 2 meters of cabling between the two ESXi hosts if that.

This image may help describe the problem

This May Help


Gluster doesn't replicate so much as allow each client to write to every relevant gluster node at the same time. This fan-out method of connectivity is what makes it so fast, but it is imperative that every client be able to connect to every gluster node.

There is a method of server-to-server replication, but it's for back-end processes and for correcting replication problems caused by clients that weren't connected to all gluster nodes.

Make sure that all clients have a stable connection to all storage nodes. If you have problems still, ort that doesn't help, please elaborate and I'll provide additional help.

  • Not sure if you're still paying attention here, but, if you are, can you add information about the server-to-server replication method to your answer? (I have a replicated pair where one server was down for four days and am having a devil of a time trying to find out how to force server-to-server replication and catch it up. I assumed heal or rebalance would do it, but no such luck.) – Dave Sherohman Feb 15 '18 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.