GlusterFS - Replication Over Odd Numbers Of Nodes

Question

I tried to create a replicated volume across my 3-node cluster with the following command:

sudo gluster volume create \
replicatedStorage \
replica 2 \
transport tcp \
gluster1.mydomain.com:/data \
gluster2.mydomain.com:/data \
gluster3.mydomain.com:/data

I got this error message:

number of bricks is not a multiple of replica count

From reading the online documentation, it sounds like one has to always have an even number of nodes to have replication, and if you want to replicate twice, you have to have multiples of 4?

Is the following diagram not possible, where each colour represents a file?

Answer 1

Replica 2 means that for each file, it will keep two copies. You can't use 3 nodes because the client should always knows where files should be.

As a workaround, you can create multiple volumes with replica 2 across different couples of nodes, if you are interested in using the third node with replica 2.

Or you can use replica 3, and get improved fault-tolerance.

Answer 2

You can't use a replica on odd number of drivers, but you can use disperse instead:

gluster volume create cache disperse 3 redundancy 1 transport tcp gluster{1...3}.mydomain.com:/data

It will cut your file in pieces and have each 2 copies of piece (since redundancy is 1), so will be able to loose 1 disk on the 3.

The doc says it should perform worse than replica on the same configuration.

https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes/#creating-distributed-dispersed-volumes