I am not really good at gluster as I just started using this yesterday.

I have 2 servers. Both are running glusterfs-servers.

From server 1: I run sudo glusterfs peer probe server2 and it is added to the cluster. There was no questions asked. I did nothing to tell server2 to allow server1 to add it to the cluster. Does not make sense to me.

This confuses me. I mean, what if someone adds my glusterfs servers to their cluster. Seemed like there was absolutely no security. It is insane and I do not get it.


TL,DR: Adding servers to the cluster (called pool) is safe, because a 3rd party cannot join an existing cluster on it's own, it needs to be invited from within. But make sure to restrict which clients can mount the volumes and encrypt the connections.

There was no questions asked. I did nothing to tell server2 to allow server1 to add it to the cluster.

I had this question myself so I went to take a look at the documentation.

When you are creating a new cluster, you start on one server and add others using gluster peer probe OTHER_SERVER. Additional security isn't strictly required, because you are adding new, uninitialized glusterfs servers. (Unless you leave a freshly installed, uninitialized gluster running with public access - then you are in trouble).

So what prevents an attacker from joining your existing cluster? The key is the following paragraph:

Once this pool has been established, only trusted members may probe new servers into the pool. A new server cannot probe the pool, it must be probed from the pool. (source)

As the documentation says, a 3rd party/adversary cannot join your cluster, it needs to be invited from within.

Note that clients do not need to be in the pool to mount the volumes.

Gluster also provides other security mechanisms to restrict access to data and to mitigate against related attacks:

  • IP allow/deny on volume level, gluster volume set VOL_NAME auth.allow IP1,IP2
  • TLS - transport encryption as well as using identities from certificates instead of IP addresses for authorization (not enabled by default and out of scope of this answer, see link)

So if the volumes are not meant to be public:

  • use encrypted and authenticated links between servers and servers-clients (Gluster TLS/IPSec/VPN)
  • with a firewall setup that does not allow any connections from the outside
  • and use auth.allow (auth.ssl-allow for TLS) for finer control on the volume level.
| improve this answer | |
  • 1
    It should be worth mentioning that the protocol will apparently not be encrypted by default, inversely indicated by gluster.readthedocs.io/en/latest/Administrator%20Guide/SSL - so saying it's safe would be a very long stretch in my book. – dualed Apr 5 '17 at 12:36
  • Thanks for mentioning it is not enabled by default - it is in the link that I included, but I will specifically mention it in the text too. However I still recommend running it over IPSec. It is easier to maintain a cipher list and other security configuration in one place than to watch out for TLS implementation specifics of each application such as glusterfs. – Matej Snoha Apr 6 '17 at 13:47
  • 1
    One should add to the discussion, that it is possible to mount a non-TLS-protected volume while not being a peer. So the only way to have any privacy protection on the data you need to either use auth.allow with specific IP's, or turn on TLS. – P Marecki Oct 14 at 10:00
  • @PMarecki That is certainly true and important if someone runs Gluster on a public network. I have clarified the answer that there are other security considerations besides the peer probe from the question. – Matej Snoha Oct 18 at 15:26

Your Gluster servers should be on their own, firewalled off section of your infrastructure. They are not intended to be exposed to the public internet.

| improve this answer | |
  • Well, GlusterFS sports geo-replication, and does not mention VPNs or dedicated links even once, and they never mention what you're saying in their documentation. They simply never address the issue of authentication and authorization, as if it doesn't exist, which doesn't make sense for me. – haimg Jun 13 '15 at 2:19
  • 3
    @haimg I'd say if you're big enough to need Gluster you're big enough to figure out the security side of things on your own. Anyone doing multiple datacenters with replication between them should already have VPN/dedicated links well in hand. – ceejayoz Jun 13 '15 at 2:42

I agree that this is insane: security is an add-on to glusterfs. As pointed out by @ceejayoz, glusterfs is designed to only be run on a physically secured and isolated network.

Fortunately, glusterfs has added on ssl support, which is sadly almost completely undocumented. Presumably using ssl will make things better, although since it's undocumented it's hard to say for certain. What documentation exists is in this blog. Sadly, it only gives a sequence of steps.

| improve this answer | |
  • I'd add that nowadays GlusterFS SSL documentation has got much better. – Luke404 Oct 27 '17 at 5:44

I'd like to add to this topic for reference, because I too initially had security concerns with glusterfs.

My organization is in the process of deploying a quite large RHGS cluster to consolidate multiple dispersed aging storage systems.

The security concerns I had was related to the ability to run the gluster console commmands as root from a client system such as...

"yes | gluster --remote-host=rhgs1 volume delete data"

Yikes! It looks like anybody with root privileges on a system that you don't control the root account on, can destroy your data!

Luckily this is not the case. Any of the commands that modify volumes return an exit status of 1 and fail with EPOLLERR as indicated by /var/log/glusterfs/cli.log. It appears you can only get info about the volumes for which that client has access.

Basically the system has to be a cluster peer to be able to perform cluster maintenance tasks from any of the gluster nodes. Now I get why they call a glusterfs cluster a "Secure Storage Pool".

| improve this answer | |

In my opinion, there are some ways to solve the problem:

  1. firewalld or iptables
  2. SELinux
  3. SSL/TLS
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.