Questions tagged [glusterfs]
GlusterFS is a scale-out NAS file system that runs over Ethernet or Infiniband RDMA interconnect. GlusterFS is now owned by Red Hat.
110
questions with no upvoted or accepted answers
7
votes
0
answers
3k
views
excessive glusterfs memory usage
I run a 3-node glusterfs 3.10 cluster based on Heketi to automatically provision and deprovision storage via Kubernetes. Currently, there are 20 volumes active - most with the minimum allowed size of ...
- 458
6
votes
1
answer
19k
views
GlusterFS Transport endpoint not connected from time to time
I'm using GlusterFS 3.7.9, currently on a single server with 4 bricks.
Each brick has 4TB and the volume is set up as distribute only.
The volume is mounted on a secondary server and I use it for ...
- 315
5
votes
2
answers
2k
views
GlusterFS fails to mount on boot but mounts later in Ubuntu 12.04
Having two machines, profitmargin and revisionist, I created a volume in profitmargin:
root@profitmargin:~# gluster volume info
Volume Name: uploads
Type: Distribute
Status: Started
Number of Bricks:...
- 7,508
4
votes
0
answers
107
views
Data Replication Approaches for Hot Standby Architecture
I'm relatively new to the world of storage & filesystems (please forgive me if I'm not using the correct language), and I'm currently researching ways to achieve a hot standby configuration -- i.e....
- 141
4
votes
0
answers
2k
views
Previously mountable GlusterFS volume now fails to mount with several errors
I've previously set up a couple of GlusterFS volumes (called cyclorana0 and cyclorana1) across two nodes (with IPs 10.0.2.4 (hostname alboguttata) and 10.0.2.5 (hostname verrucosa)) on my LAN. The ...
- 875
4
votes
0
answers
5k
views
GlusterFS peer appears disconnected in one of the peers
I have am using a GlusterFS cluster with a Trusted Storage Pool consisting out of 4 peers.
example-prod (100.100.250.197)
example-storage1 (100.100.248.178)
example-storage2 (100.100.250.25)
example-...
- 141
4
votes
0
answers
734
views
root access to GlusterFS volume with root-squash
I want to export my glusterfs volume with the option server.root-squash enabled, so that clients (over which I have no control) can't perform actions as root on the volume.
However I, as and admin, ...
- 141
3
votes
0
answers
2k
views
Gluster Performance
currently I try to setup a Gluster cluster and the performance is strange and I'm not sure, if I configured something wron. I'm using 4x Hetzner root server running Debian Buster with Intel i7, 128GB ...
- 518
3
votes
1
answer
1k
views
CTDB Samba failover not highly available
TL;DR
Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection.
here and here state there is work in progress to make this possible
here states it is ...
3
votes
0
answers
1k
views
Multiple disk sizes for best case parity in gluster
I'm currently stuck with my current situation in creating a Distributed File System.
I have
2x 8TB drives
2x 4TB drives
2x 2TB drives
2x 1TB drives
I've been looking into glusterfs and it seems ...
- 31
3
votes
1
answer
1k
views
glusterfs uid/gid mapping on Debian Wheezy
So I setup a glusterfs volume and it works all fine, but I can't seem to figure out how to mount it with permissions other than root. Supposedly there's options like translate-uid that can be ...
- 41
3
votes
1
answer
2k
views
glusterfs mounts get unmounted when 1 of the 2 bricks goes offline
I have an odd case where 1 of the 2 replicated glusterfs bricks will go offline and take all of the client mounts down with it. As I understand it, this should not be happening. It should fail over ...
- 31
2
votes
1
answer
134
views
Gluster: How to replace a brick in a dispersed volume?
I set up a 4+2 dispersed volume. It worked well so far.
gluster volume info
Volume Name: disperseVol
Type: Disperse
Volume ID: 35386b55-829c-4bac-bdba-609427269cf4
Status: Started
Snapshot Count: 0
...
- 81
2
votes
4
answers
820
views
Creating replicated gluster volume always gives Distributed-Replicate volume
Peace upon you,
I am trying to create a replicated gluster volume but always it gives me Distributed-Replicate volume
i followed the following commands
Pool Peers
root@tiny1:~# gluster peer status
...
- 121
2
votes
0
answers
425
views
GlusterFS volume stalls / blocks after quick access for PHP session files from PHP Docker container
2021-10-05 UPDATED QUESTION AND TEXT AFTER MORE ANALYSIS, STRIPPED DOWN TO MINIMAL CASE
Short description
A Nomad / Consul cluster is running, with Traefik (with minimal configuration) as a system ...
- 51
2
votes
0
answers
2k
views
Glusterfs on Synology NAS?
I run a machine-learning cluster for my small business. The cluster consists of three Ubuntu servers. The 10TB /home directory, provided by Server 1, is shared across all servers and is almost full.
I'...
- 21
2
votes
0
answers
568
views
I'm looking for a way to configure distance clustering of postfix and dovecot
I currently have a postfix server and dovecot server sharing files from an NFS server in the UK and the same config in Switzerland. I tried to mailsync but that appeared to lose emails, so then I ...
- 630
2
votes
0
answers
3k
views
Glusterfs - Clients randomly disconnect until volume is restarted
Ubuntu 18.04
Glusterfs-7.0
I have created a volume for my file share and started it:
sudo gluster volume create NAME replica 3 transport tcp host0:/path0 host1:/path1 host2:/path2
sudo gluster ...
- 35
2
votes
1
answer
2k
views
GlusterFS not running on correct port! (peer disconnected / brick not starting)
On CentOS 7 witch two bricks on srv1 and srv2
I've upgraded gluster from 313 to 6 by using yum. I then rebooted server 1, started and mounted the drive successfully.
This is my mount command:
/usr/...
- 21
2
votes
0
answers
42
views
Is it possible to recover a previously used brick in the same volume? Is it better to reuse it or clear and add as new?
Was testing how glusterfs behaves after an HDD crash and came to this issue where a previously used but not corrupted brick cannot be added back to a volume because glusterfs thinks it already is part ...
- 151
2
votes
0
answers
461
views
GlusterFS for simple replication of existing dirs on ext4 FS?
I have an already existing CentOS 6.9 system with an already existing EXT4 filesystem (/data) housing our application. Certain directories within this file system contain critical per-transaction xml ...
- 23
2
votes
0
answers
55
views
How do I use GlusterFS in a reuse scenario?
I am going to repurpose 5 servers from their previous use as computational servers to be a KVM system. This will become my sandbox for testing patches and for testing new software, configuration ...
- 870
2
votes
0
answers
1k
views
Scale out a Gluster Disperse Volume
I've been playing with Gluster for a few hours to see if it will fit our needs and so far so good. However, I've been looking around the internet for the "how to" of expanding a dispersed volume and ...
- 31
2
votes
0
answers
2k
views
GlusterFS Split Brain issue
I have been facing issues in performance with GlusterFS setup. We took a new build of application live and suddenly all GlusterFS clients and masters also started showing high utilization of CPU. This ...
- 121
2
votes
0
answers
516
views
glusterfs unable to read gsyncd status file
Goal
I wanted to try glusterfs geo-replication in some virtualboxes on my computer for later use distributed on multiple sites.
Configuration
I installed glusterfs 3.6 on ubuntu 32bit servers like ...
- 121
2
votes
0
answers
2k
views
CentOS 7 - Systemctl dependency problems with network mount and OpenSM
I have the follwoing problem. I want to mount a glusterfs share over Infiniband on an CentOS 7 client. The glusterfs share '/data' is severed by a host called storage1. For the client I I setup and ...
- 141
2
votes
2
answers
278
views
Automatically scale a Gluster cluster
I'm trying to configure a proof of concept where a server automatically gets spun up and joins a replication cluster. When the server is no longer needed, it gets spun down, and automatically removed ...
- 213
2
votes
1
answer
1k
views
Certain operations like 'ls' hang in 4-node gluster setup
I have a 4-node gluster setup with replication set to 2. The 'ls' command hangs on the Gluster mount that I have created. The other operations such as creating a file or reading a file work fine.
...
- 239
2
votes
0
answers
323
views
Gluster with s3ql mounted brick?
I am trying to test run Gluster with a volume consisting of two local bricks, the first being a plain directory and the second being a s3ql based mount.
Having just the plain directory as a unique ...
- 858
2
votes
2
answers
617
views
Hypervisor and GlusterFS (or similar) on same physical host?
I've inherited a bunch of hardware from a failed project. They're SuperMicro 2U servers with 24x SATA HDD or SSDs. I've been tasked with turning this rack of generic hardware into a IaaS cluster. I'm ...
- 2,596
2
votes
1
answer
735
views
GlusterFS as elastic file storage?
Is there any way to run GlusterFS in a replicated mode, but with the ability to dynamically scale the volume up and down?
Say you have 3 servers all running glusterd. your Gluster volume would have ...
- 21
2
votes
1
answer
6k
views
Firewall/iptables rules for GlusterFS
A great pleasure would be to know which exact iptables rules to use for securing GlusterFS 3.4.2 (with Debian Wheezy) in shared environment. I'm running some dedicated servers at ISP. There are two ...
- 304
1
vote
0
answers
26
views
GlusterFS Critical Error: Negative fsal_fd_global_counter Freezes NFS Shares - Seeking Root Cause and Resolution
I'm encountering a critical issue in my GlusterFS setup that is causing NFS share freezing, leading to significant production downtime. The logs indicate a recurring error message:
21/11/2023 16:03:34 ...
- 11
1
vote
0
answers
38
views
How do I restrict some directories to some clients in GlusterFS?
GlusterFS makes it possible with auth.ssl-allow option to allow or deny access to given volumes for a given client, identified by its SSL certificate, but the access is global: either the client has ...
- 2,315
1
vote
0
answers
57
views
Gluster replication / propagation to multiple remote servers
I'm new to the concept of Gluster, and I think I'm missing some key knowledge from Gluster documentation.
In my company, there are 10 application servers each with their own storage, and the current ...
- 23
1
vote
0
answers
104
views
sync apache2 config directory with glusterfs and restart service on directory change
I am trying to create a small 3 node ha cluster with apache2, php, mariadb with galera, glusterfs, haproxy and keepalived on debian 11. I want to sync the majority of config files (which are the same ...
- 11
1
vote
0
answers
1k
views
How to remove brick from failed Gluster peer?
I have a host that is damaged and is marked as Disconnected in the pool list. To remove the host and replace it I need to remove the brick.
Info of my bricks:
Volume Name: myvol
Type: Distributed-...
- 293
1
vote
0
answers
120
views
GlusterFS not Sync geo
I activated GlusterFS synchronization with master/slave geo location.
At first everything working, however elw does not synchronize new files, only if I stop and delete-sync-time.
The number of ...
1
vote
0
answers
101
views
Heavy network usage withg GlusterFS FUSE client
I have a simple GlusterFS 7.2 setup with 3 servers:
data1 and data2 contribute several bricks (4 each) to a distributed replicated GlusterFS volume.
host1 uses the GlusterFS FUSE client to mount the ...
- 111
1
vote
0
answers
271
views
Single Gluster peer rebalancing inconsistently
I have a pool of 5 Gluster servers with individual bricks for a volume operating in Dispersed mode, to which I have added an extra 5 peers with individual bricks in another datacenter, making this ...
- 33
1
vote
0
answers
70
views
Optimal volume setup in Glusterfs
I have a number of servers and am wondering what the most optimal volume setup would be. Initially, there'll be 5 servers but over 20 later on. Each server has 16 x 3TB drives.
Essentially, I want to ...
- 85
1
vote
0
answers
138
views
Should I use GlusterFS for mail server?
I'm planing to use GlusterFS for my mail server. But I read that GlusterFS have problems with small file ( and mail are small files). What's the problems with GlusterFS when come to working with small ...
- 301
1
vote
1
answer
4k
views
Choosing between GlusterFS replica 2 or 3
I can't figure out the relation between the number of bricks and replica count.
I have 4 server nodes and after running the command to create a distributed replicated volume, with a replica count of ...
- 137
1
vote
0
answers
344
views
Glusterfs support for user namespaces
I have a gluster volume mounted on a host machine and tried to run a namespaced docker container with a bind mount from the gluster mount on the host.
It seems that the user namespacing is not ...
- 194
1
vote
0
answers
302
views
Increasing size of .glusterfs directory
I'm using glusterfs 3.3 version.
One of my mounted drive (15TB) is show full. After checking, I got that there is a directory .glusterfs which is uses 3TB.
I don't know the actual reason of .glusterfs ...
- 11
1
vote
1
answer
1k
views
glusterfs for wordpress hosting
I've seen many tutorials recommending glusterfs for distributed web application hosting. Is this really the best practice for small scale load balancing?
It seems the latency issue is slowing us ...
- 11
1
vote
1
answer
897
views
Glusterfs FUSE client black magic (automatic high availability)
Quick question regarding the Glusterfs FUSE client black magic. I currently have a gluster 3 node doing a single replica volume. On my gluster fuse client i am pointing to one of the glusterfs nodes ...
- 11
1
vote
0
answers
86
views
Glusterfs dont mirror supposed mirrored folder
Gluster dont replicate mirrored content:
xfce4-terminal -T LocalTerm
exit
At LocalTerm:
PS1='[\u@\h]-[\D{%T}]-[\W] => '
lxc profile copy default default-bkp
lxc profile set default raw....
- 47
1
vote
1
answer
200
views
Glusterfs 2x server issue
I am having almost identical issue, while so can provide more details on how the set up is:
2x server replica 2 gluster volume from two bricks. Brick
IMG-01:/images/storage/brick1 49152 ...
- 131
1
vote
1
answer
1k
views
Gluster snapshot directory error: Transport endpoint is not connected
When I try to access the snapshot directory for a volume, I keep getting .snaps: Transport endpoint is not connected I killed and started the glusterd service but still keep getting that error message....
