0

UPDATE: Upgrade to latest version 5.2 and thus update the log. However, issue stays the same UPDATE 2: also update client to 5.2, still the same issue.

I have a gluster cluster setup with 3 nodes.

  • server1, 192.168.100.1
  • server2, 192.168.100.2
  • server3, 192.168.100.3

They are connected via an internal network 192.160.100.0/24. However, I want to connect with a client from outside the network using the public ip of one of the servers, which does not work:

sudo mount -t glusterfs x.x.x.x:/datavol /mnt/gluster/

Gives something like this in the log:

[2018-12-15 17:57:29.666819] I [fuse-bridge.c:4153:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.26
[2018-12-15 18:23:47.892343] I [fuse-bridge.c:4259:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.24 kernel 7.26
[2018-12-15 18:23:47.892375] I [fuse-bridge.c:4870:fuse_graph_sync] 0-fuse: switched to graph 0
[2018-12-15 18:23:47.892475] I [MSGID: 108006] [afr-common.c:5650:afr_local_init] 0-datavol-replicate-0: no subvolumes up
[2018-12-15 18:23:47.892533] E [fuse-bridge.c:4328:fuse_first_lookup] 0-fuse: first lookup on root failed (Transport endpoint is not connected)
[2018-12-15 18:23:47.892651] W [fuse-resolve.c:127:fuse_resolve_gfid_cbk] 0-fuse: 00000000-0000-0000-0000-000000000001: failed to resolve (Transport endpoint is not connected)
[2018-12-15 18:23:47.892668] W [fuse-bridge.c:3250:fuse_statfs_resume] 0-glusterfs-fuse: 2: STATFS (00000000-0000-0000-0000-000000000001) resolution fail
[2018-12-15 18:23:47.892773] W [fuse-bridge.c:889:fuse_attr_cbk] 0-glusterfs-fuse: 3: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-12-15 18:23:47.894204] W [fuse-bridge.c:889:fuse_attr_cbk] 0-glusterfs-fuse: 4: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-12-15 18:23:47.894367] W [fuse-bridge.c:889:fuse_attr_cbk] 0-glusterfs-fuse: 5: LOOKUP() / => -1 (Transport endpoint is not connected)
[2018-12-15 18:23:47.916333] I [fuse-bridge.c:5134:fuse_thread_proc] 0-fuse: initating unmount of /mnt/gluster
The message "I [MSGID: 108006] [afr-common.c:5650:afr_local_init] 0-datavol-replicate-0: no subvolumes up" repeated 4 times between [2018-12-15 18:23:47.892475] and [2018-12-15 18:23:47.894347]
[2018-12-15 18:23:47.916555] W [glusterfsd.c:1481:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494) [0x7f90f2306494] -->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xfd) [0x5591a51e87ed] -->/usr/sbin/glusterfs(cleanup_and_exit+0x54) [0x5591a51e8644] ) 0-: received signum (15), shutting down
[2018-12-15 18:23:47.916573] I [fuse-bridge.c:5897:fini] 0-fuse: Unmounting '/mnt/gluster'.
[2018-12-15 18:23:47.916582] I [fuse-bridge.c:5902:fini] 0-fuse: Closing fuse connection to '/mnt/gluster'.

What I can see is

0-datavol-replicate-0: no subvolumes up

And

0-fuse: 00000000-0000-0000-0000-000000000001: failed to resolve (Transport endpoint is not connected)

Firewall ports (24007-24008,49152-49156) are open on the public network interface.

gluster volume heal datavol info

Brick 192.168.100.1:/data/gluster/brick1
Status: Connected
Number of entries: 0

Brick 192.168.100.2:/data/gluster/brick1
Status: Connected
Number of entries: 0

Brick 192.168.100.3:/data/gluster/brick1
Status: Connected
Number of entries: 0

cluster info:

  1: volume datavol-client-0
  2:     type protocol/client
  3:     option ping-timeout 42
  4:     option remote-host 192.168.100.1
  5:     option remote-subvolume /data/gluster/brick1
  6:     option transport-type socket
  7:     option transport.address-family inet
  8:     option send-gids true
  9: end-volume
 10:  
 11: volume datavol-client-1
 12:     type protocol/client
 13:     option ping-timeout 42
 14:     option remote-host 192.168.100.2
 15:     option remote-subvolume /data/gluster/brick1
 16:     option transport-type socket
 17:     option transport.address-family inet
 18:     option send-gids true
 19: end-volume
 20:  
 21: volume datavol-client-2
 22:     type protocol/client
 23:     option ping-timeout 42
 24:     option remote-host 192.168.100.3
 25:     option remote-subvolume /data/gluster/brick1
 26:     option transport-type socket
 27:     option transport.address-family inet
 28:     option send-gids true
 29: end-volume
 30:  
 31: volume datavol-replicate-0
 32:     type cluster/replicate
 33:     subvolumes datavol-client-0 datavol-client-1 datavol-client-2
 34: end-volume
 35:  
 36: volume datavol-dht
 37:     type cluster/distribute
 38:     option lock-migration off
 39:     subvolumes datavol-replicate-0
 40: end-volume
 41:  
 42: volume datavol-write-behind
 43:     type performance/write-behind
 44:     subvolumes datavol-dht
 45: end-volume
 46:  
 47: volume datavol-read-ahead
 48:     type performance/read-ahead
 49:     subvolumes datavol-write-behind
 50: end-volume
 51:  
 52: volume datavol-readdir-ahead
 53:     type performance/readdir-ahead
 54:     subvolumes datavol-read-ahead
 55: end-volume
 56:  
 57: volume datavol-io-cache
 58:     type performance/io-cache
 59:     subvolumes datavol-readdir-ahead
 60: end-volume
 61:  
 62: volume datavol-quick-read
 63:     type performance/quick-read
 64:     subvolumes datavol-io-cache
 65: end-volume
 66:  
 67: volume datavol-open-behind
 68:     type performance/open-behind
 69:     subvolumes datavol-quick-read
 70: end-volume
 71:  
 72: volume datavol-md-cache
 73:     type performance/md-cache
 74:     subvolumes datavol-open-behind
 75: end-volume
 76:  
 77: volume datavol
 78:     type debug/io-stats
 79:     option log-level INFO
 80:     option latency-measurement off
 81:     option count-fop-hits off
 82:     subvolumes datavol-md-cache
 83: end-volume
 84:  
 85: volume meta-autoload
 86:     type meta
 87:     subvolumes datavol
 88: end-volume

gluster peer status:

root@server1 /data # gluster peer status 
Number of Peers: 2

Hostname: 192.168.100.2
Uuid: 0cb2383e-906d-4ca6-97ed-291b04b4fd10
State: Peer in Cluster (Connected)

Hostname: 192.168.100.3
Uuid: d2d9e82f-2fb6-4f27-8fd0-08aaa8409fa9
State: Peer in Cluster (Connected)

gluster volume status

Status of volume: datavol
Gluster process                             TCP Port  RDMA Port  Online  Pid
------------------------------------------------------------------------------
Brick 192.168.100.1:/data/gluster/brick1    49152     0          Y       13519
Brick 192.168.100.2:/data/gluster/brick1    49152     0          Y       30943
Brick 192.168.100.3:/data/gluster/brick1    49152     0          Y       24616
Self-heal Daemon on localhost               N/A       N/A        Y       3282 
Self-heal Daemon on 192.168.100.2           N/A       N/A        Y       18987
Self-heal Daemon on 192.168.100.3           N/A       N/A        Y       24638

Task Status of Volume datavol

What do I miss?

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.