3

TL;DR

Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection.

  • here and here state there is work in progress to make this possible
  • here states it is already possible with Samba 3.0 (currently using 4.7)

My Setup

3 nodes running ceph + cephfs
2 of these nodes running CTDB & Samba 1 client (not one of the 3 servers)

It is a Lab setup, so only one nic per server=node, one subnet as well as all Ceph components plus Samba on the same servers. I'm aware, that this is not the way to go.

The problem

I want to host a clustered Samba file share on top of Ceph with ctdb. I followed the CTDB documentation (https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba#Configuring_Clusters_with_CTDB) and parts of this: https://wiki.samba.org/index.php/Samba_CTDB_GPFS_Cluster_HowTo. The cluster is running and a share is reachable, readable and writeable on both nodes, my smb.conf looks as follows:

[global]
netbios name = CEPHFS
workgroup = SIMPLE
clustering = yes
idmap config * : backend = autorid
idmap config * : range = 1000000-1999999
log file = /var/log/samba/smb.log

# Set files creation permissions
create mask = 664
force create mode = 664

# Set directory creation mask
directory mask = 2775
force directory mode = 2775

[public]
comment = public share
path = /mnt/mycephfs/testshare
public = yes
writeable = yes
only guest = yes
ea support = yes

CTDB manages Samba and reports both nodes as OK.

But when i read or write to one of the nodes via the public IP and let it fail (restarting ctdb), the read or write fails. A second write attempt succeeds (the public IP gets taken by the other host successfully).

But CTDB should be able to do this according to https://ctdb.samba.org/ -> IP Takeover? I have a tcpdump of the new server (the one taking over the public ip) sending a tcp RST to my client after the client sending retransmissions to the server.

Any idea, what the problem could be?

PS: I'm more than happy to provide you with more Information (ctdb config file, firewall configuration, pcap, whatever ;) ) but this is long enough ....

Edit:

Tested also with GlusterFS as storage backend on a virtualized environment and a Windows 10 Client. Needed a kernel share modes = No, using the gluster vfs.

1 Answer 1

0

The problem is that the Samba version is not supporting persistent handles yet, the following repository is supporting it but is not quiet ready to create a stable version on the official Samba repository. See YouTube for a full explanation about this implementation.

Durable handles are supported by Samba, so network failures to a node will be resolved but failures where CTDB is performing IP takeover will require the SMB client to reconnect. The Windows application/program has to manage and catch the error and retry, e.g. xcopy on windows has a flag /Z (see options) which reestablish the connection when an IP takeover has occurred to a new node.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .