3

TL;DR

Failing a node in a ctdb + samba cluster while interacting with a share interrupts share connection.

  • here and here state there is work in progress to make this possible
  • here states it is already possible with Samba 3.0 (currently using 4.7)

My Setup

3 nodes running ceph + cephfs
2 of these nodes running CTDB & Samba 1 client (not one of the 3 servers)

It is a Lab setup, so only one nic per server=node, one subnet as well as all Ceph components plus Samba on the same servers. I'm aware, that this is not the way to go.

The problem

I want to host a clustered Samba file share on top of Ceph with ctdb. I followed the CTDB documentation (https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba#Configuring_Clusters_with_CTDB) and parts of this: https://wiki.samba.org/index.php/Samba_CTDB_GPFS_Cluster_HowTo. The cluster is running and a share is reachable, readable and writeable on both nodes, my smb.conf looks as follows:

[global]
netbios name = CEPHFS
workgroup = SIMPLE
clustering = yes
idmap config * : backend = autorid
idmap config * : range = 1000000-1999999
log file = /var/log/samba/smb.log

# Set files creation permissions
create mask = 664
force create mode = 664

# Set directory creation mask
directory mask = 2775
force directory mode = 2775

[public]
comment = public share
path = /mnt/mycephfs/testshare
public = yes
writeable = yes
only guest = yes
ea support = yes

CTDB manages Samba and reports both nodes as OK.

But when i read or write to one of the nodes via the public IP and let it fail (restarting ctdb), the read or write fails. A second write attempt succeeds (the public IP gets taken by the other host successfully).

But CTDB should be able to do this according to https://ctdb.samba.org/ -> IP Takeover? I have a tcpdump of the new server (the one taking over the public ip) sending a tcp RST to my client after the client sending retransmissions to the server.

Any idea, what the problem could be?

PS: I'm more than happy to provide you with more Information (ctdb config file, firewall configuration, pcap, whatever ;) ) but this is long enough ....

Edit:

Tested also with GlusterFS as storage backend on a virtualized environment and a Windows 10 Client. Needed a kernel share modes = No, using the gluster vfs.

0

The problem is that the Samba version is not supporting persistent handles yet, the following repository is supporting it but is not quiet ready to create a stable version on the official Samba repository. See YouTube for a full explanation about this implementation.

Durable handles are supported by Samba, so network failures to a node will be resolved but failures where CTDB is performing IP takeover will require the SMB client to reconnect. The Windows application/program has to manage and catch the error and retry, e.g. xcopy on windows has a flag /Z (see options) which reestablish the connection when an IP takeover has occurred to a new node.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.