Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

It is Samba 3.5.8 on Ubuntu Natty server with kernel 2.6.38-15-server. The server and samba service ran without problems for months.

Since approximately 2 weeks, all samba child processes occasionally (few times a day) die and the (WinXP) users get all sorts of messages about lost network connection, most prominent of them being "{Delayed-Write Failed} Windows was unable to save all the data for the file x. The data has been lost. ..." At the same time, the clients are able to continue to browse shares and open or save files - clients automatically reestablishes connection with the samba service.

log.smbd related entries (apparently, on this occasion, three child processes died - PIDs 4784, 4766 and 4782):

[2012/05/15 21:15:59.957431,  1] smbd/server.c:267(remove_child_pid)
  Scheduled cleanup of brl and lock database after unclean shutdown
[2012/05/15 21:16:19.976345,  1] smbd/server.c:240(cleanup_timeout_fn)
  Cleaning up brl and lock database after unclean shutdown
[2012/05/15 21:16:19.976526,  2] lib/messages_local.c:289(message_notify)
  message to process 4784 failed - No such process
[2012/05/15 21:16:19.976548,  2] lib/messages_local.c:379(messaging_tdb_send)
  pid 4784 doesn't exist - deleting messages record
[2012/05/15 21:16:19.976563,  2] lib/messages.c:127(traverse_fn)
  pid 4784 doesn't exist - deleting connections -1 []
[2012/05/15 21:16:19.976584,  2] lib/messages_local.c:289(message_notify)
  message to process 4766 failed - No such process
[2012/05/15 21:16:19.976595,  2] lib/messages_local.c:379(messaging_tdb_send)
  pid 4766 doesn't exist - deleting messages record
[2012/05/15 21:16:19.976608,  2] lib/messages.c:127(traverse_fn)
  pid 4766 doesn't exist - deleting connections -1 []
[2012/05/15 21:16:19.976637,  2] lib/messages_local.c:289(message_notify)
  message to process 4782 failed - No such process
[2012/05/15 21:16:19.976649,  2] lib/messages_local.c:379(messaging_tdb_send)
  pid 4782 doesn't exist - deleting messages record
[2012/05/15 21:16:19.976659,  2] lib/messages.c:127(traverse_fn)
  pid 4782 doesn't exist - deleting connections -1 []

The child process logs do not show anything usable (at log level 2)

I've already checked:

  • it does not relate to high network traffic, there are no packet errors on the network interface
  • only samba child processes are affected, other network connections aren't
  • file systems are healthy
  • it might relate to file system activity, e.g. on two occasions it happened while the rsync snapshot copying files from one file system (/home) to the backup file system (/srv/backup/snapshot/...) was running. Both file systems have samba shares - /home/... is, naturally, in use by clients, while /srv/backup/snapshot is only occasionally used. But, it happened also on other occasions. I was thinking if it might be a kernel / file system / inotify-related problem.
  • I've tried to reproduce the problem by running rsync without success.
  • disabled oplocks, checked pdc (samba 3.5.11) / ldap / winbind / ip config, fully restarted the server and the switch
  • There are no coredumps. The log files are on a separate /var file system.
  • I've found a similar report https://bugzilla.novell.com/show_bug.cgi?id=750417

The questions ...

  • What might be a probable cause?
  • What to try / check next?
share|improve this question
    
Since there are no error messages directly related to the termination of the child processes, coredumps should be produced. Make sure they are enabled in samba ("enable core files = yes") and core size limit for root in ulimit is set greater than zero. Increase log level to 3 or even 5 in samba. Check dmesg output, maybe the kernel logs something about segfaults. More way-out-there idea: Check the physical memory of the server with memtest, sometimes there is a defect in a high memory bank and it is not triggered with "normal" low memory activity. –  AndreasM May 16 '12 at 18:44
    
Thank you for the comment! I've checked dmesg and tested memory with memtester - no problems found. Enabled core files, checked limits, wrote a small utility to record connection loss from client side. Set log level to 5, waiting for next occurrence. –  Zrin May 19 '12 at 0:17
    
i'm facing the same problem here ? Did you find any resolution ? thank's –  user1807075 Jan 30 '13 at 11:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.