on one of servers running 64bit debian lenny i can see from time to time corrupted kernel messages in syslog file.

they look ok in dmesg:

usb-storage: device found at 21
usb-storage: waiting for device to settle before scanning

but in /var/log/syslog [ produced sysklog daemon ] i can see:

Aug 19 10:01:09 hostname kernel: usb-storage: device found at 21
Aug 19 10:01:09 hostname kernel: usb-storage: waitin fordvc ostl eoesann

this affects only kernel logs. entries from CRON / dhcpd and other services come cleanly. all happens on local host - it's not syslog-over-the-network.

have you ever seen similar problems? any hints? i'm quite sure it's not related to io subsytem or corrupted memory... machine is used heavily, does not have stability issues and consistency of data stored on it is actually checked daily [ witch checksums being calculated on other machines and verified locally ].


ps - i think actual content of messages is irrelevant in this case, corruption occurs to different complains from kernel including entering/leaving promiscuous mode.

  • Did you get anywhere with this? – SirStan Aug 20 '09 at 15:45
  • @SirStan - nope; not yet. – pQd Aug 20 '09 at 22:12
  • do you have hardware problems? faulty memory. Try running a few continious kernel compiles to stress test the hardware? This is really weird, and you should find out what's causing? You using ECC ram? I hope you are! You maybe slowly churning you data on disk it to a corrupted mess. – The Unix Janitor Mar 20 '10 at 15:15
  • @user37899 - thanks for your hint. problem still exists, this is rather not faulty hardware since server is heavily loaded and yet rock-solid. i've run out of options but do not want to reinstall it. i've reinstalled syslog/sysklog, tried another kernels - not much luck so far. – pQd Mar 20 '10 at 16:20

I have seen this before from remote syslog messages. Since it uses UDP (and UNIX sockets for local events), if the content is modified in a way that the basic UDP checksum doesn't notice the change, it will go through..

The funny think is that it went away after I started using rsyslog...

  • thanks for info. unfortunately in my case it's local logging. – pQd Aug 19 '09 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.