a couple of weeks ago, my linux server (kubuntu 10.04) started to give me trouble.
it freezes after a certain uptime, seemingly between a couple of minutes and a few hours - GUI is unresponsive, no reaction to mouse or keyboard (not even REISUB),
top in an ssh session stops updating and the session itself is aborted after a timeout:
Read from remote host 10.1.1.9: Operation timed out Connection to 10.1.1.9 closed.
back then, I assumed a hardware issue, so i started replacing more and more hardware - graphics card, motherboard, cpu, ram, harddrives, psu. now i've replaced the entire machine and it still freezes.
/var/log/messages and some other logs - there is no clue in them at all. a hardware issue seems unlikely considering it's all been replaced, but is still possible.
i've stripped the machine down to the bare minimum. i boot a kubuntu live system from a usb stick, mount a couple of harddrives read-only and start diffing folders on them. this seems to produce the freeze somewhat reliably. so far, i haven't gotten beyond a few hours of uptime.
my server is down, this has been going on for weeks now. i am at the end of my wisdom and i am clutching at straws.
how can i reliably determine if this is a hardware or a software issue ? how would you approach a problem like that ?