I've 4 NetApp 2240-4 filer heads. They're single chassis 'cluster in a box' so two separate units.
Over the last few days, at about the same time - all of them started logging a LOT of Low water mark consistency points.
Running wafl_susp -w
gives me cp_from_low_water
clocking up at a rate of 10/sec or more. Before this started, they were almost entirely cp_from_timer
at a rate of 1 every 10s or so.
Two of my boxes have become unresponsive and been rebooted, and the problem has now gone again. I'm not 100% sure that's connected, but it seems a reasonable bet as to a culprit.
The other two - are completely idle, as in they have a base OS, and a couple of vfilers - and nothing else. But yet - Low watermark, suggests they're running out of memory, for some reason. I can only assume some sort of denial of service condition is occurring (perhaps 'failed SSH logins'?).
Can anyone offer an insight on how to troubleshoot this? Specifically from a NetApp perspective, I'm looking for some hints as to how to extract what's hogging my memory.