Finding what caused the memory spike on a live server is not an easy task. It will be even trickier since you have a range of services, any of which could be the culprit.
As a cheat, I have left top running through a terminal sorted by memory usage. This way, when the server crashes, it leaves my terminal at the last top screen which can at least tell me which process is the likely culprit.
It is then working with that service to try and figure out the culprit.
In our case, it turned out to be caused by the maximum memory for PHP set to too high a value.
The reason this causes a problem is that apache does not clear the memory once a child uses up a certain amount of memory until that child is reaped which may not happen for a long time.
This can build up and eventually finish off all the memory.
I have no experience with qmail so can't comment on its memory usage but mysql on default settings tend to use around 50m as shown on your top.