I gave you the answer to this in the comments over on Server not responding to SSH and HTTP but ping works, but apparently you don't believe me. Really, it's true!
You need to size
ServerLimit to your system. The "5-10 settings for Min/Max Servers" which you mention are basically irrelevant — that's just the number of extra servers hanging around not doing anything that Apache will retain.
In order to set MaxClients appropriately, look at the typical high-water mark for your
apache2) processes, and then divide your available memory by that. Best to drop down by a little bit to give the rest of the system room to breathe. Since you've got 4GB of RAM, and 185MB processes, that means your
ServerLimit value should be 21 at most — probably 20 or 19.
Now, it may be that 190MB is atypical. You can set the ServerLimit higher, based on a different estimate of typical usage, but then you're basically gambling that you'll never have a spike. If it does happen, your system will be out of memory.
If you can find a way to constrain your per-worker memory usage, that's gonna be a win. I'm betting this is a case of PHP Ate My RAM. Can you code your app to live within a lower
memory_limit? If you can't do that, you need a different model under which to run your PHP. If you can't do that, you need to buy more RAM.