page loading stops half-way, no server or sql load

Question

i'm running a hostgator vps php/apache/linux/mysql

i have a problem that i cannot seem to properly diagnose/fix :

1. sometimes when loading a page, the browser will load have the page, then it hangs for 10-15s, sometimes more, then load the rest of the page
2. if during the hang, i refresh the page, it loads quickly
3. there is no abnormal load either on apache or sql, no broken links, bad javascript or so

I'm quite sure this is a server-side issue but have no idea where to start looking, any pointers much appreciated. i can attach any logs or config files that could help.