just spent two days this week troubleshooting dev issue in EC2 logged by offshore team.
have run apache/tomcat Version 7.0.21 in muliple Dev instances in EC2 for weeks with no problem.
then major performance issues in D3 env. re-ran the scripts on shore with no problem first time.
again offshore logged defects in D3 env, this time they ran scripts in D2 clone had no issues. ran scripts in D3 again onshore in morning and this time had major issues.
had a feeling it was infrastructure but no way to prove it.
tuned servlet container looking at garbage collection, heap, jdbc pool - in sandbox env, nothing wrong.
then scripts passed in D3 clone image. all logged defects passed. we had changed nothing.
it looks like an EC2 issue, either on Xen VMs, network or RDS. No idea what it was.
How can you isolate fault in cloud when you are flying blind. With no visibility of infrastructure where do you begin?
Anyone have similar problem?
Can EC2 infrastructure be monitored?