Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Note: I'm not sure what to search for so guidance on that may be just as valuable as an answer.

I'm looking for a way to visually compare activity of two applications (in this case a webserver with php communicating with the system or mysql or network devices, etc) such that I can compare the performance at a glance.

I know there are tools to generate data dumps from benchmarks for apache and some available for php for tracing that you can dump and analyse but what I'm looking for is something that can report performance metrics visually from data on calls (what called what, how long did it take, how much memory did it consume, how can that be represented visually in a call stack) and present it graphically as if it were a topology or layered visual with different elements of system calls occupying different layers.

A typical visual may consist of (e.g. using swim diagrams as just one analogy):

Network (details here relevant to network diagnostics)
   |                                                                       ^ back out
   v                                                                       |
 Linux (details here related to firewall/routing diagnostics)              ^ back to network
   |                                                                       |
   V                                                                       ^ back to system
 Apache (details here related to web request)                              |
   |                                                                       ^ response to
   V                                                                       |  apache
  PHP (etc)         PHP---------->other accesses to php files/resources-----
   |                 ^
   v                 |
 MySQL (total time) MySQL
   |                 ^
   V                 |
Each call listed + time + tables hit/record returned

My aim would be to be able to 'inspect' a request/range of requests over a period of time to see what constituted the activity at that point in time and trace it from beginning to end as a diagnostic tool.

Is there any such work in this direction?

I realise it would be intensive on the server, but the intention is to benchmark and analyse processes against each other for both educational and professional reasons and a visual aid is a great eye-opener compared to raw statistics or dozens of discrete activity vs time graphs. It's hard to show the full cycle.

Any pointers welcome.



> XHProf in conjunction with other programs such as Perconna toolkit
> ( for mySQL run apache
> with httpd -X & (Single threaded debug mode and background) then
> attach with strace -> kcache grind
share|improve this question

You may want to look at NewRelic. Though it's not free, it will instrument your application and give you some of the data you are looking for.

share|improve this answer
Hi @Bill Weiss, this is a great application, but summarises the time spent in different layers of the applications, primarily. I'm looking specifically for calls, akin to a debug_backtrace()'s data, but with time and a visual output, other than just a line-by-line text output. Similarly, something like that at each layer of the application execution. So far, most of what I can find is like kcachegrind, which works after execution and only after setting it up for each run. – MyStream Feb 26 '12 at 22:55

You will want to try out XHProf, whilst setup is far from trivial it can produce call graphs. (e.g. random sample found on google images:

The web interface can also enumerate various stats such as time on cpu and memory usage, if you want to side by side comparison of application calls this is the way in my opinion.

share|improve this answer
For PHP this looks very close to the information I am looking to collect for PHP, although its output is post-call rather than live I think? Following the PECL info through though it may be possible to push this output to another application that can show it in combination with mysql, network stack and others. Possible part solution here. – MyStream Feb 28 '12 at 17:38
@MyStream unfortunately as each part of the application is independent of the other you will need to run XHProf in conjunction with other programs such as Perconna toolkit ( for mySQL, run apache with httpd -X & (Single threaded debug mode and background) then attach with strace -> kcache gind, afaik there are no "All in one" solutions (Unless you intend to write one, in which case github it ;-) – Oneiroi Feb 29 '12 at 9:43
I think this is the closest to an answer that is likely and it covers the points. Thanks @Oneiroi. I was looking to pipe output in sequence as one process handed off to another, but I think it's not yet something someone's needed enough to put together. I'd be interested in supporting a github project on it though! – MyStream Feb 29 '12 at 17:35
See also this excellent fork of xhprof that adds a better GUI and improved comparisons... – Ian Jun 23 '12 at 21:02

Unless you have a really, really busy system, the network, Linux and Apache layers do consume only a fraction of time compared to rest of the stack.

Just use PHP XDebug extension and then load the datafile generated by XDebug to KCachegrind or similar.

If KCachegrind reveals your MySQL is the actual bottleneck, then just mysqltuner, MySQL Workbench, mytop, innotop or plain good old slow query log browsing to find out what's your bottleneck there.

If you truly believe Apache is your bottleneck, then take a look at its server-status page. Try to tune values such as TimeOut, KeepAlive and consider using something other than default prefork FPM. If everything else fails, switch to lighttpd or nginx.

Linux or network will only be your bottleneck after you have fine-tuned the PHP+MySQL and Apache layers.

share|improve this answer
I'm not looking to solve a particular problem at this point, but simply to provide an all-around visual understanding of requests and responses. For example, if you can visually see how much time is spent on waiting on mysql or IO operations, you wouldn't need to check logs and test to find bottlenecks, you could see at a glance where requests are spending most of the time and where cpu/memory are being most frequently accessed/allocated. It's the visual for the whole process I'm attempting to produce. But the tools are useful if they can be daisychained? – MyStream Feb 28 '12 at 17:40

Tracelytics is another full stack performance analyzer, similar to New Relic. Tracelytics has some beautiful graphs showing application latency, and if I recall, does show down to the call level. Check it out, it might be useful.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.