0

I need to debug some issue on a production server, and I'm trying to decide whether to use strace or the log4cxx logging library; one crucial characteristic I need to know before, is which of the two would have lower impact on performance (speed, mostly)?

For strace, I'm planning for a commandline roughly like below:

strace -p PID -tt -f -e SOME,SELECTED,SYSCALLS -o FILENAME
# or maybe with "-ff" instead? which one would be faster?

For log4cxx, for simplicity we can probably assume I'm calling the logging functions just before doing each syscall I want to trace. Assume version log4cxx-0.10.0, 64 bit.

If what you would suggest is benchmarking this by myself, then I'd be grateful for help on how to write the benchmarks.

Improve this question
2
  • See superuser.com/questions/583023/…
    – Hennes
    Apr 15, 2013 at 12:55
  • @Hennes: that's also my question, I asked on superuser first, but then I remembered about serverfault and though maybe here is the better place for it as it is related to on-line debugging of production issue?
    – akavel
    Apr 15, 2013 at 13:37

1 Answer 1

Reset to default
1

Strace uses a moderately expensive mechanism, while the logging is normally more lightweight. One thing to look out for, though, is DIY logging that uses synchronous writes without buffering much. That tends to make the application wait while the logging completes for every operation logged. A colleague did that once, while measuring performance, and wondered why his performance was suddenly so bad (;-))

--dave

Improve this answer
4
  • "Strace uses a moderately expensive mechanism" - could you provide any details, references? Or, most desirably, benchmarks?..
    – akavel
    Apr 18, 2013 at 18:05
  • 1
    Strace (or truss) uses ptrace, which to oversimplify sets breakpoints in the code and traps to the tracing process when they're struck. This has start-up costs in finding and setting them, and two process-switches when they're used. As a rule of thumb, think of them as being like a system call. Ltrace/apptrace is lighter-weight, and logging more so again. I was one of the apptrace authors, which we wrote to be able to trace production program. Customers found truss too heavy to allow us to use on production machines.
    – davecb
    Apr 19, 2013 at 16:50
  • Oh, I supposed there's rather some hook point in syscalls which allows for such a mechanism! Probably argument against this is that it would incur unnecessary cost on the "frequent path". Thank you very much for caring to elaborate! And actually I thus expected ltrace to be less efficient, interesting that you say it's opposite!
    – akavel
    Apr 19, 2013 at 18:46
  • Ltrace sticks an "interposer" function between the application and the library that implements the syscall, so the cost is one extra call and return, plus one printf. Plus linking the library at start-up, of course!
    – davecb
    Apr 20, 2013 at 16:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .