Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The problem

I'm manually inserting events into Zenoss through the zensendevent command line tool. The events are successfully inserted into Zenoss. The problem is that I can just insert about 8 events per second.

I have also experimented with inserting events through zensyslog. Although zensyslog receives the syslog events immediately (I can see them with tail -f $ZENHOME/log/origsyslog.log), the events are then chewed and inserted into Zenoss asynchronously at about the same rate (8 per second).

I haven't done a great deal of performance tuning for Zenoss or MySQL besides incrementing some cache settings for Zope and MySQL itself (as seen here).

Is the throughput I'm getting normal? What can I do to dramatically increase the number of inserted events per second (I need to be able to insert at least 100 per second). From what I've seen in the zensendevent code, it seems to use Zenoss' XML-RPC interface to insert the events.

Our machine and software specs are the following:

  • Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz
  • 4GB RAM
  • Ubuntu Server 10.04.2 LTS
  • Zenoss 3.1.0 compiled from source
  • This is a dedicated physical installation, not a virtual machine

Some context if it helps:

We want to use Zenoss to monitor our machines and the software they run. Our machines are often not directly accessible from the central server Zenoss is supposed to run on. However, we can get the data we want form any machine to the central server. Right now we generate about 1.5 million events per day which gives an average of about 17 events per second which have to be inserted into Zenoss.

I'm using the following bash script to test the events insertion performance through zensendevent (160 events take about 20 seconds to be inserted):

#!/bin/bash
for a in {1..160}
do
  zensendevent -d 'somemachinename' -y 'some_event_key' -p 'some_component' -s Error -c /App "Example Test message"
done

Thank you very much in advance.

share|improve this question
add comment

1 Answer 1

As answered here, the issue was with the way I was performing my tests. Basically, I was running zensendevent in a loop instead of just performing the XML-RPC request in a loop. Starting up the python script was the biggest bottleneck. I can now insert with the same configuration about 120 events per second.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.