0

My first post so be gentle please! So I have an installation of graphite up and running on Ubuntu 16.04 server. (version 0.10)

I am sending in metrics and everything works fine, but I am sending in multiple timestamps at once. As in I am sending in data once per minute with per 10s timestamps. Every minute new data comes in. It can happen, that a key with a corresponding timestamp (same as before) is resent a minute later. So the new value is being saved and last one lost, because of the way graphite works. So I put a carbon-aggregator in front of carbon-cache thinking it will aggregate the values per timestamp. It does not. It aggregates all values received in a time interval ignoring the timestamps.

I found that statsd has the same functionality. What I want is for the aggregator to aggregate values per timestamp. As in keep data for lets say 1 minute and see if a datapoint for the same key and timestamp is received. If so sum them. If no such data came, forward the original data to carbon-cache.

Is there a way to do this with graphite or statsd, or do I have to write my own little buffer in front of carbon-cache (instead of carbon-aggregator), that does this? It could be a linked list of self made structures in C resulting in about 150 lines of code, but would take precious time.

Someone has definitely had the same problem, but has anyone found a solution?

Bump: has noone really encountered this issue?

Thanks in advance!

1

Logstash might be one solution.

It can receive/understand carbon metrics (https://www.elastic.co/guide/en/logstash/current/plugins-inputs-graphite.html), and can be made to generate metrics, which seems to do everything except the summing you mention (https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html). It can also re-emit the metrics in a format graphite will understand (https://www.elastic.co/guide/en/logstash/current/plugins-outputs-graphite.html).

You may also be able to do something clever with the aggregate plugin (https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html).

It does sound, though, like if you were able to do something about the way the stats/metrics are being generated in the first place, it might be an easier way to address the issue.

2
  • Changing the way stats are generated is not quite an option since it is done by a 3rd party software. I will look into logstash. Thank you! Will mark the answer as solution if no other appears in a couple of days. – Girtsd Mar 21 '17 at 13:55
  • It appears, the metrics-plugin aggregates per period of log processing -- rather than per the period of the aggregated events occuring. That is, if I "replay" the last week's log, the events recorded in it will be processed as if occurring at the time of replaying... – Mikhail T. Jun 22 '17 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.