Add new comment

Hi!

statsd is a simple server that listens to UDP messages, aggregates them and sends them on to carbon every 10 seconds. The problem we had with it is that it's single-threaded and doesn't seem to be very efficient, so under load it quickly goes up to 100% CPU usage on a single core and then becomes a bottleneck for logging, when multiple high-frequency systems send UDP messages to it.

The solution was jus to bypass statsd and have the systems that needed to log things just do a little bit of local buffering/aggregating and then send the aggregated stats directly to carbon. This is more efficient and spreads the load around multiple services and cores. So the answer is that aggregation is happening in the origin application itself, in order to cut down on messages being sent to carbon, just like it was being done in statsd before.

Hope that helps,
Nicolas

Submitted by Nicolas Kruchten on

RSS Feed