Consistent Reliable Messaging

Question

Hay All..

Im working a new project,

Im currently deciding between

flume & scribe for messaging systems... ( most probably sent to logs or hadoop )

I cannot lose a message ever.. What are your thoughts on which is better? Configuration/reliablity/adminitration wise.

If you have used / administrated both is also quite relative to answering my question.. :D

Cheers

Answer 1

By far Flume is a better alternative..

Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data..

Flume provides

• Continuous, high-performance, scalability
• Reliable data loading and data flow monitoring from feeds
• Logging directly to HDFS securely (made for hadoop)
• Flexible reliability across the messaging system ( never lose a message )
• Multiple source/collection types ( running on a port / logging from Apache / Syslog securely )
• Robust and fault tolerant with tunable reliability mechanisms
• Failover and recovery mechanisms

Scribe provides "somewhat" reliability, but not as comprehensive or flexible. secondly its compilation and usability is difficult in comparison to flume..

Hope this Helps :D