The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
15 views

Apache Flume on IBM iSeries

I am new to Flume. My organization uses iSeries (AS400) as its core platform and is currently on version V7R1M0. I intend to capture our server logs for further analysis regarding performance related ...
1
vote
0answers
296 views

Most efficient way to pipe web beacon logs to Apache Flume?

The Setup I've setup a simple Nginx server that logs (in a JSON) format, which is then piped to an S3 bucket with Apache Flume. All the Nginx server does is respond with a web beacon tracking pixel ...
0
votes
1answer
130 views

Host Interceptor gives localhost as hostname in aws

This is a repost of the question I asked in stackoverflow I'm trying to write my apache logs from my ec2 instances. I want the logs separated by hostname. I'm using host interceptor to get the ...
5
votes
1answer
2k views

simple and reliable centralized logging inside Amazon VPC

I need to set up centralized logging for a set of servers (10-20) in an Amazon VPC. The logging should be as to not lose any log messages in case any single server goes offline - or in the case that ...
1
vote
1answer
408 views

Processing pre-existing log files with Flume

I have a large set of log files that I need to extract data from. Is it possible to use Flume to read these files and dump them into an HDFS (Cassandra, or another data source) which I can then ...
1
vote
3answers
772 views

Hadoop Rolling Small files

I am running Hadoop on a project and need a suggestion. Generally by default Hadoop has a "block size" of around 64mb.. There is also a suggestion to not use many/small files.. I am currently having ...
1
vote
1answer
182 views

Consistent Reliable Messaging

Hay All.. Im working a new project, Im currently deciding between flume & scribe for messaging systems... ( most probably sent to logs or hadoop ) I cannot lose a message ever.. What are ...