Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
0answers
275 views

Most efficient way to pipe web beacon logs to Apache Flume?

The Setup I've setup a simple Nginx server that logs (in a JSON) format, which is then piped to an S3 bucket with Apache Flume. All the Nginx server does is respond with a web beacon tracking pixel ...
0
votes
1answer
126 views

Host Interceptor gives localhost as hostname in aws

This is a repost of the question I asked in stackoverflow I'm trying to write my apache logs from my ec2 instances. I want the logs separated by hostname. I'm using host interceptor to get the ...
5
votes
1answer
2k views

simple and reliable centralized logging inside Amazon VPC

I need to set up centralized logging for a set of servers (10-20) in an Amazon VPC. The logging should be as to not lose any log messages in case any single server goes offline - or in the case that ...
1
vote
1answer
378 views

Processing pre-existing log files with Flume

I have a large set of log files that I need to extract data from. Is it possible to use Flume to read these files and dump them into an HDFS (Cassandra, or another data source) which I can then ...
1
vote
3answers
762 views

Hadoop Rolling Small files

I am running Hadoop on a project and need a suggestion. Generally by default Hadoop has a "block size" of around 64mb.. There is also a suggestion to not use many/small files.. I am currently having ...
1
vote
1answer
181 views

Consistent Reliable Messaging

Hay All.. Im working a new project, Im currently deciding between flume & scribe for messaging systems... ( most probably sent to logs or hadoop ) I cannot lose a message ever.. What are ...