The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
27 views

Distributing Master node ssh key

For the master node to passwordless-ly ssh into the slaves, the master needs to distribute its ssh key to the slaves. Copying key using ssh-copy-id asks for the user password. If there are hundreds of ...
1
vote
1answer
23 views

hadoop-config.sh in bin/ and libexec/

While setting up hadoop, I found that hadoop-config.sh script is present in two directories, bin/ and libexec/. Both the files are identical. While looking onto scripts, I found that if ...
0
votes
0answers
20 views

Can I use Yarn to dynamically assign nodes for map reduce tasks

When I use Hadoop for executing the Map reduce tasks, I am unable to find the API to allow me assign the compute nodes for executing the task. Is there any simple way to let Resource manager ...
0
votes
1answer
834 views

MongoDB Locking - Very, very, slow to read

This is the output from db.currentOp(): > db.currentOp() { "inprog" : [ { "opid" : 2153, "active" : false, "op" : "update", "ns" : "", ...
1
vote
1answer
112 views

MapReduce job is hung after 1 of 5 reducers completed on single-node environment

I have only one Data Node on my dev environment on EC2. I ran heavy MR job and in 6 hours noticed that 100% of mappers and 20% of reducers finished (1 of reducer shows 100% competition, other ones - ...
1
vote
0answers
126 views

How to increase the performance on Amazon Elastic Mapreduce for job execution?

My task is: Initially I want to import the data from MS SQL Server into HDFS using SQOOP. Through Hive I am processing the data and generating the result in one table That result containing table ...
4
votes
1answer
198 views

How do I define the timeout for bootstrap actions on Amazon's Elastic MapReduce?

How do I change the timeout for bootstrap actions on Amazon's Elastic MapReduce?
3
votes
1answer
565 views

Best practice for administering a (hadoop) cluster

I've recently been playing with Hadoop. I have a six node cluster up and running - with HDFS, and having run a number of MapRed jobs. So far, so good. However I'm now looking to do this more ...
1
vote
3answers
728 views

Hadoop Rolling Small files

I am running Hadoop on a project and need a suggestion. Generally by default Hadoop has a "block size" of around 64mb.. There is also a suggestion to not use many/small files.. I am currently having ...
2
votes
4answers
621 views

Distributed grep

Is there a solution for a distributed grep? here's the story: I have a bunch of web servers and want to query their application logs (I'm using tomcat, if it matters). I don't want to have to copy ...