The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
1answer
20 views

Updating group without log out or subshell

I'm trying to run Docker on Elastic MapReduce streaming but am having trouble with a permissions issue. In my bootstrap script, I need the "hadoop" user to be part of the "docker" group (as described ...
0
votes
0answers
17 views

TEZ: DAG execution time vs Application execution time in Tez-ui

I have a situation when DAG execution time, displayed in tez-ui, is much different from Application execution time. For example DAG task may take about 2-3 minutes, but total App exec time would be ...
0
votes
1answer
115 views

How to view status of recent AppEngine mapreduce jobs?

We recently upgraded our App Engine application to GAE SDK 1.9, and upgraded the older MapReduce library we'd been using to the most recent version hosted on GitHub. We now find that the old MapReduce ...
0
votes
0answers
161 views

Distributing Master node ssh key

For the master node to passwordless-ly ssh into the slaves, the master needs to distribute its ssh key to the slaves. Copying key using ssh-copy-id asks for the user password. If there are hundreds of ...
3
votes
2answers
1k views

hadoop-config.sh in bin/ and libexec/

While setting up hadoop, I found that hadoop-config.sh script is present in two directories, bin/ and libexec/. Both the files are identical. While looking onto scripts, I found that if hadoop-config....
0
votes
1answer
4k views

MongoDB Locking - Very, very, slow to read

This is the output from db.currentOp(): > db.currentOp() { "inprog" : [ { "opid" : 2153, "active" : false, "op" : "update", "ns" : "", ...
1
vote
1answer
297 views

MapReduce job is hung after 1 of 5 reducers completed on single-node environment

I have only one Data Node on my dev environment on EC2. I ran heavy MR job and in 6 hours noticed that 100% of mappers and 20% of reducers finished (1 of reducer shows 100% competition, other ones - 0%...
1
vote
0answers
142 views

How to increase the performance on Amazon Elastic Mapreduce for job execution?

My task is: Initially I want to import the data from MS SQL Server into HDFS using SQOOP. Through Hive I am processing the data and generating the result in one table That result containing table ...
4
votes
1answer
411 views

How do I define the timeout for bootstrap actions on Amazon's Elastic MapReduce?

How do I change the timeout for bootstrap actions on Amazon's Elastic MapReduce?
3
votes
1answer
657 views

Best practice for administering a (hadoop) cluster

I've recently been playing with Hadoop. I have a six node cluster up and running - with HDFS, and having run a number of MapRed jobs. So far, so good. However I'm now looking to do this more ...
1
vote
3answers
861 views

Hadoop Rolling Small files

I am running Hadoop on a project and need a suggestion. Generally by default Hadoop has a "block size" of around 64mb.. There is also a suggestion to not use many/small files.. I am currently having ...