Tag Info

New answers tagged

0

Specifically, the Datanode needs to have the dfs.datanode.data.dir configuration property set with the path of each mount point or directory where it is allowed to write data to. This property is a set of comma-separated values. For example: <property> <name>dfs.datanode.data.dir</name> ...


3

zless/zcat/zgrep are just shell wrappers that make gzip output the decompressed data to stdout. To do what you want, you'll just have to write a wrapper around the hadoop fs commands. Aside: The reason this probably didn't work for you is that you're missing an additional slash in your hdfs URI. You wrote: hadoop fs -cat hdfs://myfile.gz | zgrep "hi" ...


0

In short: Yes, when running Hadoop with Kerberos, the authenticated user must exist in the passwd file (or equivalent user directory such as LDAP) on every node where the TaskTracker (MRv1) or YARN nodemanager runs. For MRv1, the TaskTracker launches a program called the task-controller prior to each task starting. The task-controller is a setuid root ...


2

That depends on your topology. If you are spreading your OpenStack environment across multiple racks/switches and plan on moving Hadoop HDFS nodes around, then well, you can't specify a topology (or you need to change it after each move of a VM). It will still work correctly, it will just not be as efficient as on bare metal servers and less outage ...


1

This one has a simple solution. First, if you want to connect to port 8020 of your hadoop container, you should make sure that that port is exposed as well. Second, these containers each have their own loopback interface (localhost) and IP addresses. They are connected through the bridge network docker0 to the host's eth0 interface. So, you need to use the ...


7

My suggestion is to read the information provided. java.net.ConnectException: Connection refused; This generally means nothing is listening on the specified IP:Port. Note also this information is provided http://wiki.apache.org/hadoop/ConnectionRefused and it's first line You get a ConnectionRefused Exception when there is a machine at the ...



Top 50 recent answers are included