1

I have 3 node instances -

master, slave1 and slave2

SSHing between these nodes works fine.

Here is the processes that starts on each node when I say ./sbin/start-dfs.sh

master:

SecondaryNameNode
Jps
NameNode
DataNode

slave1:

Jps
DataNode

slave2:

Jps
DataNode

But when I try to access hdfs from slave nodes, I get a connection timed out.

Also when I check the hdfs dfsadmin -report, I only see one datanode(on master's localhost) as part of hdfs.

Datanodes available: 1 (1 total, 0 dead)

Live datanodes:
Name: 127.0.0.1:50010 (localhost)
Hostname: master
Decommission Status : Normal
Configured Capacity: 8309932032 (7.74 GB)
DFS Used: 24576 (24 KB)
Non DFS Used: 2309738496 (2.15 GB)
DFS Remaining: 6000168960 (5.59 GB)
DFS Used%: 0.00%
DFS Remaining%: 72.20%

Here is my /etc/hosts file mapping on all the three nodes.

127.0.0.1 localhost
<ip of master> master
<ip of slave1> slave1
<ip of slave2> slave2

Here is my $HADOOP_HOME/etc/hadoop/slaves file on the master

localhost
slave1
slave2

In short, datanodes on slaves are unable to connect with hdfs.

What am I doing wrong?

5
  • Have you set fs.default.name correctly on all three nodes in conf/core-site.xml? Aug 7, 2014 at 16:20
  • @VictorJerlin I have set fs.default.name as hdfs://master:9000 in all the 3 nodes.
    – Prasanna
    Aug 7, 2014 at 16:23
  • Just noticed the part about connection timing out. If you can't telnet to port 9000 on the master from the slaves(and the error is connection timed out), then it's most likely a firewall issue. Check that the ec2 instances are in the same ec2 security group, and that there's no iptables firewall rules active blocking the connections. Aug 7, 2014 at 16:32
  • They all belong to the same security group. Though, the rule is SSH TCP 22 0.0.0.0/0 Is this fine?
    – Prasanna
    Aug 7, 2014 at 16:50
  • @VictorJerlin The answer about the firewall is right. It works when I opened all incoming traffic. Please post your answer so that I would accept.
    – Prasanna
    Aug 7, 2014 at 17:22

1 Answer 1

1

If you can't telnet to port 9000 on the master from the slaves(and the error is connection timed out), then it's most likely a firewall issue.

Check that the ec2 instances are in the same ec2 security group, and that there's no iptables firewall rules active blocking the connections(you can check this by running iptables -vnxL).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .