Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

When I run the start-all.sh script from my master node, some of my DataNodes fail to start; the log file reports a Java IOException: Incompatible Namespace IDs in /tmp/$MY_USER_NAME.

share|improve this question

2 Answers 2

up vote 1 down vote accepted

When the NameNode is formatted a namespace ID is generated, which essentially identifies that specific instance of the distributed filesystem. When DataNodes first connect to the NameNode they store that namespace ID along with the data blocks, because the blocks have to belong to a specific filesystem.

If a DataNode later connects to a NameNode, and the namespace ID which the NameNode declares does not match the namespace ID stored on the DataNode, it will refuse to operate with the "incompatible namespace ID" error. It means that the DataNode has connected to a different NameNode, and the blocks which it is storing don't belong to that distributed filesystem.

This usually means that you've misplaced your NameNode metadata somehow. If you have multiple HDFS installations your DataNode may be connecting to the wrong NameNode. If you only have a single installation then your NameNode is either running with a different metadata directory, or you've somehow lost the metadata and started with a newly formatted filesystem (which should only happen by running hadoop namenode -format). Try to find the correct NameNode metadata or restore it from a backup.

share|improve this answer

Every time the NameNode reformats it generats a new namespace ID, and some of your machines didn't get the memo. The easiest way to fix this problem is by reformatting the distributed filesystem; retrieve any files you can from the HDFS, then run rm -Rf /tmp/hadoop-$MY_USER_NAME on each server (the directory path may be different depending on your configuration). Then run hadoop namenode -format from your master server. After that, bin/start-all.sh should do the trick.

share|improve this answer
    
Note that this will delete everything stored in HDFS. –  mgorven Jun 28 '12 at 18:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.