Hadoop Full Mode

Hadoop Fully Distributed Mode [ 하둡 완전분산 모드 ]

1. host file change 

     NameNode h001 / Secondary NameNode h002 / DataNode h002, h003, h004

2. java installation

3. java env configuration setting

4. ssh check among h001 ~ h004

5. hadoop installation

6. hadoop conf file change

  • hadoop-env

  • masters  - secondary name node

  • slaves - data nodes

  • core-site.xml

  • hdfs-site.xml

  • mapred-site.xml

7. hadoop name node format

data nodes don't go up and run so I checked the logs 

and got this message below

      "datanode.DataNode: java.io.IOException: Incompatible namespaceIDs"

It seems that some others had same kind of proble when they run Hadoop

[ from http://stackoverflow.com/questions/3425688/why-does-the-hadoop-incompatible-namespaceids-issue-happen ]

Namenode generates new namespaceID every time you format HDFS. I think this is possibly to differentiate current version and previous version. You can always rollback to previous version if something is not proper which may not be possible if namespaceID is not unique for every formatted instance.

NamespaceID also connects namenode and datanodes. Datanodes bind themselves to namenode through namespaceID