1

i've configured hadoop 2.7.3 on ubuntu 16.04 and run all (word count and other mapReduce run all). after restart pc, i launch start-dfs, but the namenode not started. other guide says remove temporay directory, but i don't have it.

that are my files:

core-site.xml

<configuration>
 <property>
   <name>fs.defaultFS</name>
   <value>hdfs://localhost:9000</value>
 </property>
</configuration>

hdfs-site.xml

<configuration>
 <property>
   <name>dfs.replication</name>
   <value>1</value>
 </property>
</configuration>
Community
  • 1
  • 1
Alessio
  • 23
  • 9
  • Can you share the name node logs ? – Deepan Ram Apr 04 '17 at 12:59
  • Possible duplicate of [Pseudo Distributed Mode Hadoop](http://stackoverflow.com/questions/43177670/pseudo-distributed-mode-hadoop) – franklinsijo Apr 04 '17 at 13:09
  • @DeepanRam this is the log: 2017-04-04 15:25:36,095 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-alessio/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. – Alessio Apr 04 '17 at 13:29
  • The folder and structure under the "/tmp" are deleted after reboot. Try changing them to local structure and try to give 775 permissions to them and restart the services. – Deepan Ram Apr 04 '17 at 13:57
  • @DeepanRam how i can change "/tmp" and give 775 permissions? – Alessio Apr 04 '17 at 14:01
  • chmod 775 -R . Please change the **dfs.namenode.name.dir** and **dfs.datanode.data.dir** in the hdfs-site.xml with local dir (other than tmp) and give the permissions on the directory. – Deepan Ram Apr 04 '17 at 15:38

1 Answers1

0
Step 1: Create the folders like below:
      -- sudo mkdir /dfs
      -- sudo chown username:username /dfs/*
      -- sudo chmod 755 /dfs/
      -- cd /dfs
      If the machine is name node then:
          -- mkdir nn 
      if data node: 
          -- mkdir data

Step 2:
add below properties in hdfs-site.xml
<configuration>
        <configuration>
            <property>
                <name>dfs.replication</name>
                <value>1</value>
            </property>
        </configuration>
        <configuration>
            <property>
                <name>dfs.namenode.name.dir</name>
                <value>/dfs/nn</value>
            </property>
        </configuration>
        <configuration>
            <property>
                <name>dfs.datanode.data.dir</name>
                <value>/dfs/data</value>
            </property>
        </configuration>


</configuration>
step 3:
 format the namenode: hadoop namenode -format
step 4:
 start all services
Kanagaraj Dhanapal
  • 451
  • 2
  • 10
  • now don't start datanode (and namenode)! there are errors on paths of namenode and datanode (dfs/nn and dfs/data) – Alessio Apr 04 '17 at 13:15
  • ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. org.apache.hadoop.hdfs.server.common.InconsistentFSStateExce‌​ption: Directory /tmp/hadoop-alessio/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. – Alessio Apr 04 '17 at 13:33
  • yes, but at every restart, the folder tmp are deleted, and the the namenode is not created – Alessio Apr 04 '17 at 14:00
  • could you share the log message for "hadoop namenode -format"? – Kanagaraj Dhanapal Apr 04 '17 at 14:11
  • send me the log message to this mail id: kanagaraj.vp@gmail.com – Kanagaraj Dhanapal Apr 04 '17 at 14:11
  • i have clearly mentioned the steps but i am not sure whether he followed all the steps or not. could you tell me in which part i am making confusions – Kanagaraj Dhanapal Apr 04 '17 at 15:12
  • You have mentioned the paths as `/dfs/nn` and `/dfs/data`. The path `/dfs` does not exist by default. If the OP is new to linux, he wouldn't know and the format would still fail. Instruct the OP that such directories must be created and should be owned by the superuser. – franklinsijo Apr 08 '17 at 15:38