Datanode process not running in Hadoop

You need to do something like this:

• bin/stop-all.sh (or stop-dfs.sh and stop-yarn.sh in the 2.x serie)
• rm -Rf /app/tmp/hadoop-your-username/*
• bin/hadoop namenode -format (or hdfs in the 2.x serie)

the solution was taken from: http://pages.cs.brandeis.edu/~cs147a/lab/hadoop-troubleshooting/. Basically it consists in restarting from scratch, so make sure you won't loose data by formating the hdfs.

I faced similar issue while running the datanode. The following steps were useful.

1. In [hadoop_directory]/sbin directory use ./stop-all.sh to stop all the running services.
2. Remove the tmp dir using rm -r [hadoop_directory]/tmp (The path configured in [hadoop_directory]/etc/hadoop/core-site.xml)
3. sudo mkdir [hadoop_directory]/tmp (Make a new tmp directory)

4. Go to */hadoop_store/hdfs directory where you have created namenode and datanode as sub-directories. (The paths configured in [hadoop_directory]/etc/hadoop/hdfs-site.xml). Use

   rm -r namenode
   
   rm -r datanode
   

5. In */hadoop_store/hdfs directory use

   sudo mkdir namenode
   
   sudo mkdir datanode
   

In case of permission issue, use

   chmod -R 755 namenode 

   chmod -R 755 datanode

6. In [hadoop_directory]/bin use

    hadoop namenode -format (To format your namenode)
   

7. In [hadoop_directory]/sbin directory use ./start-all.sh or ./start-dfs.sh to start the services.
8. Use jps to check the services running.

I ran into the same issue. I have created a hdfs folder '/home/username/hdfs' with sub-directories name, data, and tmp which were referenced in config xml files of hadoop/conf.

When I started hadoop and did jps, I couldn't find datanode so I tried to manually start datanode using bin/hadoop datanode. Then I realized from error message that it has permissions issue accessing the dfs.data.dir=/home/username/hdfs/data/ which was referenced in one of the hadoop config files. All I had to do was stop hadoop, delete the contents of /home/username/hdfs/tmp/* directory and then try this command - chmod -R 755 /home/username/hdfs/ and then start hadoop. I could find the datanode!