Namenode format and starting the namenode in hadoop2.2.0

Question

I am trying to install Single node configuration, I succeeded to certain point like downloading and unzipping the tar and editing all the configuration files, but faced problems when formatting the namenode.

The errors are:

14/07/14 02:56:40 WARN fs.FileUtil: Failed to delete file or dir [/home/renu123/yarn/yarn_data/hdfs/namenode/current/seen_txid]: it still exists.
14/07/14 02:56:40 WARN fs.FileUtil: Failed to delete file or dir [/home/renu123/yarn/yarn_data/hdfs/namenode/current/VERSION]: it still exists.
14/07/14 02:56:40 WARN fs.FileUtil: Failed to delete file or dir [/home/renu123/yarn/yarn_data/hdfs/namenode/current/fsimage_0000000000000000000]: it still exists.
14/07/14 02:56:40 WARN fs.FileUtil: Failed to delete file or dir [/home/renu123/yarn/yarn_data/hdfs/namenode/current/fsimage_0000000000000000000.md5]: it still exists.
14/07/14 02:56:40 FATAL namenode.NameNode: Exception in namenode join
java.io.IOException: Cannot remove current directory: /home/renu123/yarn/yarn_data/hdfs/namenode/current
    at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:299)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:523)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:544)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:147)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:837)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1213)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)
14/07/14 02:56:40 INFO util.ExitUtil: Exiting with status 1
14/07/14 02:56:40 INFO namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/

After searching on google I formatted namenode by switching to root account as below:

root@ubuntu:~/yarn/yarn_data/hdfs/namenode# hdfs namenode -format
Re-format filesystem in Storage Directory /home/renu123/yarn/yarn_data/hdfs/namenode ? (Y or N) Y
14/07/14 02:59:30 INFO common.Storage: Storage directory /home/renu123/yarn/yarn_data/hdfs/namenode has been successfully formatted.
14/07/14 02:59:30 INFO namenode.FSImage: Saving image file /home/renu123/yarn/yarn_data/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 using no compression
14/07/14 02:59:30 INFO namenode.FSImage: Image file /home/renu123/yarn/yarn_data/hdfs/namenode/current/fsimage.ckpt_0000000000000000000 of size 196 bytes saved in 0 seconds.
14/07/14 02:59:30 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
14/07/14 02:59:30 INFO util.ExitUtil: Exiting with status 0
14/07/14 02:59:30 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/

Even after formatting still I can see the directories in hdfs/namenode folder i.e current

Assuming the namenode is formatted I proceeded further and tried to start namenode using the command start-dfs.sh, but wasn't able to.

This is the log take out from usr/local/hadoop/logs:

2014-07-14 02:49:10,478 WARN org.apache.hadoop.hdfs.server.common.Storage: Cannot access storage directory /home/renu123/yarn/yarn_data/hdfs/namenode
2014-07-14 02:49:10,484 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:50070
2014-07-14 02:49:10,585 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system...
2014-07-14 02:49:10,585 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped.
2014-07-14 02:49:10,586 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete.
2014-07-14 02:49:10,587 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/renu123/yarn/yarn_data/hdfs/namenode is in an inconsistent state: storage directory does not exist or is not accessible.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:292)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:200)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:787)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:568)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:443)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:491)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:684)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:669)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1254)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1320)
2014-07-14 02:49:10,590 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1
2014-07-14 02:49:10,602 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:

Help me find the solution to this.

My configuration files are

core-site.xml

<configuration>
<property>
   <name>fs.default.name</name>

   <value>hdfs://localhost:9000</value>
</property>
</configuration>

yarn-ste.xml

<configuration>
<!-- Site specific YARN configuration properties -->
<property>
   <name>yarn.nodemanager.aux-services</name>

   <value>mapreduce_shuffle</value>
</property>
<property>
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>

   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

mapred-site.xml

<configuration>
<property>
   <name>mapreduce.framework.name</name>

   <value>yarn</value>
</property>
</configuration>

hdfs-site.xml

<configuration>
<property>
   <name>dfs.replication</name>

   <value>1</value>
 </property>
 <property>
   <name>dfs.namenode.name.dir</name>

   <value>file:/home/renu123/yarn/yarn_data/hdfs/namenode</value>
 </property>
 <property>
   <name>dfs.datanode.data.dir</name>

   <value>file:/home/renu123/yarn/yarn_data/hdfs/datanode</value>
 </property>
</configuration>

Can you update please with your configuration files? It seems like your resource manager isn't making the node fail — eliasah, Jul 14 '14 at 10:34
If you couldn't edit your configuration files how did you configure your cluster then? — eliasah, Jul 14 '14 at 15:41
I said i am not able to edit the question with configuration files.but Parto edited my question with good format. now u can see the exact question if u know the solution plz provide thanks — Renukeswar, Jul 15 '14 at 02:25
Finally i am able to complete the installation of hadoop in my machine , mistake i have done is namenode format is incorrect.solution is to give proper file permissions to hadoop installation folder.Command used for this is sudo chown -R hduser:hadoop hadoop — Renukeswar, Jul 15 '14 at 12:53
It is a very common problem. Hadoop fails are usually related to permissions and configuration files. — eliasah, Jul 15 '14 at 13:12

Vikas Hardia · Answer 1 · 2015-06-15T12:05:45.630

0

there may be a problem due to file keyword you have used in

<configuration>
<property>
   <name>dfs.replication</name>

   <value>1</value>
 </property>
 <property>
   <name>dfs.namenode.name.dir</name>

   **<value>file:/home/renu123/yarn/yarn_data/hdfs/namenode</value>**
 </property>
 <property>
   <name>dfs.datanode.data.dir</name>

   **<value>file:/home/renu123/yarn/yarn_data/hdfs/datanode</value>**
 </property>
</configuration>

you may need to remove this "file:" and specify as below

/home/renu123/yarn/yarn_data/hdfs/namenode

and

/home/renu123/yarn/yarn_data/hdfs/datanode

edited Jun 15 '15 at 12:05

answered Jul 15 '14 at 07:36

Vikas Hardia

833
1
9
12

Thanks for reply but it did not worked.I removed file: from the configuration file and tried to start namenode same error i got. – Renukeswar Jul 15 '14 at 11:02
I followed 'https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-on-ubuntu-13-10' URL to set up my Hadoop environment – Renukeswar Jul 15 '14 at 11:20
@Renukeswar link is not working and it must be the issue of rights if not of the file – Vikas Hardia Jul 15 '14 at 13:05

score 0 · Answer 2 · edited Feb 08 '17 at 10:52

@Renukeshwar, I had the same problem. If you look at these lines in your error logs,

java.io.IOException:Cannot remove directory:/home/renu123/yarn/yarn_data/hdfs/namenode/current
java.io.IOException: Cannot create directory: /home/renu123/yarn/yarn_data/hdfs/namenode/current

This happens when the current user does not have read and write access to that folder.

Solution

After changing all the configurations in all .xml files, type this in terminal. This gives the user hduser of group hadoop access to the folder.

sudo chown -R hduser:hadoop /usr/local/hadoop

Note: To check which group user belongs, type groups your-username-here

Edit: you may have to remove tmp directories.

sudo rm -R /path-to-your-common-storage

score 0 · Answer 3 · edited Oct 10 '14 at 06:58

If you have followed the installation steps given in link (@Vikas Hardia - Link is working just ignore appended ' at last - Might be typo :) - https://www.digitalocean.com/community/tutorials/how-to-install-hadoop-on-ubuntu-13-10)

Then below should work for you.

Solution

sudo chown -R hduser:hadoop /usr/local/hadoop
sudo chown -R hduser:hadoop /usr/local/hadoop_store

This has worked for me, If any issue just make sure you have changed .bashrc file for your hadoop-user properly.

score 0 · Answer 4 · edited Feb 08 '17 at 15:37

0

Format the namenode using

hadoop namenode -format

Further reading: https://wiki.apache.org/hadoop/GettingStartedWithHadoop#Formatting_the_Namenode

edited Feb 08 '17 at 15:37

Elder Geek

36,023
25
98
183

answered Feb 08 '17 at 10:50

Shahida Khan

1

Namenode format and starting the namenode in hadoop2.2.0

4 Answers4

Solution