1

I'm attempting to run Hadoop in pseudo-distributed mode to learn how the system work. To install it, I've downloaded Hadoop-3.0.0 from the site, untarred it. I've done my configurations as follows (leaving out the configuration tags for brevity):

core-site.xml

<property>
   <name>fs.defaultFS</name>
   <value>hdfs://localhost/</value>
</property>

hdsf-site.xml

<property>
   <name>dfs.replication</name>
   <value>1</value>
</property>

mapred-site.xml

<property>
   <name>mapreduce.framework.name</name>
   <value>yarn</value> 
</property>

yarn-site.xml

<property>
   <name>yarn.resourcemanager.hostname</name>
   <value>localhost</value> </property>
<property>
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value> 
</property>

After doing this, I've formatted my hdfs using 

hdfs namenode -format

I've also setup passwordless ssh using the following:

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa2
cat ~/.ssh/id_rsa2.pub >> ~/.ssh/authorized_keys

(I've also added id_rsa2.pub as the default for localhost using a config file, since I already was using id_rsa.pub for something else and didn't want to mix-and-match in case I broke something)

I'm able to ssh into localhost. All looks well.

Then I run start-dfs.sh, and I see this error:

Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [zm.local]
zm.local: zm@zm.local: Permission denied (publickey,password,keyboard-interactive).
2018-01-16 17:31:35,807 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

If I run jps (after starting yarn and mapreduce history server), I have the following:

37921 NodeManager
38070 Jps
37434 NameNode
38060 JobHistoryServer
37821 ResourceManager

Noticeably, the SecondaryNameNode is missing, my assumption being it's due to the error above.

I can then try to use hadoop's fs command and I'm able to create a folder and look it up. But if I try to copy any data over, I get notified that the NameNode is in SAFEmode. If I turn off save mode using:

hdfs dfsadmin -safemode leave

It immediately turns back on. By going to the namenode port on localhost, I see the following message:

Safe mode is ON. Resources are low on NN. Please add or free up more resourcesthen turn off safe mode manually. NOTE: If you turn off safe mode before adding resources, the NN will immediately return to safe mode. Use "hdfs dfsadmin -safemode leave" to turn safe mode off.

However, I have plenty of resources. The single datanode is using less than 8% of it's allotted space, the namenode as almost 100GB of space. The datanode and namenode are both reporting as healthy. Thus, I think the problem is the lack of a secondary namenode. With that in mind, is anyone aware what might be causing the SecondaryNameNode to have different permission issues from the PrimaryNameNode? It seems to be trying to put the sNN somewhere on the local machine instead - but when I check in /tmp/hadoop*, all of the file permissions seem to be normal.

Thanks for any help.

Zach
  • 341
  • 1
  • 3
  • 14
  • You ideally should not have your files in /tmp and the secondary namenode is not really a "standby namenode", despite the naming – OneCricketeer Jan 18 '18 at 03:20
  • Thanks for the info. Is there a config setting to tell it where to store the secondary namenode? Would storing it in /tmp be the cause of the Permission Error? – Zach Jan 18 '18 at 19:28
  • In `hdfs-site.xml`, `dfs.namenode.checkpoint.dir` property defaults to `file://${hadoop.tmp.dir}/dfs/namesecondary` and `hadoop.tmp.dir` exists in the `core-site.xml` and defaults to `/tmp/hadoop-${user.name}`. My point is that your OS might be installed in such a way that `/tmp` is mounted on its own partition of the filesystem with limited space. – OneCricketeer Jan 18 '18 at 19:46
  • Got it. I see now what you mean. Thanks, I'll try changing that and see if that works. – Zach Jan 18 '18 at 20:07

0 Answers0