0

I have created virtual box, ubuntu 14.04LTS environment on my mac machine. In virtual box of ubuntu, I've created cluster of three lxc-containers. One for master and other two nodes for slaves.

On master, I have started installation of CDH5 using following link http://archive.cloudera.com/cm5/installer/latest/cloudera-manager-installer.bin

I have also made necessary changes in the /etc/hosts including FQDN and hostnames. Also created passwordless user named as "ubuntu".

While setting up the CDH5, during installation I'm constantly facing following error on datanodes. Max locked memory size: dfs.datanode.max.locked.memory of 922746880 bytes is more than the datanode's available RLIMIT_MEMLOCK ulimit of 65536 bytes.

Exception in secureMain: java.lang.RuntimeException: Cannot start datanode because the configured max locked memory size (dfs.datanode.max.locked.memory) of 922746880 bytes is more than the datanode's available RLIMIT_MEMLOCK ulimit of 65536 bytes.
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1050)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:411)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2297)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2184)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2231)
at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2407)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2431)
Krunal C
  • 1
  • 3

2 Answers2

0

Krunal,

This solution will be probably be late for you but maybe it can help somebody else so here it is. Make sure your ulimit is set correctly. But in case its a config issue.

goto:

/run/cloudera-scm-agent/process/

find latest config dir, in this case:

1016-hdfs-DATANODE

search for parameter in this dir:

grep -rnw . -e "dfs.datanode.max.locked.memory"
./hdfs-site.xml:163:    <name>dfs.datanode.max.locked.memory</name>

and edit the value to the one he is expecting in your case(65536)

JakeJ
  • 13
  • 1
  • 8
0

I solved by opening a seperate tab in Cloudera and set the value from there

enter image description here

Jacob Nordfalk
  • 3,533
  • 1
  • 21
  • 21