I am working with hadoop and need to change number of open files ulimit -n. I have seen similar questions on stackoverflow and elsewhere and have tried everything in those answers, but it still does not work. I am working with ubuntu 12.04 LTS. Here is what i have done:
change limits in /etc/security/limits.conf, i have put in settings for * and root. I have also changed limits to some number like 10000 and unlimited.
* soft nofile 1513687
* hard nofile 1513687
root soft nofile 1513687
root hard nofile 1513687
I have also tried above setting with - instead of soft and hard. After these changes, i have made changes to /etc/pam.d/ files such as:
common-session
common-session-noninterative
login
cron
sshd
su
sudo
i have added session required pam_limits.so to the beginning of each file. I made a restart of the box in question and the settings did not take effect.
I have also found that there were files inside /etc/security/limits.d/ directory for the users hbase mapred and hdfs. I have tried changing limits in these individual files as well to no avail.
I have tried putting ulimit -S -n unlimited inside /etc/profile as well. It did not work.
Finally, i have tried putting limit nofile unlimited unlimited inside /etc/init.d/hadoop* files as first line. Did not work.
One interesting thing though is, i do not have hbase installed on the box but i do have a hbase.conf file inside /etc/security/limits.d/ directory. The settings in this file are reflected with ulimit -n. But settings from hdfs.conf and mapred.conf are not reflected. Which suggests that something is overwriting settings for hdfs and mapred.
I guess i have tried everything people suggested on several forums, is there anything else that i may have missed or done incorrectly?
I am using CDH 4.4.0 as my hadoop distribution.
