56

I need to set the file descriptor limit correctly on the docker container I connect to container with ssh (https://github.com/phusion/baseimage-docker)

Already tried:

  • edit limits.conf the container ignore this file
  • upstart procedure found at https://coderwall.com/p/myodcq but this docker image has different kind of init process. (runit)
  • I tried to modify configuration of pam library in /etc/pam.d
  • try to enabled pam for ssh in sshd_config

The output it always the same. 

bash: ulimit: open files: cannot modify limit: Operation not permitted
tshepang
  • 12,111
  • 21
  • 91
  • 136
dnocode
  • 1,928
  • 1
  • 18
  • 21

9 Answers9

68

The latest docker supports setting ulimits through the command line and the API. For instance, docker run takes --ulimit <type>=<soft>:<hard> and there can be as many of these as you like. So, for your nofile, an example would be --ulimit nofile=262144:262144

Glenn
  • 7,262
  • 1
  • 17
  • 23
  • 15
    Does this mean that specific container has higher ulimit than the others? Is the host machine's ulimit remain unchanged? – garbagecollector Sep 03 '15 at 20:14
  • 3
    @garbagecollector Yes, it means that the specific container has higher ulimit than others. This is because if **--ulimit** is not specified in the **docker run** command, then the container inherits the default ulimit from the docker daemon. And also, the host machine's ulimit remain totally unchanged. – Suhas Chikkanna Jan 09 '18 at 18:32
  • 5
    @SuhasChikkanna just to make sure, if the container max "open files" limit is higher than the underlined host max "open files", would the container limit just get ignored? – Victor Bouhnik Jan 28 '18 at 15:08
  • see also https://docs.docker.com/engine/reference/commandline/run/#configure-namespaced-kernel-parameters-sysctls-at-runtime for `--sysctl fs.mqueue.msg_max=10000` style settings – Greg Bray May 02 '19 at 00:16
32

After some searching I found this on a Google groups discussion:

docker currently inhibits this capability for enhanced safety.

That is because the ulimit settings of the host system apply to the docker container. It is regarded as a security risk that programs running in a container can change the ulimit settings for the host.

The good news is that you have two different solutions to choose from.

  1. Remove sys_resource from lxc_template.go and recompile docker. Then you'll be able to set the ulimit as high as you like.

or

  1. Stop the docker demon. Change the ulimit settings on the host. Start the docker demon. It now has your revised limits, and its child processes as well.

I applied the second method:

  1. sudo service docker stop;

  2. changed the limits in /etc/security/limits.conf

  3. reboot the machine

  4. run my container

  5. run ulimit -a in the container to confirm the open files limit has been inherited.

See: https://groups.google.com/forum/#!searchin/docker-user/limits/docker-user/T45Kc9vD804/v8J_N4gLbacJ

Kick Buttowski
  • 6,709
  • 13
  • 37
  • 58
dnocode
  • 1,928
  • 1
  • 18
  • 21
  • 1
    Also there is a useful tutorial about changing `/etc/security/limits.conf` in https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/5/html/Tuning_and_Optimizing_Red_Hat_Enterprise_Linux_for_Oracle_9i_and_10g_Databases/chap-Oracle_9i_and_10g_Tuning_Guide-Setting_Shell_Limits_for_the_Oracle_User.html – Fernando Correia Mar 12 '15 at 00:42
  • 1
    Do you know why it requires a reboot? Why not just restart the shell? – CMCDragonkai Aug 16 '15 at 07:11
17

If using the docker-compose file, Based on docker compose version 2.x We can set like as below, by overriding the default config.

ulimits:
  nproc: 65535
  nofile:
    soft: 26677
    hard: 46677

https://docs.docker.com/compose/compose-file/compose-file-v3/

MohanBabu
  • 407
  • 5
  • 14
10

I have tried many options and unsure as to why a few solutions suggested above work on one machine and not on others.

A solution that works and that is simple and can work per container is:

docker run --ulimit memlock=819200000:819200000 -h <docker_host_name> --name=current -v /home/user_home:/user_home -i -d -t docker_user_name/image_name
Ofir Farchy
  • 7,657
  • 7
  • 38
  • 58
Kabeer Ahmed
  • 141
  • 1
  • 5
5

Actually, I have tried the above answer, but it did not seem to work.

To get my containers to acknowledge the ulimit change, I had to update the docker.conf file before starting them:

$ sudo service docker stop
$ sudo bash -c "echo \"limit nofile 262144 262144\" >> /etc/init/docker.conf"
$ sudo service docker start
Ofir Farchy
  • 7,657
  • 7
  • 38
  • 58
orcaman
  • 6,263
  • 8
  • 54
  • 69
  • These changes don't prevail when machine comes up after reboot but once we restart the services of the docker after machine comes up then container takes the required configuration. – Avinash Singh Dec 17 '14 at 09:14
5

Here is what I did.

set ulimit -n 32000 in the file /etc/init.d/docker

and restart the docker service

docker run -ti node:latest /bin/bash

run this command to verify

user@4d04d06d5022:/# ulimit -a

should see this in the result

open files (-n) 32000

[user@ip ec2-user]# docker run -ti node /bin/bash
user@4d04d06d5022:/# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 58729
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 32000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 58729
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
Jay
  • 4,347
  • 1
  • 17
  • 17
3

The docker run command has a --ulimit flag you can use this flag to set the open file limit in your docker container.

Run the following command when spinning up your container to set the open file limit.

docker run --ulimit nofile=<softlimit>:<hardlimit> the first value before the colon indicates the soft file limit and the value after the colon indicates the hard file limit. you can verify this by running your container in interactive mode and executing the following command in your containers shell ulimit -n

PS: check out this blog post for more clarity

Rifhan Akram
  • 71
  • 3
1

For boot2docker, we can set it on /var/lib/boot2docker/profile, for instance:

ulimit -n 2018

Be warned not to set this limit too high as it will slow down apt-get! See bug #1332440. I had it with debian jessie.

neofreko
  • 136
  • 4
  • 9
0

To set the max_locked_memory while doing docker build itself add the following argument,

docker build --ulimit memlock=-1

which sets the max_locked_memory to unlimited(which can be viewed by using ulimit -l). This is because docker build is running in a seperate user space and has its own File system. As we don't want it to take the host machine's specifications we are adding this command line argument.

To replicate the same changes inside the docker container, run the docker image with the same command

Ex, docker run --ulimit memlock=-1
Vinubama KrishnaMoorthy
  • 1