1

Sometimes we experience a drone pipeline failing due to a lack of disk space, but there is a lot of space.

drone@drone2:~$ df -H
Filesystem      Size  Used Avail Use% Mounted on
udev            8.4G     0  8.4G   0% /dev
tmpfs           1.7G  984k  1.7G   1% /run
/dev/vda2       138G   15G  118G  12% /
tmpfs           8.4G     0  8.4G   0% /dev/shm
tmpfs           5.3M     0  5.3M   0% /run/lock
tmpfs           8.4G     0  8.4G   0% /sys/fs/cgroup
overlay         138G   15G  118G  12% /var/lib/docker/overlay2/cf15a2d5e215d6624d9239d28c34be2aa4a856485a6ecafb16b38d1480531dfc/merged
overlay         138G   15G  118G  12% /var/lib/docker/overlay2/c4b441b59cba0ded4b82c94ab5f5658b7d8015d6e84bf8c91b86c2a2404f81b2/merged
tmpfs           1.7G     0  1.7G   0% /run/user/1000

We use docker images for running drone infostracture. The console command:

docker run \
  --volume=/var/lib/drone:/data \
  --publish=80:80 \
  --publish=443:443 \
  --restart=always \
  --detach=true \
  --name=drone \
  <drone image>

My assumption may be due to the docker container's limitations, and we need to configure it somehow manually.

Any suggestion on how to fix it?

112Legion
  • 1,129
  • 12
  • 25
  • How much space, and free inodes, are on the host when the error occurs (not after drone has cleaned up, or before drone has gotten to the error)? – BMitch Oct 28 '21 at 12:34
  • @BMitch I will try to set up prometheus metic on that one and will give you the detail feedback. – 112Legion Oct 28 '21 at 16:08

1 Answers1

0

We have manage to identify the problem. Our cloud provider has confirmed that our software has used the disc a lot this lead to triggering limitations on disk operation. They gave us a recommendation to increase the disk plan. Or optimise the disk usage.

112Legion
  • 1,129
  • 12
  • 25