No Space left on device after Jenkins installation

Question

Is there any reason i would get a 'No space left on device" error on mkdir. I have confirmed that i have enough space left. around 68% is free. and also i have 1% iNodes consumed. This is CentOS 7 with SELinux disabled.

Following is the out of df -h:

Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/xvda1      33540488 10571368  22969120  32% /
devtmpfs         3731924        0   3731924   0% /dev
tmpfs            3616424        0   3616424   0% /dev/shm
tmpfs            3616424   254592   3361832   8% /run
tmpfs            3616424        0   3616424   0% /sys/fs/cgroup
tmpfs            2097152     2880   2094272   1% /tmp

And df -i:

Filesystem       Inodes  IUsed    IFree IUse% Mounted on
/dev/xvda1     33550720 251008 33299712    1% /
devtmpfs         932981    296   932685    1% /dev
tmpfs            904106      1   904105    1% /dev/shm
tmpfs            904106    426   903680    1% /run
tmpfs            904106     13   904093    1% /sys/fs/cgroup
tmpfs            904106     14   904092    1% /tmp

What else could cause this?

edit Although this seems to arise when Jenkins runs a build which would install a newer version of ruby through rbenv and ruby-build.

Depending on how you install, the process might start by downloading and/or extracting archives to /tmp and that fills up more quickly than your root volume. — HBruijn, Jun 09 '15 at 06:39
@MichaelHampton Yes Definitely. i'm just getting familiar with it first. — Saad Masood, Jun 09 '15 at 07:20

score 4 · Answer 1 · answered Jun 09 '15 at 06:59

4

maybe quota is enabled on the system. Check with

repquota -as

or

cat /etc/fstab | grep quota

(did not check if mkdir return No space left in case of quota)

answered Jun 09 '15 at 06:59

Henrik Pingel

9,380
2
28
39

repquota -as returns nothing. – Saad Masood Jun 09 '15 at 07:02
cat /etc/fstab | grep quota return nothing as well. – Saad Masood Jun 09 '15 at 07:02
1

try `lsof | grep deleted` maybe there is a large file deleted but still opened by a process. see [stackoverflow question](http://stackoverflow.com/questions/24671621/no-space-left-on-device) – Henrik Pingel Jun 09 '15 at 07:05
`lsof | grep deleted` return empty – Saad Masood Jun 09 '15 at 09:51

score -1 · Answer 2 · answered Oct 06 '19 at 16:00

To check the free space as Jenkins Job:

Parameters

FREE_SPACE: Needed free space in GB.

Job

#!/usr/bin/env bash

free_space="$(df -Ph . | awk 'NR==2 {print $4}')"

if [[ "${free_space}" = *G* ]]; then
  free_space_gb=${x/[^0-9]*/}

  if [[ ${free_space_gb} -lt ${FREE_SPACE} ]]; then
    echo "Warning! Low space: ${free_space}"
    exit 2
  fi
else
  echo "Warning! Unknown: ${free_space}"
  exit 1
fi


echo "Free space: ${free_space}"

Plugins

Set build description

Post-Build Actions

Regular expression: Free space: (.*)
Description: Free space: \1
Regular expression for failed builds: Warning! (.*)
Description for failed builds: \1

No Space left on device after Jenkins installation

2 Answers2

Parameters

Job

Plugins

Set build description

Post-Build Actions