2

Is there any reason i would get a 'No space left on device" error on mkdir. I have confirmed that i have enough space left. around 68% is free. and also i have 1% iNodes consumed. This is CentOS 7 with SELinux disabled.

Following is the out of df -h:

Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/xvda1      33540488 10571368  22969120  32% /
devtmpfs         3731924        0   3731924   0% /dev
tmpfs            3616424        0   3616424   0% /dev/shm
tmpfs            3616424   254592   3361832   8% /run
tmpfs            3616424        0   3616424   0% /sys/fs/cgroup
tmpfs            2097152     2880   2094272   1% /tmp

And df -i:

Filesystem       Inodes  IUsed    IFree IUse% Mounted on
/dev/xvda1     33550720 251008 33299712    1% /
devtmpfs         932981    296   932685    1% /dev
tmpfs            904106      1   904105    1% /dev/shm
tmpfs            904106    426   903680    1% /run
tmpfs            904106     13   904093    1% /sys/fs/cgroup
tmpfs            904106     14   904092    1% /tmp

What else could cause this?

edit Although this seems to arise when Jenkins runs a build which would install a newer version of ruby through rbenv and ruby-build.

3
  • Depending on how you install, the process might start by downloading and/or extracting archives to /tmp and that fills up more quickly than your root volume.
    – HBruijn
    Jun 9 '15 at 6:39
  • 2
    Don't disable SELinux. Jun 9 '15 at 6:42
  • @MichaelHampton Yes Definitely. i'm just getting familiar with it first. Jun 9 '15 at 7:20
4

maybe quota is enabled on the system. Check with

repquota -as

or

cat /etc/fstab | grep quota

(did not check if mkdir return No space left in case of quota)

4
  • repquota -as returns nothing. Jun 9 '15 at 7:02
  • cat /etc/fstab | grep quota return nothing as well. Jun 9 '15 at 7:02
  • 1
    try lsof | grep deleted maybe there is a large file deleted but still opened by a process. see stackoverflow question Jun 9 '15 at 7:05
  • lsof | grep deleted return empty Jun 9 '15 at 9:51
-1

To check the free space as Jenkins Job:

Parameters

  • FREE_SPACE: Needed free space in GB.

Job

#!/usr/bin/env bash

free_space="$(df -Ph . | awk 'NR==2 {print $4}')"

if [[ "${free_space}" = *G* ]]; then
  free_space_gb=${x/[^0-9]*/}

  if [[ ${free_space_gb} -lt ${FREE_SPACE} ]]; then
    echo "Warning! Low space: ${free_space}"
    exit 2
  fi
else
  echo "Warning! Unknown: ${free_space}"
  exit 1
fi


echo "Free space: ${free_space}"

Plugins

Set build description

Post-Build Actions

  • Regular expression: Free space: (.*)
  • Description: Free space: \1

  • Regular expression for failed builds: Warning! (.*)

  • Description for failed builds: \1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.