Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

This question already has an answer here:

My Java Application returns following Exception when saving a new file in /opt/wso2 on a CentOS 6.4:

Caused by ... (No space left on device)

Caused by: /opt/wso2/FrameworkFiles/trk_2014062500042488825_TRCK_PatfallHospis_pFromHospis_66601fb3-a03c-4149-93c3-6892e0a10fea.txt (No space left on device)
at Method)
at com.avintis.esb.framework.adapter.wso2.FrameworkAdapterWSO2.sendMessages(
... 23 more

But when I run df -a I can see that the partition still has plenty of space available:

[root@stzsi466 wso2]# df -a
Filesystem           1K-blocks      Used Available Use% Mounted on
                      12054824   2116092   9326380  19% /
proc                         0         0         0   -  /proc
sysfs                        0         0         0   -  /sys
devpts                       0         0         0   -  /dev/pts
tmpfs                  4030764         0   4030764   0% /dev/shm
/dev/sda1               495844     53858    416386  12% /boot
/dev/sdb1             51605436   1424288  47559744   3% /opt/wso2
none                         0         0         0   -  /proc/sys/fs/binfmt_misc

[root@stzsi466 ~]# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
                      765536   45181  720355    6% /
tmpfs                1007691       1 1007690    1% /dev/shm
/dev/sda1             128016      44  127972    1% /boot
/dev/sdb1            3276800    6137 3270663    1% /opt/wso2

What is the problem here? Is it caused by the Java on CentOS 6.4? I have another server running Redhat REHL 6.4 and all works fine - same Java etc.

Does anyone know of this problem?

share|improve this question

marked as duplicate by Ward, mdpc, Rex, Michael Hampton Aug 23 '14 at 18:47

This question was marked as an exact duplicate of an existing question.

Please, clirify your question in following: - FS type in use - mount params - file permissions of target file directory and it's parents. This will help to narrow correct answer. Also: check inodes with command: df -i – cgi Aug 19 '14 at 12:41
Is this a standalone box or is it a shared cloud 'instance'? I have seen virtual servers do this. – Jeff Clayton Aug 19 '14 at 12:43
In the virtual server that was problematic, the server instance was allocated a specific amount of hard drive space. The server that contained it also covered automatic backups. The containing server took up more space and encroached on the virtual server's space. The virtual server reported what it was told it should have, but could not expand to it's full size. VMWare based servers are susceptible for example, and the cause could also have been a swap space or other server instance 'crowding it out'. The virtual server I am referring to was a CentOS 6.5 instance as well. – Jeff Clayton Aug 19 '14 at 12:54
In a VM it is often pre-set to tell the internal server that it has a certain space, but to grow to that size as files are added, instead of requisitioning the whole block of space so it doesn't take up space on the containing box until needed. If your server is standalone, this should not happen of course. – Jeff Clayton Aug 19 '14 at 12:56
Thanks for the responses. YES it is a Virtual Server. I'll post now the different FS informations... – FiveO Aug 19 '14 at 13:15

So from the stack trace we can see that when the FileOutputStream object is being created, inside that code, it makes a call to open() that fails and then it throws a FileNotFoundException with the "No space left on device" error message.

It's interesting to look at the code that is failing (

       public FileOutputStream(File file, boolean append)
           throws FileNotFoundException
           String name = (file != null ? file.getPath() : null);
           SecurityManager security = System.getSecurityManager();
           if (security != null) {
           if (name == null) {
               throw new NullPointerException();
           this.fd = new FileDescriptor();
           this.append = append;

           open(name, append);                /* HERE */

So this is a call to native code (FileOutputStream_md.c):

Java_java_io_FileOutputStream_open(JNIEnv *env, jobject this,
                                   jstring path, jboolean append) {
    fileOpen(env, this, path, fos_fd,
             O_WRONLY | O_CREAT | (append ? O_APPEND : O_TRUNC));

And fileOpen (io_util_md.c):

fileOpen(JNIEnv *env, jobject this, jstring path, jfieldID fid, int flags)
    WITH_PLATFORM_STRING(env, path, ps) {
        FD fd;

#ifdef __linux__
        /* Remove trailing slashes, since the kernel won't */
        char *p = (char *)ps + strlen(ps) - 1;
        while ((p > ps) && (*p == '/'))
            *p-- = '\0';
        fd = JVM_Open(ps, flags, 0666);              /* HERE IS WHERE THINGS BREAK */
        if (fd >= 0) {
            SET_FD(this, fd, fid);
        } else {
            throwFileNotFoundException(env, path);   /* EXCEPTION */
    } END_PLATFORM_STRING(env, ps);

And JVM_Open (jvm.cpp):

JVM_LEAF(jint, JVM_Open(const char *fname, jint flags, jint mode))
  JVMWrapper2("JVM_Open (%s)", fname);

  //%note jvm_r6
  int result = os::open(fname, flags, mode);
  if (result >= 0) {
    return result;
  } else {
    switch(errno) {
      case EEXIST:
        return JVM_EEXIST;       /* returns -100, will trigger Exception */
        return -1;               /* returns -1, will trigger Exception */

And os::open is just a wrapper around fdopen(2) syscall.

Can you reproduce this error whenever you want? If so, please run the command with strace -e trace=fopen,fdopen,freopen and post results (or attach to running process with strace -p <PID>).

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.