Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The mysql server is running on a newly installed ubuntu 8.04. (I wanted to move the mysql server into KVM. KVM itself is running on ubuntu 10.04. )

/mysql belongs to mysql:mysql and is accessible. all subfiles are also accessible

i've already set the /etc/security/limits.conf to the following value:

*         soft    nofile          65535
*         hard    nofile          65535

when i try to startup i get the following error:

110406 10:34:45 [Warning] Could not increase number of max_open_files to more than 1024 (request: 2858)
110406 10:34:45 [Warning] Can't create test file /mysql/datadir/dbslave3.lower-test
110406 10:34:45 [Warning] Can't create test file /mysql/datadir/dbslave3.lower-test
/usr/sbin/mysqld: File '/mysql/logs/mysql-slow.log' not found (Errcode: 13)
110406 10:34:45 [ERROR] Could not use /mysql/logs/mysql-slow.log for logging (error 13). Turning logging off for the whole duration of the MySQL server process. To turn it on again: fix the cause, shutdown the MySQL server and restart it.
/usr/sbin/mysqld: File '/mysql/logs/mysql-bin.index' not found (Errcode: 13)
110406 10:34:45 [ERROR] Aborting

110406 10:34:45 [Note] /usr/sbin/mysqld: Shutdown complete

when i enter ulimit -n 65535 and then try to startup the server, i get the following error:

  110406 10:41:43 [Warning] Can't create test file /mysql/datadir/dbslave3.lower-test
  110406 10:41:43 [Warning] Can't create test file /mysql/datadir/dbslave3.lower-test
  /usr/sbin/mysqld: File '/mysql/logs/mysql-slow.log' not found (Errcode: 13)
  110406 10:41:43 [ERROR] Could not use /mysql/logs/mysql-slow.log for logging (error 13). Turning logging off for the whole duration of the MySQL server process. To turn it on again: fix the cause, shutdown the MySQL server and restart it.
  /usr/sbin/mysqld: File '/mysql/logs/mysql-bin.index' not found (Errcode: 13)
  110406 10:41:43 [ERROR] Aborting

  110406 10:41:43 [Note] /usr/sbin/mysqld: Shutdown complete

So what's wrong here? when doing a su - mysql -s /bin/bash, i can create the files and open them.

Is it because i've moved the mysql into KVM?

EDIT: I've also changed the disks from /dev/vda (VirtIO) to /dev/sda (IDE). But still the same behaviour. Maybe it's not KVM, just something wrong in the guest itself.

See you,

share|improve this question
add comment

5 Answers

up vote 0 down vote accepted

first, under ubuntu, there are 3 things that can limit the file access:

  1. filesystem access permissions (but you already said, that this is not the case)
  2. ulimit nofile (but this error message: Can't create test file /mysql/datadir/dbslave3.lower-test appears in the startup, where mysql checks your filesystem for it's case-sensitivity. So mysql hasn't yet created that many handles!)
  3. apparmor! (this should be your problem. especially, since you have moved your datadir!)

To solve it, you have to configure /mysql/** rwk, in /etc/apparmor.d/usr.sbin.mysqld

share|improve this answer
    
thank you so much! it was indeed apparmor! –  b.f. May 3 '11 at 8:28
add comment

Warning, after fedora 19

to increase open_files_limit you must update : /etc/systemd/system/mysql.service

and add in Service section : [Service] LimitNOFILE=65535

Best regards

David

share|improve this answer
1  
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. –  HBruijn Jun 21 at 9:35
add comment

In Linux, the limits are configured by the PAM (pam_limits) at login time and are applied per process. You have 2 limits one is a hard limit and the other is the soft limit. The soft one can be changed by the process and increased until it reaches the hard limit.

You can find the limits for a running process with:

cat /proc/<PID>/limits

To find the limits for current shell you can use:

ulimit -a    #soft limits
ulimit -Ha   #hard limits

You can also strace the process and see why is opening so many file descriptors (which can be sockets, files, pipes...):

strace -f mysql_start_script 2> strace.log
strace -e 'open,socket,pipe,accept' -f mysql_start_script 2> strace.log    # To have only the calls that generates file descriptors

In the strace.log file search for open, socket, pipe.

For a running process you can see the file descriptors with:

ls -l /proc/<PID>/fd

Make sure you have /mysql/logs/ folder. The logs should be somewhere inside /var/log/

Hope it helps.

share|improve this answer
    
And the problem is related to KVM or disk controller used in KVM –  Mircea Vutcovici Apr 6 '11 at 20:31
add comment

My guess is that this is a permissions or ownership issue in one of the MySQL subdirectories, probably in the datafiles directory.

The error message says "Errcode: 13". You can use perror to check the message related to this specific error and confirm that this is really a permission error somewhere.

shell> perror 13
error code  13:  Permission denied

Make sure that the MySQL user can access the datafiles directory and write inside it. This seems to be the problem.

On the other hand, sometimes logfiles are misleading or not as accurate as they should be. In this case, strace comes to the rescue. strace is a very powerful system tracer that monitors al system calls made by a process.

Its usage is easy:

strace -f -o strace.output /etc/init.d/mysql.server start

This will print all the system calls made when starting the MySQL server in a file named strace.output.

Hope this helps.

share|improve this answer
    
+1 for strace and perror –  Mircea Vutcovici Apr 6 '11 at 20:43
    
thank you, here is some output: 4974 open("/mysql/datadir/dbslave3.lower-test", O_RDWR|O_CREAT, 0666) = -1 EACCES (Permission denied) but doing a touch /mysql/datadir/dbslave3.lower-test as mysql user works –  b.f. Apr 7 '11 at 7:21
add comment

Check the read/write permissions on your folder, most of the errors appear to be the result of a write-failure. Ensure that the partition or disk is not in read only mode, and that the mysql folder is writable. Try creating a test file with vi, nano/pico to try and replicate the read/write issues.

share|improve this answer
    
when doing a su - mysql -s /bin/bash, i can create/read/write the files, that mysql says it cannot. file permissions and partition was the frist thing i've checked. i think it must have something to do with either KVM or ulimit –  b.f. Apr 6 '11 at 10:07
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.