This is a repost of a post from an old blog, made on December 28, 2013, that used to be on:
http://adminramble.com/mysql-failing-start-message-cant-find-file-mysqlplugin-frm-errno-23/
Original post:
Recently I had a problem, where MySQL service was failing to start.
When tailing the MySQL log the following would be recorded while service was being started.
131224 06:04:53 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql /usr/libexec/mysqld: Can't find file: './mysql/plugin.frm' (errno: 23) 131224 6:04:53 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it. /usr/libexec/mysqld: Can't create/write to file '/tmp/ibqcFQMW' (Errcode: 23) 131224 6:04:53 InnoDB: Error: unable to create temporary file; errno: 23 131224 6:04:53 [ERROR] Plugin 'InnoDB' init function returned error. 131224 6:04:53 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 131224 6:04:53 [ERROR] Can't create IP socket: Too many open files in system 131224 6:04:53 [ERROR] Aborting 131224 6:04:53 [Note] /usr/libexec/mysqld: Shutdown complete 131224 06:04:53 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Now, the message ‘Can’t find file: ‘./mysql/plugin.frm’ (errno: 23)‘ at the begging of the startup process might make you think that the problems is a missing file, but you can see that at the end of a startup process this message is logged ‘[ERROR] Can’t create IP socket: Too many open files in system‘.
This suggest that there is a problem with the number of files that is open on the system.
You can confirm this by using a perror utility, which prints system error messages.
If we check the error number 23, which is reported in the error message, we see that the cause of the failure is not a missing file, but that the file can’t be open because to many file handles are in use.
# perror 23 OS error code 23: Too many open files in system
You can check the current maximum number of file descriptors by checking the fs.file-max value in /etc/sysctl.conf, or use the sysctl command to check the current value.
# sysctl -a | grep file-max fs.file-max = 65536
To increase the maximum number of file handlesm you can edit /etc/sysctl.conf, change the value of fs.file-max to 200000 or some other value higher then the one you currently have, and then run sysctl -p to apply the new value to the system.
Now, after the file handle number has been increased, you should be able to start the MySQL service normally.