6

Last night, my MySQL server went down unexpectedly. On attempting to restart (with service mysql restart - I'm root) it merely hangs. With the mysql -u root -p command, I get ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2). I then tried to start the daemon manually (mysqld). The prompt would hang for about 2 seconds, and then return. On closer inspection of the error logs, I got:

2016-01-22T19:18:32.399584Z 0 [ERROR] Could not create unix socket lock file /var/run/mysqld/mysqld.sock.lock.

2016-01-22T19:18:32.399622Z 0 [ERROR] Unable to setup unix socket lock file.

2016-01-22T19:18:32.399646Z 0 [ERROR] Aborting

I then tried chown mysql /var/run/mysqld, chmod -R 775 /var/run/mysqld, touch /var/run/mysqld/mysqld.sock.lockand apt-get install mysql-community-server --reinstall. No luck.

I have looked around, and couldn't find a solution. Any help here?

Note: I am Running Debian 8 (Jessie) with MySQL community Server 5.7.10

SamboyCoding
  • 171
  • 1
  • 1
  • 5
  • May be you have multiple instanc of mysql What is the output of `ps aux | grep mysql`? – Diamond Jan 23 '16 at 22:00
  • Row 1: `root 4330 0.1 0.1 13336 3036 ? Ss 17:01 0:00 /bin/bash /usr/share/mysql/mysql-systemd-start post` Row 2: `root 6266 0.0 0.1 12728 2292 pts/0 S+ 17:10 0:00 grep mysql` – SamboyCoding Jan 24 '16 at 17:10
  • The first row lists a hanging process i guess. Try to kill it and then Start mysql service. – Diamond Jan 24 '16 at 19:07
  • Ok fine. So with kill -9 proccessid it died, but a quick rerun of PS aux gave me /usr/bin/mysqld_safe restarting, and the process immediately reappeared. Any ideas? – SamboyCoding Jan 24 '16 at 19:45
  • -_-. Ok I'm dumb. The hanging process is the service, which I stopped with `service mysql stop`. It's gone, but manually starting the daemon still doesn't work, and still gives the same error. – SamboyCoding Jan 25 '16 at 13:52
  • Ok so in a fit of desperation, I backed up the database files from `/var/lib/mysql` and then did `apt-get purge mysql*`, then reinstalled community server AND IT STILL DOESN'T WORK! I really have no clue what is going on here! – SamboyCoding Jan 25 '16 at 14:20
  • By any chance, there is enough free space in /var I suppose. And pls remove the mysqld.sock.lock file you created manually before doing service start, it can create problem. – Diamond Jan 25 '16 at 14:28
  • Yes, there's over 50GB free space, and I did, when it didn't run. – SamboyCoding Jan 25 '16 at 15:36

3 Answers3

11

Note to future travelers: It depends on your specific configuration but this is very likely an issue with apparmor. If you don't want to disable locking take a look at syslog and see if you're getting apparmor denies on that file.

You'll see something like: apparmor="DENIED" operation="open" parent=29871 profile="/usr/sbin/mysqld" name="/run/mysqld/mysqld.sock.lock"

And can fix it by adding /run/mysqld/mysqld.sock.lock rw to /etc/apparmor.d/usr.sbin.mysqld near the other /run/* entries and reloading apparmor.

anq
  • 211
  • 1
  • 3
  • 1
    That doesn't really apply to me: I checked and definitely don't have apparmor installed. – SamboyCoding Feb 13 '16 at 05:23
  • @SamboyCoding hmm I'm really curious as to why mysqld couldn't access that file. I now see that you had it installed just fine and it crashed-- what I'm talking about wouldn't just crop up in that case so I'm way off. Anyway I wonder if simply trying to delete the lockfile (and potentially socket if it was lignering) would have done the trick? Crashed processes leaving around lockfiles is an age-old problem. – anq Feb 13 '16 at 20:34
  • I had deleted the entire /run/mysqld folder at one point so that's not it either. – SamboyCoding Feb 13 '16 at 21:19
  • 1
    Note for future readers: To view the system log use `dmesg` – Weston Ganger Jan 23 '17 at 16:12
  • @anq it is exactly my case, but I am not sure about the solution. Should I add the string "/run/mysqld/mysqld.sock.lock rw" to /etc/apparmor.d/usr.sbin.mysqld shoulnd't I? – Carmine May 30 '19 at 23:01
  • Just access that file and it does have the line you just suggested – Carmine May 30 '19 at 23:05
1

Fixed. Add skip-external-locking to my.cnf under the [mysqld] section, and then reboot the entire system. This should fix it, for anyone who finds this. Also if you backed up your data, then attempted to restore and were told to rm ib* to make it work, you need the ibdata file.

SamboyCoding
  • 171
  • 1
  • 1
  • 5
0

I just had the same problem with the same exact error messages, but in my case it was due to an upgrade with the app repository which didn't end up the way it was supposed to. So in my case it was just a matter of manually running mysql_upgrade -u adminname -p and then restarting the server did the trick.

Andyc
  • 97
  • 2
  • Huh... I didn't think of that. I don't THINK there was an update just before this happened, but definitely try this before you do something as drastic as reinstalling. – SamboyCoding Dec 19 '16 at 22:01