1

I have a basic DigitalOcean droplet (1gb ram and 1 cpu) that has a few websites on it. There are a couple nodejs sites and a couple of Wordpress sites. All are pretty low traffic. It normally works just fine but today none of the wordpress sites were working. So I logged in and noticed that the cpu usage was pegged at 100% and it normally is at like 2-3%. I restarted the server and the cpu usage went down. But the sites were still not working. So when I tried to access the sites it errors and says there is not database connection. So I checked the status of mysql and it was not started. I tried to restart the service but that just fails. Here is the result of sudo systemctl status mysql

mysql.service - MySQL Community Server
     Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
     Active: activating (start) since Tue 2023-05-09 19:44:41 CDT; 48s ago
    Process: 3859 ExecStartPre=/usr/share/mysql/mysql-systemd-start pre (code=exited, status=0/SUCCESS)
   Main PID: 3867 (mysqld)
     Status: "Server upgrade in progress"
      Tasks: 28 (limit: 1115)
     Memory: 491.4M
        CPU: 5.561s
     CGroup: /system.slice/mysql.service
             └─3867 /usr/sbin/mysqld

May 09 19:44:41 sk-webservice.com systemd[1]: mysql.service: Scheduled restart job, restart counter is at 7.
May 09 19:44:41 sk-webservice.com systemd[1]: Stopped MySQL Community Server.
May 09 19:44:41 sk-webservice.com systemd[1]: mysql.service: Consumed 5.529s CPU time.
May 09 19:44:41 sk-webservice.com systemd[1]: Starting MySQL Community Server...

And then I get the following in /var/log/mysql/error.log

2023-05-10T00:41:48.072177Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.33-0ubuntu0.22.04.1) starting as process 3037
2023-05-10T00:41:48.081239Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-05-10T00:41:48.660106Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-05-10T00:41:49.429920Z 4 [System] [MY-013381] [Server] Server upgrade from '80032' to '80033' started.
2023-05-10T00:42:44.456107Z 4 [ERROR] [MY-013178] [Server] Execution of server-side SQL statement '-- Deprecate spatial reference systems -- Deprecation must happen before new SRSs are added/updated since there -- typically are new SRSs with the same names. REPLACE INTO mysql.st_spatial_reference_systems(id, catalog_id, name, organization, organization_coordsys_id, definition, description) VALUES (6956, 1, 'VN-2000 / TM-3 zone 481 (6956 deprecated)', 'EPSG', 6956, 'PROJCS["VN-2000 / TM-3 zone 481",GEOGCS["VN-2000",DATUM["Vietnam 2000",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],TOWGS84[-191.90441429,-39.30318279,-111.45032835,0.00928836,-0.01975479,0.00427372,0.252906278],AUTHORITY["EPSG","6756"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.017453292519943278,AUTHORITY["EPSG","9122"]],AXIS["Lat",NORTH],AXIS["Lon",EAST],AUTHORITY["EPSG","4756"]],PROJECTION["Transverse Mercator",AUTHORITY["EPSG","9807"]],PARAMETER["Latitude of natural origin",0,AUTHORITY["EPSG","8801"]],PARAMETER["Longitude of natural origin",102,AUTHORITY["EPSG","8802"]],PARAMETER["Scale factor at natural origin",0.9999,AUTHORITY["EPSG","8805"]],PARAMETER["False easting",0,AUTHORITY["EPSG","8806"]],PARAMETER["False northing",500000,AUTHORITY["EPSG","8807"]],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["E",EAST],AXIS["N",NORTH],AUTHORITY["EPSG","6956"]]', 'Deprecated since EPSG Dataset 9.3. Superseded by SRID 5896.'); ' failed with error code = 1205, error message = 'Lock wait timeout exceeded; try restarting transaction'.
2023-05-10T00:42:44.463355Z 0 [ERROR] [MY-013380] [Server] Failed to upgrade server.
2023-05-10T00:42:44.463830Z 0 [ERROR] [MY-010119] [Server] Aborting
2023-05-10T00:42:45.325713Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.33-0ubuntu0.22.04.1)  (Ubuntu).

So it seems like mysql was trying to upgrade and just failed? I don't remember running any upgrades recently. This is just a personal server and I am by no means a wiz on the server side of things. Does anyone know how I can fix this?

4
  • The error at the end of your long string is Lock wait timeout exceeded; try restarting transaction. That's unusual on its own upgrade process. I can't readily find anything about this elsewhere. You may need to restore a backup...
    – Halfgaar
    Commented May 10, 2023 at 1:14
  • 1
    Please add the output of grep -i mysql var/log/apt/history.log
    – shodanshok
    Commented May 10, 2023 at 5:38
  • @shodanshok I ended up just uninstalling and reinstalling mysql. I have no Idea what happened. I am about 100mb away from my ram limit most of the time so I wonder if maybe I ran out of ram and that caused some kind of issue and then when I rebooted something got corrupted. I have no idea.
    – Steve K
    Commented May 10, 2023 at 16:48
  • @SteveK Digital Ocean may have attempted to upgrade your instance - check with DO support team. Commented May 10, 2023 at 20:23

2 Answers 2

0

I got this same issue today and this is How I solved this.

  1. First of all I stopped mysql: service mysql stop

  2. After stopping mysql I ran MYSQLD with upgrade: /usr/sbin/mysqld --upgrade=minimal Make sure upgrade argument is minimal.

  3. Now if you get error like unable to create sock file then you can create folder like /var/run/mysqld

  4. Do check error logs in a seperate terminal while doing this: tail -f /var/log/mysql/error.log

  5. Now you will see mysqld running. Finally you can DUMP all the database by this cmd:

for DB in $(mysql -u root -pPassword -e 'show databases' -s --skip-column-names); do mysqldump -u root -pPassword $DB > "$DB.sql"; done

  1. Now you can uninstall mysql and install a stable version and Import all the databases.
0

First of all I stopped mysql: service mysql stop

After stopping mysql I ran MYSQLD with upgrade: /usr/sbin/mysqld --upgrade=minimal Make sure upgrade argument is minimal.

Now if you get error like unable to create sock file then you can create folder like /var/run/mysqld

I followed the above steps and started mysql service and it is working for me

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .