故障描述：今天发现zabbix页面无法打开，提示“Database error”，这种报错一看就是数据库问题，下面介绍排查步骤。
一、排查步骤：
首先查看mysql服务状态，发现服务是停止。
sh@zabbix-11:~$ systemctl status mysql
○ mysql.service - MySQL Community Server
Loaded: loaded (/usr/lib/systemd/system/mysql.service; enabled; preset: enabled)
Active: inactive (dead) since Wed 2026-02-04 06:54:35 CST; 2h 37min ago
Duration: 12h 30min 38.768s
Process: 1092 ExecStart=/usr/sbin/mysqld (code=exited, status=0/SUCCESS)
Main PID: 1092 (code=exited, status=0/SUCCESS)
Status: "Server shutdown complete"
CPU: 23min 38.127s
Feb 03 18:23:44 zabbix-11.20 systemd\[1]: Starting mysql.service - MySQL Community Server...
Feb 03 18:23:52 zabbix-11.20 systemd\[1]: Started mysql.service - MySQL Community Server.
Feb 04 06:54:31 zabbix-11.20 systemd\[1]: Stopping mysql.service - MySQL Community Server...
Feb 04 06:54:35 zabbix-11.20 systemd\[1]: mysql.service: Deactivated successfully.
Feb 04 06:54:35 zabbix-11.20 systemd\[1]: Stopped mysql.service - MySQL Community Server.
Feb 04 06:54:35 zabbix-11.20 systemd\[1]: mysql.service: Consumed 23min 38.127s CPU time, 3.3G memory peak, 0B memory swap peak.
再次查看日志发现MySQL 在 2月4日早上 06:54 被正常停止（Server shutdown complete），并不是因为崩溃或错误，而是被系统或管理员执行了 systemctl stop mysql。所以现在 Zabbix 前端报 “Database error: No such file or directory”，根本原因就是 数据库服务没有运行。
sh@zabbix-11:~$ journalctl -u mysql -xe
详细日志这边省略
但是系统管理员我没有手动终止，下面查看计划任务，发现也没有类似终止的任务存在
sudo crontab -l
cat /etc/crontab
ls -l /etc/cron.d/
ls -l /etc/cron.daily/
ls -l /etc/cron.weekly/
grep -R "systemctl stop mysql" /etc/cron*
检查日志是否有人为操作，如果是管理员手动执行，会显示 systemd 收到 stop 请求。
journalctl -u mysql --since "2026-01-28 06:00" --until "2026-01-28 07:00"
没有计划任务触发 systemctl stop mysql，而两台 Zabbix 虚拟机都出现这个问题，且都跑在同一台宿主机上，并且日志里显示 MySQL 内存峰值很高（2.5G、3.3G），那几乎可以确定是宿主机资源不足导致 MySQL 被 OOM（Out Of Memory）杀掉。
于是我查看内存使用率监控，但是从监控上看在6点54分宿主机和虚拟机内存使用率都没有超出限制。
PVE宿主机上查看有没有OOM日志信息，都没有相关日志。
dmesg | grep -i "kill"
dmesg | grep -i "out of memory"
journalctl -k | grep -i "kill"
journalctl -k | grep -i "out of memory"
journalctl -k | grep -Ei "oom|Killed process|memory"
dmesg | grep -Ei "oom|Killed process|memory"
虚拟机上查看有没有OOM日志信息，都没有相关日志。
dmesg | grep -Ei "oom|Killed process"
journalctl -k | grep -Ei "oom|Killed process"
虚拟机再次查看 MySQL 错误日志
sudo journalctl -u mysql -n 50
sudo cat /var/log/mysql/error.log
2026-02-03T22:54:31.628370Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.44-0ubuntu0.24.04.1).
2026-02-03T22:54:33.685058Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 16 user: 'zabbix'.
2026-02-03T22:54:33.685263Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 10 user: 'zabbix'.
2026-02-03T22:54:33.685382Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 20 user: 'zabbix'.
2026-02-03T22:54:33.685504Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 21 user: 'zabbix'.
2026-02-03T22:54:33.685633Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 13 user: 'zabbix'.
2026-02-03T22:54:33.685744Z 0 [Warning] [MY-010909] [Server] /usr/sbin/mysqld: Forcing close of thread 22 user: 'zabbix'.
2026-02-03T22:54:35.388664Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.44-0ubuntu0.24.04.1) (Ubuntu).
2026-02-04T02:37:35.201953Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.44-0ubuntu0.24.04.1) starting as process 1074
2026-02-04T02:37:35.499237Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2026-02-04T02:37:38.663960Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2026-02-04T02:37:41.314248Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2026-02-04T02:37:41.314285Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2026-02-04T02:37:41.457196Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.44-0ubuntu0.24.04.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu).
2026-02-04T02:37:41.457262Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '127.0.0.1' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2026-02-04T02:38:20.936402Z 0 [System] [MY-013172] [Server] Received SHUTDOWN from user <via user signal>. Shutting down mysqld (Version: 8.0.44-0ubuntu0.24.04.1).
2026-02-04T02:38:22.771668Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.44-0ubuntu0.24.04.1) (Ubuntu).
2026-02-04T02:40:13.723821Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.44-0ubuntu0.24.04.1) starting as process 1025
2026-02-04T02:40:13.828178Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2026-02-04T02:40:16.256501Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2026-02-04T02:40:16.881350Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2026-02-04T02:40:16.881381Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2026-02-04T02:40:16.948638Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '127.0.0.1' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2026-02-04T02:40:16.948681Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.44-0ubuntu0.24.04.1' socket: '/var/run/mysqld/mysqld.sock' port: 3306 (Ubuntu).
error.log 显示 MySQL 是 被外部 stop 信号关闭，而不是 OOM 崩溃，确认是谁在 06:54 发出了 stop 指令
sh@zabbix-11:/var/log/mysql$ journalctl --since "2026-02-04 06:50" --until "2026-02-04 07:00"
Feb 04 06:50:05 zabbix-11.20 systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
Feb 04 06:50:05 zabbix-11.20 systemd[1]: sysstat-collect.service: Deactivated successfully.
Feb 04 06:50:05 zabbix-11.20 systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
Feb 04 06:50:10 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:50:10.906+0800","log.logger":"input.harvester","log.origin":{"function":"github.com/elastic/beats/v7/fileb>
Feb 04 06:50:18 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:50:18.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:50:48 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:50:48.949+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:51:18 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:51:18.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:51:48 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:51:48.949+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:52:18 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:52:18.949+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:52:48 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:52:48.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:53:18 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:53:18.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:53:48 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:53:48.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:54:05 zabbix-11.20 systemd[1]: Starting apt-daily-upgrade.service - Daily apt upgrade and clean activities...
Feb 04 06:54:07 zabbix-11.20 grafana[1215]: logger=cleanup t=2026-02-04T06:54:07.266925779+08:00 level=info msg="Completed cleanup jobs" duration=2.40257ms
Feb 04 06:54:07 zabbix-11.20 grafana[1215]: logger=plugins.update.checker t=2026-02-04T06:54:07.913063827+08:00 level=info msg="Update check succeeded" duration=350.243441ms
Feb 04 06:54:11 zabbix-11.20 apt.systemd.daily[9215]: /usr/bin/unattended-upgrade:567: DeprecationWarning: This process (pid=9215) is multi-threaded, use of fork() may lead to deadlocks in the child.
Feb 04 06:54:11 zabbix-11.20 apt.systemd.daily[9215]: pid = os.fork()
Feb 04 06:54:18 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:18.948+0800","log.logger":"monitoring","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/mo>
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Reexecuting requested from client PID 9439 ('systemctl') (unit apt-daily-upgrade.service)...
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Reexecuting.
Feb 04 06:54:20 zabbix-11.20 systemd[1]: systemd 255.4-1ubuntu8.10 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +F>
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Detected virtualization kvm.
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Detected architecture x86-64.
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Starting fwupd-refresh.service - Refresh fwupd metadata and update motd...
Feb 04 06:54:20 zabbix-11.20 systemd[1]: fwupd-refresh.service: Deactivated successfully.
Feb 04 06:54:20 zabbix-11.20 systemd[1]: Finished fwupd-refresh.service - Refresh fwupd metadata and update motd.
Feb 04 06:54:30 zabbix-11.20 dbus-daemon[866]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.33' (uid=0 pid=10117 comm="/usr/b>
Feb 04 06:54:30 zabbix-11.20 systemd[1]: Starting packagekit.service - PackageKit Daemon...
Feb 04 06:54:30 zabbix-11.20 PackageKit[10121]: daemon start
Feb 04 06:54:30 zabbix-11.20 dbus-daemon[866]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Feb 04 06:54:30 zabbix-11.20 systemd[1]: Started packagekit.service - PackageKit Daemon.
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping apache2.service - The Apache HTTP Server...
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping cron.service - Regular background program processing daemon...
Feb 04 06:54:31 zabbix-11.20 systemd[1]: cron.service: Deactivated successfully.
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopped cron.service - Regular background program processing daemon.
Feb 04 06:54:31 zabbix-11.20 systemd[1]: cron.service: Consumed 1min 22.022s CPU time, 416.6M memory peak, 0B memory swap peak.
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Started cron.service - Regular background program processing daemon.
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping filebeat.service - Filebeat sends log files to Logstash or directly to Elasticsearch....
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.625+0800","log.logger":"service","log.origin":{"function":"github.com/elastic/elastic-agent-libs/ser>
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping fwupd.service - Firmware update daemon...
Feb 04 06:54:31 zabbix-11.20 fwupd[1436]: 22:54:31.626 FuMain Received SIGTERM
Feb 04 06:54:31 zabbix-11.20 multipathd[504]: multipathd: shut down
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping multipathd.service - Device-Mapper Multipath Device Controller...
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.627+0800","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beater.(*Filebeat).Stop","f>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beat>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beat>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beat>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beat>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input.(*Runner).Run","file.>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/input.(*Runner).Run","file.>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"input.harvester","log.origin":{"function":"github.com/elastic/beats/v7/fileb>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"crawler","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/beat>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/re>
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/re>
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping mysql.service - MySQL Community Server...
Feb 04 06:54:31 zabbix-11.20 filebeat[869]: {"log.level":"info","@timestamp":"2026-02-04T06:54:31.628+0800","log.logger":"registrar","log.origin":{"function":"github.com/elastic/beats/v7/filebeat/re>
Feb 04 06:54:31 zabbix-11.20 (cron)[10170]: cron.service: Referenced but unset environment variable evaluates to an empty string: EXTRA_OPTS
Feb 04 06:54:31 zabbix-11.20 systemd[1]: Stopping ModemManager.service - Modem Manager...
Feb 04 06:54:31 zabbix-11.20 ModemManager[981]: <msg> caught signal, shutting down...
这段日志已经把 06:54 停止 MySQL 的原因揭示出来了：
- 在 06:54:05 开始执行
apt-daily-upgrade.service（系统的自动更新任务）。 - 随后 systemd 发生了 Reexecuting（systemd 自身重载），这是 apt/unattended-upgrade 在更新 systemd 包或相关依赖时常见的动作。
- 在 06:54:31，systemd 开始批量停止服务：
apache2、cron、filebeat、fwupd、multipathd、mysql等。
MySQL 在 06:54 被停止，不是 OOM，也不是崩溃，而是系统的自动更新任务触发了服务重启。
二、解决方法：
避免自动更新在生产环境中随意重启服务
可以禁用或调整
apt-daily-upgrade的定时任务：sudo systemctl disable apt-daily-upgrade.timer sudo systemctl disable apt-daily.timer- 或者修改
/etc/apt/apt.conf.d/20auto-upgrades，关闭自动升级。
改为手动或受控更新
- 在维护窗口手动运行
apt upgrade，避免业务高峰时自动停止服务。
- 在维护窗口手动运行