Bug 1538066
Summary: | Fails to start after upgrade to 10.1.30: /run/mariadb missing | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Cesar Eduardo Barros <cesarb> |
Component: | mariadb | Assignee: | Michal Schorm <mschorm> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 26 | CC: | daniell1, dciabrin, dedourek, hhorak, jscalia, jstanek, linux4desktop, mbayer, mkocka, mmuzila, mschorm, praiskup, setthemfree, skpgkp1, tle |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | mariadb-10.1.30-2.fc26 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-01-28 21:15:26 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Cesar Eduardo Barros
2018-01-24 12:18:39 UTC
Manually creating /run/mariadb and changing ownership to mysql.mysql allows mariadb to start normally. However, on the next reboot, the directory again disappears. I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, version 3:10.2.10-2.fc27. In my case, /var/log/mariadb/mariadb.log shows: [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' (Errcode: 2 "No such file or directory") And I had to manually create /run/mysqld and chown to mysql.mysql for mariadb to start up again. (In reply to Tim Elliott from comment #2) > I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, > version 3:10.2.10-2.fc27. > > In my case, /var/log/mariadb/mariadb.log shows: > > [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' > (Errcode: 2 "No such file or directory") > > And I had to manually create /run/mysqld and chown to mysql.mysql for > mariadb to start up again. This might help you: https://bugzilla.redhat.com/show_bug.cgi?id=1483331 (In reply to John DeDourek from comment #1) > Manually creating /run/mariadb and changing ownership to mysql.mysql allows > mariadb to start normally. However, on the next reboot, the directory again > disappears. Thanks. Quick workaround that works for me: echo -e '#!/bin/bash\n\n#see https://bugzilla.redhat.com/show_bug.cgi?id=1538066\nmkdir /run/mariadb && chown mysql.mysql /run/mariadb\n' >> /usr/local/bin/mariadbfix.sh && chown root:root /usr/local/bin/mariadbfix.sh && chmod 744 /usr/local/bin/mariadbfix.sh && echo -e '[Unit]\nDescription=Fix for mariadb, see https://bugzilla.redhat.com/show_bug.cgi?id=1538066\nBefore=mariadb.service\n\n[Service]\nExecStart=/usr/local/bin/mariadbfix.sh\n\n[Install]\nWantedBy=multi-user.target\n' >> /usr/lib/systemd/system/mariadbfix.service && chown root:root /usr/lib/systemd/system/mariadbfix.service && chmod 644 /usr/lib/systemd/system/mariadbfix.service && systemctl enable mariadbfix.service It basically starts a script which creates /run/mariadb before mariadb is started. Remove the workaround with: systemctl disable mariadbfix.service && rm /usr/local/bin/mariadbfix.sh && rm /usr/lib/systemd/system/mariadbfix.service I took a deep look at what might be the cause. I found out, it is most likely caused by incorrect configuration of tmpfiles. (/usr/lib/tmpfiles.d/mariadb.conf) The upstream has recently started to ship their own, however it wasn't compatible with our downstream configuration. I fixed the isue in the SPECfile and prepared the new build. The issue *should be* solved by update to the new version. I didn't need any manual work in my test environment. Please test it and let me know, if the issues persists. mariadb-10.1.30-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-e2bc1c0aab mariadb-10.1.30-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-e2bc1c0aab Hi, i tried testing the fix with dnf config-manager --set-enabled updates-testing dnf update --refresh it offers to Install 1 Package Upgrade 58 Packages But mariadb- is not in the list. Do i need to take another repo for that package or just wait a bit? Dunno, but on my side, both your method and my # dnf update mariadb --enablerepo=updates-testing found the correct version. Maybe, the update did not yet reached your mirror, but I'd call it unlikely. You still can download it manually from the KOJI: https://koji.fedoraproject.org/koji/taskinfo?taskID=24423405 (yeah, a little pain but should always work) seemed to be the mirror: 3hrs later i got the packages and today i got one more mariadb-configuration. But now there is another error: [root@oc ~]# systemctl status mariadb ● mariadb.service - MariaDB 10.1 database server Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled) Active: deactivating (stop-sigterm) (Result: timeout) Process: 794 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mariadb.service (code=exited, status=0/SUCCESS) Process: 718 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS) Main PID: 844 (mysqld) Status: "InnoDB: Error: Could not find a valid tablespace file for 'owncloud/oc_authtoken'. See http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html for how to resolve the issue." Tasks: 16 (limit: 4915) CGroup: /system.slice/mariadb.service └─844 /usr/libexec/mysqld --basedir=/usr Jan 26 09:06:33 oc.server.com systemd[1]: Starting MariaDB 10.1 database server... Jan 26 09:06:33 oc.server.com mysql-check-socket[718]: Socket file /var/lib/mysql/mysql.sock exists. Jan 26 09:06:33 oc.server.com mysql-prepare-db-dir[794]: Database MariaDB is probably initialized in /var/lib/mysql already, nothing is done. Jan 26 09:06:33 oc.server.com mysql-prepare-db-dir[794]: If this is not the case, make sure the /var/lib/mysql is empty before running mysql-prepare-db-dir. Jan 26 09:06:34 oc.server.com mysqld[844]: 2018-01-26 9:06:34 139838212134080 [Note] /usr/libexec/mysqld (mysqld 10.1.30-MariaDB) starting as process 844 ... Jan 26 09:11:34 oc.server.com systemd[1]: mariadb.service: Start operation timed out. Terminating. [root@oc ~]# ps aux | grep -i mysq mysql 844 0.1 5.1 744852 104496 ? Ssl 09:06 0:00 /usr/libexec/mysqld --basedir=/usr root 1203 0.0 0.0 119468 964 pts/0 S+ 09:14 0:00 grep --color=auto -i mysq I don't know if it is still from the broken update, or from my tries to fix it. Now it's missing a tablespace, although the file is lying in /var/lib/mysql/owncloud/oc_authtoken. This looks like restoring the backup :-( Fixed the issue. owncloud databse folder had root:root. Chowning that to mysql:mysql fixed it. Probably this was me trying to fix it. 8 drwx------. 2 root root 4096 Jan 24 17:05 owncloud chown -R mysql:mysql /var/lib/mysql/owncloud/ Now the service is starting fine also after reboot. Thanks! Glad to hear it fully works again :) mariadb-10.1.30-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. I had similar issue with mariadb startup on Fedora 27. Manually creating directory /run/mariadb and setting it's ownership to mysql:mysql works but then again reboot wipe out /run directory and mariadb startup fail again. Debug messages are not very helpful. My mariadb version is mariadb-10.2.10-2.fc27.x86_64. Here is my custom mariadb startup file which survive reboot. $cat /etc/systemd/system/mariadb.service #Custom service to fix mariadb startup issue with system reboot. .include /usr/lib/systemd/system/mariadb.service [Unit] Description=MariaDB 10.2 database server After=network.target [Service] Type=Idle User=mysql Group=mysql PermissionsStartOnly=true ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown mysql:mysql /run/mariadb" $systemctl daemon-reload (In reply to Daniel Laczi from comment #3) > (In reply to Tim Elliott from comment #2) > > I'm seeing a similar problem in F27 after upgrading to mariadb.x86_64, > > version 3:10.2.10-2.fc27. > > > > In my case, /var/log/mariadb/mariadb.log shows: > > > > [ERROR] mysqld: Can't create/write to file '/run/mysqld/mariadb.pid' > > (Errcode: 2 "No such file or directory") > > > > And I had to manually create /run/mysqld and chown to mysql.mysql for > > mariadb to start up again. > > This might help you: https://bugzilla.redhat.com/show_bug.cgi?id=1483331 Just wanted to give you a belated thanks. I do have a change in /etc/mariadb-server.cnf (to change the datadir setting out of the root filesystem) and I had to merge my config with the .rpmnew version. Everything works now. (In reply to Sunil P from comment #14) > I had similar issue with mariadb startup on Fedora 27. Manually creating > directory /run/mariadb and setting it's ownership to mysql:mysql works but > then again reboot wipe out /run directory and mariadb startup fail again. > Debug messages are not very helpful. > > My mariadb version is mariadb-10.2.10-2.fc27.x86_64. > > Here is my custom mariadb startup file which survive reboot. > > $cat /etc/systemd/system/mariadb.service > #Custom service to fix mariadb startup issue with system reboot. > .include /usr/lib/systemd/system/mariadb.service > [Unit] > Description=MariaDB 10.2 database server > After=network.target > > [Service] > Type=Idle > User=mysql > Group=mysql > PermissionsStartOnly=true > ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown > mysql:mysql /run/mariadb" > > $systemctl daemon-reload It was working but fedora update broke this setup as it doesn't understand Type parameter and doesn't start mariadb. /etc/systemd/system/mariadb.service:8: Failed to parse service type, ignoring: Idle Here is corrected one which work with latest Fedora and survive reboot as of now. $cat /etc/systemd/system/mariadb.service #Custom service to fix mariadb startup issue with system reboot. .include /usr/lib/systemd/system/mariadb.service [Unit] Description=MariaDB 10.2 database server After=network.target [Service] User=mysql Group=mysql PermissionsStartOnly=true ExecStartPre=/bin/bash -c "/bin/mkdir -p /run/mariadb && /bin/chown mysql:mysql /run/mariadb" $systemctl daemon-reload |