Bug 832757 - mysql can't prepare directories when started from systemd
mysql can't prepare directories when started from systemd
Status: CLOSED WONTFIX
Product: Fedora
Classification: Fedora
Component: mysql (Show other bugs)
17
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Tom Lane
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-17 04:00 EDT by Stan King
Modified: 2012-06-21 02:08 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 21:25:55 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
mysqld config file pointing to non-standard data directory (656 bytes, application/octet-stream)
2012-06-17 04:00 EDT, Stan King
no flags Details

  None (edit)
Description Stan King 2012-06-17 04:00:52 EDT
Created attachment 592388 [details]
mysqld config file pointing to non-standard data directory

Description of problem:

I'm attempting to run mysqld with a data directory different than the default of /var/lib/mysql.  I think I've gotten beyond the selinux issues, but I noticed that the systemd unit file contains ExecStartPre=/usr/libexec/mysqld-prepare-db-dir.  This script will be run as user mysql, so its attempts at chown are doomed to failure.  For example, I receive a message of the type "mysqld-prepare-db-dir[3740]: chown: changing ownership of `/home/mysql/mysqldata': Operation not permitted".

I've worked around this by running /usr/libexec/mysqld-prepare-db-dir as root, and mysqld now appears to start, despite some similar permission complaints from mysqld_safe.

I'll attach the /etc/my.cnf file that I'm using.

Version-Release number of selected component (if applicable):

mysql-server-5.5.23-1.fc17.x86_64

How reproducible:

It should happen anytime the below steps are followed.

Steps to Reproduce:
1. Prepare an empty directory for mysqld to use, massage it for selinux happiness, and point to it in /etc/my.cnf
2. Issue the "systemctl start mysqld.service" command.
3. Observe the mysql errors in /var/log/message (followed by a long burst of systemd errors).
  
Actual results:

Errors from /usr/libexec/mysqld-prepare-db-dir concerning failed chown and chgrp commands.


Expected results:

No errors of that type.

Additional info:

Provided upon request.
Comment 1 Tom Lane 2012-06-20 21:25:55 EDT
It's intentional that we don't run mysqld-prepare-db-dir as root, for security.  This makes its chown and restorecon calls more in the nature of documentation of the expected state than useful functionality.  If you are going to ask mysql to run in a non-default data directory, then I think that chown'ing that directory to mysql is part of the required setup that you have to undertake; leaving the directory owned by root, or whatever you did, means that you didn't complete your part of the job.

I realize that you used to be able to get away with that in the initscript world, because that script did run as root.  However, IMO it was a security hole that that script was willing to chown any random directory it might be pointed at.  Consider for example that the root administrator grants write privileges on /etc/my.cnf to a database administrator, so that the DBA can configure mysql appropriately.  Perfectly sane thing to do, no?  The root guy is unlikely to realize that he just gave the DBA the ability to alter anything in the filesystem, because the DBA can now get ownership of any directory he pleases.

So I think it's a good thing that mysqld-prepare-db-dir can't fix this for you.

Now, if you *did* chown the directory and it still failed, then maybe there's an actual bug here.  But that case works for me, modulo getting SELinux to cooperate.
Comment 2 Stan King 2012-06-21 01:07:52 EDT
Tom, I see your point about altering arbitrary directories supplied by a config file that someone else may have ongoing access to.

Although the mysqld-prepare-db-dir doesn't handle the incorrect ownership particularly gracefully, it does provide a failure return code, which systemd recognizes, with "control process exited, code=exited status=1".

However, systemd's response is to spew out messages at about 12 per second, until I interrupt the systemctl command.  Could systemd be directed to stop trying in case of failure?  For it to wait for a retry to succeed seems hopeless.

Here's an excerpt from the error message flow:

Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: chown: changing ownership of `/home/mysql/mysqldata': Operation not permitted
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: chmod: changing permissions of `/home/mysql/mysqldata': Operation not permitted
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: Initializing MySQL database
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: chown: changing ownership of `/home/mysql/mysqldata': Operation not permitted
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: Cannot change ownership of the database directories to the 'mysql'
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: user.  Check that you have the necessary permissions and try again.
Jun 17 00:18:31 hp201004 mysqld-prepare-db-dir[3740]: chown: changing ownership of `/home/mysql/mysqldata': Operation not permitted
Jun 17 00:18:31 hp201004 systemd[1]: mysqld.service: control process exited, code=exited status=1
Jun 17 00:18:31 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:31 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:31 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:31 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:31 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:31 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:31 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:31 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Jun 17 00:18:32 hp201004 systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jun 17 00:18:32 hp201004 systemd[1]: Job pending for unit, delaying automatic restart.
Comment 3 Tom Lane 2012-06-21 02:08:47 EDT
(In reply to comment #2)
> However, systemd's response is to spew out messages at about 12 per second,
> until I interrupt the systemctl command.

Yeah, that aspect of it is an acknowledged systemd bug: it's supposed to have rate-limiting on restart attempts, but the rate-limiting seems to be broken at the moment.  The report that I know about is bug #832039, which unfortunately some overprotective soul seems to have marked Red Hat internal, so you probably can't see it :-(.  There might be others that are open.

Note You need to log in before you can comment on or make changes to this bug.