Bug 1066112 - [RDO] Error: Could not start Service[mysqld] [NEEDINFO]
Summary: [RDO] Error: Could not start Service[mysqld]
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: openstack-puppet-modules
Version: 21
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ivan Chavero
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1083625
TreeView+ depends on / blocked
 
Reported: 2014-02-17 17:52 UTC by Frederic Hornain
Modified: 2015-12-02 16:08 UTC (History)
26 users (show)

Fixed In Version: openstack-packstack-2014.1.1-0.31.1.dev1266.fc21
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-02 03:08:48 UTC
mmagr: needinfo? (fhornain)


Attachments (Terms of Use)
Testcase (15.55 KB, text/plain)
2014-02-18 16:47 UTC, Pablo Hinojosa
no flags Details

Description Frederic Hornain 2014-02-17 17:52:20 UTC
Description of problem:

Well, I install a fresh Fedora 20. 
I followed the following documentation : http://openstack.redhat.com/Quickstart

[root@openstack ~]# packstack --allinone

Applying X.X.X.X_qpid.pp
                                                                                           [ ERROR ]

ERROR : Error appeared during Puppet run: X.X.X.X_mysql.pp
Error: Could not start Service[mysqld]: Execution of '/sbin/service mariadb start' returned 1: 
You will find full trace in log /var/tmp/packstack/20140217-183835-y1W32P/manifests/X.X.X.X_mysql.pp.log
Please check log file /var/tmp/packstack/20140217-183835-y1W32P/openstack-setup.log for more information

Then I look at Mariadb linked issue:

[root@openstack ~]# systemctl status mariadb.service
mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
   Active: failed (Result: exit-code) since Mon 2014-02-17 18:41:24 CET; 8s ago
  Process: 16345 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=1/FAILURE)
  Process: 16344 ExecStart=/usr/bin/mysqld_safe --basedir=/usr (code=exited, status=1/FAILURE)
  Process: 16316 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS)
 Main PID: 16344 (code=exited, status=1/FAILURE)

Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: /usr/bin/mysqld_safe: line 182: /var/log/mysqld.log: Permission denied
Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: touch: cannot touch ‘/var/log/mysqld.log’: Permission denied
Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: chown: cannot access ‘/var/log/mysqld.log’: No such file or directory
Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: chmod: cannot access ‘/var/log/mysqld.log’: No such file or directory
Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: 140217 18:41:23 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended
Feb 17 18:41:23 openstack.myopenstack.com mysqld_safe[16344]: /usr/bin/mysqld_safe: line 138: /var/log/mysqld.log: Permission denied
Feb 17 18:41:23 openstack.myopenstack.com systemd[1]: mariadb.service: main process exited, code=exited, status=1/FAILURE
Feb 17 18:41:24 openstack.myopenstack.com systemd[1]: mariadb.service: control process exited, code=exited status=1
Feb 17 18:41:24 openstack.myopenstack.com systemd[1]: Failed to start MariaDB database server.
Feb 17 18:41:24 openstack.myopenstack.com systemd[1]: Unit mariadb.service entered failed state.


Then I changed the SELinux attribute of the 
[root@openstack log]# ls -alZ /var/log/mysqld.log
-rw-r-----. mysql  mysql           system_u:object_r:mysqld_log_t:s0 mysqld.log

it seems it fixes the problem but you should fix the Openstack automatic installation process. 

BR
/f

Comment 1 Pablo Hinojosa 2014-02-18 16:47:54 UTC
Created attachment 864647 [details]
Testcase

Also the same problem.

Comment 2 Pablo Hinojosa 2014-02-20 01:31:11 UTC
Still the problem with the next context over the file:
[root@openstack log]# ls -alZ /var/log/mysqld.log
-rw-r-----. mysql  mysql           system_u:object_r:mysqld_log_t:s0 mysqld.log

even with SELinux disabled

Comment 3 Pablo Hinojosa 2014-02-20 02:11:31 UTC
Removing my.cnf and restarting the service "fix the problem". 
[root@localhost fedora]# rm /etc/my.cnf
rm: remove regular file ‘/etc/my.cnf’? y
[root@localhost fedora]# systemctl restart mariadb.service -l
[root@localhost fedora]# systemctl status mariadb.service -l
mariadb.service - MariaDB database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
   Active: active (running) since Thu 2014-02-20 01:32:18 GMT; 7s ago
  Process: 13639 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID (code=exited, status=0/SUCCESS)
  Process: 13611 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n (code=exited, status=0/SUCCESS)
 Main PID: 13638 (mysqld_safe)
   CGroup: /system.slice/mariadb.service
           ├─13638 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
           └─13737 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/lib/mysql/localhost.localdomain.err --pid-file=localhost.localdomain.pid

Feb 20 01:32:16 localhost.localdomain systemd[1]: Starting MariaDB database server...
Feb 20 01:32:16 localhost.localdomain mysqld_safe[13638]: 140220 01:32:16 mysqld_safe Logging to '/var/lib/mysql/localhost.localdomain.err'.
Feb 20 01:32:16 localhost.localdomain mysqld_safe[13638]: 140220 01:32:16 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
Feb 20 01:32:18 localhost.localdomain systemd[1]: Started MariaDB database server.

I try to carry on with the packstack installation. 
I get the same error with the log file, even with the file created and the right context. Even with selinux disabled. 
So I try again installing mariadb by myself doing:
Stopping services.
Removing mariadb packages and /etc/my.cfg.
Wiping completely mariadb databases with rm -rf /var/lib/mysql
Installing mariadb packages and new database and password with mysql_secure_installation.
I run just a second packstack --allinone to get the new answers file.
I change the mysql password in the answers file 
Then I get 
"ERROR : Error appeared during Puppet run: 192.168.122.30_mysql.pp
Error: Could not find package mysql"
which I guess is in relation with this bug: https://bugzilla.redhat.com/show_bug.cgi?id=981116

Comment 4 Zane Bitter 2014-02-28 01:58:10 UTC
I see the same problem as, also on a freshly-installed F20 box during packstack installation. I'm guessing this issue should actually be in the mariadb component.

openstack-packstack-2013.2.1-0.29.dev956.fc20
mariadb-5.5.35-3.fc20
selinux-policy-3.12.1-122.fc20

audit2why says:

type=AVC msg=audit(1393551143.063:2533): avc:  denied  { open } for  pid=25412 comm="mysqld_safe" path="/var/log/mysqld.log" dev="dm-0" ino=6029577 scontext=system_u:system_r:mysqld_safe_t:s0 tcontext=unconfined_u:object_r:var_log_t:s0 tclass=file
	Was caused by:
		Missing type enforcement (TE) allow rule.

and audit2allow proposes:

#============= mysqld_safe_t ==============
allow mysqld_safe_t var_log_t:file open;


The workaround is:

$ sudo touch /var/log/mysqld.log
$ sudo chown mysql:mysql /var/log/mysqld.log
$ sudo chcon system_u:object_r:mysqld_log_t:s0 /var/log/mysqld.log

Comment 5 Matthias Runge 2014-03-03 09:29:49 UTC
@Zane, I'd expect this to be reported against selinux-policy.

Comment 6 Miroslav Grepl 2014-03-03 09:53:57 UTC
Please execute

# restorecon -v /var/log/mysqld.log

which is going to fix labeling.

Comment 7 Zane Bitter 2014-03-03 17:08:05 UTC
Whoa, how is this not a bug? You install MariaDB out of the box and it won't start - that's a bug.

I think it's actually a bug in the MariaDB packaging, not in the selinux policy. If installing MariaDB created the log file and set it up with the correct labelling then it would be fine.

"Just manually create some random file and run some obscure command on it" is NOT a resolution. You should be able to install the world's most popular database on Fedora and have it work out of the box.

Comment 8 Honza Horak 2014-03-05 09:04:18 UTC
(In reply to Zane Bitter from comment #4)
> The workaround is:
> 
> $ sudo touch /var/log/mysqld.log
> $ sudo chown mysql:mysql /var/log/mysqld.log
> $ sudo chcon system_u:object_r:mysqld_log_t:s0 /var/log/mysqld.log

We could do this (restorecon instead of chcon) in the %post script, so users don't have to do it on their own. I will fix this together with the next rebase (hopefully this week).

Comment 9 Gilles Dubreuil 2014-03-19 05:13:19 UTC
Any update?

Also, just wondering why /var/log/mariadb/mariadb.log file is not used instead?

Thanks

Comment 10 Ryan O'Hara 2014-03-20 20:23:20 UTC
I'm guessing that the puppet-mysql module is still setting the logfile to /var/log/mysqld.log when it should be /var/log/mariadb/mariadb.log for mariadb.

Comment 11 Miroslav Grepl 2014-03-21 08:01:11 UTC
/var/log/mariadb/mariadb.log would be fine and it would get the right labeling by default because of 


$ matchpathcon /var/log/mariadb
/var/log/mariadb	system_u:object_r:mysqld_log_t:s0

Comment 12 Martin Magr 2014-04-30 15:08:05 UTC
puppetlabs-mysql module from recent openstack-puppet-modules is using /var/log/mariadb/mariadb.log when MariaDB is installed, so this shouldn't be the cause now.

Frederic do you still see the issue? If yes, what version of openstack-packstack and openstack-puppet-modules do you use?

Comment 13 Longli 2014-06-05 16:55:23 UTC
Hi all, i meet this bug too.
OS: opensuse-12.2
mysql: 5.5.33-1.17.1-x86-64

Comment 16 Gabriel Somlo 2015-03-27 19:36:37 UTC
Still seeing this on F21.

Before running 'packstack --allinone' I have to

1. Manually install "mariadb-galera-server", to ensure the presence of user and
   group "mysql:mysql"

2. run "touch /var/log/mysqld.log" and
       "chown mysql:mysql /var/log/mysqld.log"

3. run "packstack --mysql-host=127.0.0.1 --allinone"


Steps #1 and #2 are required in order to avoid the
"failed to start mariadb service" error.

I'm using an up-to-date F21, with

openstack-packstack-2014.1.1-0.28.dev1238.fc21.noarch (and friends), and
mariadb-galera-server-10.0.13-7.fc21.x86_64 (and friends).

I can imagine this being due to openstack's puppet recipe using the wrong
mariadb log file name, but whatever it is, it'd be nice if it got fixed :)

Thanks much in advance!

Comment 17 Gabriel Somlo 2015-03-30 14:11:22 UTC
switching this (back to ?) openstack-packstack, since I tested with
openstack-packstack-2014.1.1-0.31.1.dev1274.fc21.noarch and
openstack-puppet-modules-2014.1.1-3.fc21.noarch
and experienced the same problem with "/var/log/mysqld.log".

Comment 18 Gabriel Somlo 2015-04-02 15:25:52 UTC
Specifically, /usr/share/openstack-puppet/modules/mysql/manifests/params.pp
contains "$log_error = '/var/log/mysqld.log'" in two places (lines 65 and 136).

That *might* have to be "/var/log/mariadb/mariadb.log". Although there
are *a lot* of other file names in that manifest that don't quite match
the current mariadb file naming scheme (starting with /etc/my.cnf, but
maybe that's where all the others get overridden ?) :)

FTR, I noticed this is fixed in the Juno package
(openstack-puppet-modules-2014.2.11-1.fc22.noarch, on
http://repos.fedorapeople.org/repos/openstack/openstack-juno/...).

Any chance that fix can be backported to 2014.1.* ? Alternatively,
can the f21 openstack packages be updated to 2014.2.* (juno) ?

As it is right now, it's relatively hard for a newcomer such as myself
to decide whether to stick with official f21 packages or go "off the
reservation" to get the Juno ones. Not that *either* set allows me
to successfully "packstack --allinone" at the moment... :)

Thanks,
--Gabriel

Comment 19 Fedora End Of Life 2015-11-04 15:57:41 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 20 Fedora End Of Life 2015-12-02 03:09:02 UTC
Fedora 21 changed to end-of-life (EOL) status on 2015-12-01. Fedora 21 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.