Bug 153927 - mysqld fails to start, due to missing rights to access /var/lib/mysql
mysqld fails to start, due to missing rights to access /var/lib/mysql
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: mysql (Show other bugs)
3
All Linux
medium Severity high
: ---
: ---
Assigned To: Tom Lane
David Lawrence
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-04-05 17:24 EDT by Boris Glawe
Modified: 2013-07-02 23:05 EDT (History)
3 users (show)

See Also:
Fixed In Version: 1.18.1-2.12
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-09-05 03:24:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Boris Glawe 2005-04-05 17:24:59 EDT
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.6) Gecko/20050323 Firefox/1.0.2 Fedora/1.0.2-1.3.1

Description of problem:
The mysqdl won't start up.

The start skript exits with the message:
# /etc/init.d/mysqld start
Timeout error occurred trying to start MySQL Daemon.
MySQL starten:                                             [FEHLGESCHLAGEN]

------------------------------------------------------------------------------------------------------

If I startup mysql for the first time I get the following output:
# /etc/init.d/mysqld start
MySQL Datenbank initialisieren:  Preparing db table
Preparing host table
Preparing user table
Preparing func table
Preparing tables_priv table
Preparing columns_priv table
Installing all prepared tables
Installation of grant tables failed!

Examine the logs in /var/lib/mysql for more information.
You can also try to start the mysqld daemon with:
/usr/libexec/mysqld --skip-grant &
You can use the command line tool
/usr/bin/mysql to connect to the mysql
database and look at the grant tables:

shell> /usr/bin/mysql -u root mysql
mysql> show tables

Try 'mysqld --help' if you have problems with paths. Using --log
gives you a log in /var/lib/mysql that may be helpful.

/usr/libexec/mysqld: Can't change dir to '/var/lib/mysql/' (Errcode: 13)
050405 23:12:22  Aborting

050405 23:12:22  /usr/libexec/mysqld: Shutdown Complete

The latest information about MySQL is available on the web at
http://www.mysql.com
Please consult the MySQL manual section: 'Problems running mysql_install_db',
and the manual section that describes problems on your OS.
Another information source is the MySQL email archive.
Please check all of the above before mailing us!
And if you do mail us, you MUST use the /usr/bin/mysqlbug script!
                                                           [FEHLGESCHLAGEN]
[root@muli boris]# /etc/init.d/mysqld start
Timeout error occurred trying to start MySQL Daemon.
MySQL starten:                                             [FEHLGESCHLAGEN]

------------------------------------------------------------------------------------------------------

/var/log/messages contains the following

Apr  5 23:12:22 muli mysqld: MySQL Datenbank initialisieren:  failed
Apr  5 23:12:24 muli kernel: audit(1112735544.782:0): avc:  denied  { search } for  pid=26750 exe=/usr/libexec/mysqld name=lib dev=hda6 ino=240482 scontext=root:system_r:mysqld_t tcontext=system_u:object_r:home_root_t tclass=dir
Apr  5 23:12:34 muli mysqld: MySQL starten:  failed

------------------------------------------------------------------------------------------------------

# ls -lZ /var/lib/ | grep mysql
drwxr-xr-x  mysql    mysql    system_u:object_r:mysqld_db_t    mysql


------------------------------------------------------------------------------------------------------

/var/log/mysqld.log contains this:


050405 23:12:24  mysqld started
^G/usr/libexec/mysqld: Can't change dir to '/var/lib/mysql/' (Errcode: 13)
050405 23:12:24  Aborting

050405 23:12:24  /usr/libexec/mysqld: Shutdown Complete

050405 23:12:24  mysqld ended

------------------------------------------------------------------------------------------------------

According to the ownership/access rights of the directory /var/lib/mysql the bug must be somewhere in the selinux settings.


Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. Simply try to start mysqld
2.
3.
  

Actual Results:  mysqld is refused to access it's directory /var/lib/mysql

Additional info:

I am having this problem since a long time. I am wondering why I am the only one who cannot start mysql!?
Comment 1 Tom Lane 2005-04-05 22:52:57 EDT
It sounds like a selinux issue, indeed ... but you have not provided either mysql or selinux package 
version numbers so there's not much hope of giving specific information.

I'd suggest updating both of those packages to latest available versions and doing "restorecon -R /var/
lib/mysql" to be sure that everything in that directory is properly labeled.
Comment 2 Boris Glawe 2005-04-06 04:10:39 EDT
I tried the restorecon command - but it didn't help!

Both server and client are in version mysql-3.23.58-16.FC3.1
The selinux-policy is in version selinux-policy-targeted-1.17.30-2.93

=> the system is up2date.

I am updating my system as soon as the updates are released. I've had this
problem for a long time. It's not a problem that came with the last update.
Comment 3 Tom Lane 2005-04-06 09:49:38 EDT
Actually, looking closer at the /var/log/messages entry, it seems to be
complaining about a directory named "lib", which suggests that the bogus selinux
setting may actually be on /var/lib.  Or maybe somewhere else ... the inode
number is the only definitive indication there.  Anyway, find the directory with
inode 240482 and try a restorecon on that.
Comment 4 Boris Glawe 2005-04-07 04:49:57 EDT
I did a

restorecon -R /var

Unfortunately with no success. Where else can be something called "lib" which
mysqld wants to access? According to /var/log/mysqld.log access to
/var/lib/mysql was refused, which caused it to terminate.

In the hope that this is the instance which controls the access, I post you both my 
/etc/selinux/targeted/src/policy/file_contexts/program/mysqld.fc
and my
/etc/selinux/targeted/src/policy/domains/program/mysqld.te


##################################################

# mysql database server
/usr/sbin/mysqld(-max)? --      system_u:object_r:mysqld_exec_t
/usr/libexec/mysqld     --      system_u:object_r:mysqld_exec_t
/var/run/mysqld(/.*)?           system_u:object_r:mysqld_var_run_t
/var/log/mysql.*        --      system_u:object_r:mysqld_log_t
/var/lib/mysql(/.*)?            system_u:object_r:mysqld_db_t
/var/lib/mysql/mysql\.sock -s   system_u:object_r:mysqld_var_run_t
/etc/my\.cnf            --      system_u:object_r:mysqld_etc_t
/etc/mysql(/.*)?                system_u:object_r:mysqld_etc_t
ifdef(`distro_debian', `
/etc/mysql/debian-start --      system_u:object_r:bin_t
')


##################################################
And here is my  /etc/selinux/targeted/src/policy/domains/program/mysqld.te
##################################################



#DESC Mysqld - Database server
#
# Author:  Russell Coker <russell@coker.com.au>
# X-Debian-Packages: mysql-server
#

#################################
#
# Rules for the mysqld_t domain.
#
# mysqld_exec_t is the type of the mysqld executable.
#
daemon_domain(mysqld)

type mysqld_port_t, port_type;
allow mysqld_t mysqld_port_t:tcp_socket name_bind;

allow mysqld_t mysqld_var_run_t:sock_file create_file_perms;

etcdir_domain(mysqld)
typealias mysqld_etc_t alias etc_mysqld_t;
type mysqld_db_t, file_type, sysadmfile;

log_domain(mysqld)

# for temporary tables
tmp_domain(mysqld)

allow mysqld_t usr_t:file { getattr read };

allow mysqld_t self:fifo_file { read write };
allow mysqld_t self:unix_stream_socket create_stream_socket_perms;
allow initrc_t mysqld_t:unix_stream_socket connectto;
allow initrc_t mysqld_var_run_t:sock_file write;

allow initrc_t mysqld_log_t:file { write append setattr ioctl };

allow mysqld_t self:capability { dac_override setgid setuid net_bind_service };
allow mysqld_t self:process getsched;

allow mysqld_t proc_t:file { getattr read };

# Allow access to the mysqld databases
create_dir_file(mysqld_t, mysqld_db_t)
allow mysqld_t var_lib_t:dir { getattr search };

can_network(mysqld_t)
can_ypbind(mysqld_t)

# read config files
r_dir_file(initrc_t, mysqld_etc_t)
allow mysqld_t { etc_t etc_runtime_t }:{ file lnk_file } { read getattr };

allow mysqld_t etc_t:dir search;

read_sysctl(mysqld_t)

can_unix_connect(sysadm_t, mysqld_t)

# for /root/.my.cnf - should not be needed
allow mysqld_t sysadm_home_dir_t:dir search;
allow mysqld_t sysadm_home_t:file { read getattr };

ifdef(`logrotate.te', `
r_dir_file(logrotate_t, mysqld_etc_t)
allow logrotate_t mysqld_db_t:dir search;
allow logrotate_t mysqld_var_run_t:dir search;
allow logrotate_t mysqld_var_run_t:sock_file write;
can_unix_connect(logrotate_t, mysqld_t)
')

ifdef(`user_db_connect', `
allow userdomain mysqld_var_run_t:dir search;
allow userdomain mysqld_var_run_t:sock_file write;
')

ifdef(`daemontools.te', `
domain_auto_trans( svc_run_t, mysqld_exec_t, mysqld_t)
allow svc_start_t mysqld_t:process signal;
svc_ipc_domain(mysqld_t)
')dnl end ifdef daemontools

ifdef(`distro_redhat', `
allow initrc_t mysqld_db_t:dir create_dir_perms;

# because Fedora has the sock_file in the database directory
file_type_auto_trans(mysqld_t, mysqld_db_t, mysqld_var_run_t, sock_file)
')
Comment 5 Tom Lane 2005-04-07 10:04:15 EDT
Dan, do you have any idea what's going on here?  What could selinux be
complaining about, if not /var/lib?
Comment 6 Daniel Walsh 2005-04-07 10:22:39 EDT
Looks like the mysql is trying to look in the directory you are currently
running in.
allow mysqld_t home_root_t:dir search;

So you attempted to run this app from the /root directory and then it tried to
chroot but failed.

If you add the above rule does it work?
Comment 7 Boris Glawe 2005-04-07 12:46:35 EDT
"So you attempted to run this app from the /root directory and then it tried to
chroot but failed."

I've run the start skript in /etc/init.d

"If you add the above rule does it work?"

Which rule do I have to add and how do I add rules?
Comment 8 Boris Glawe 2005-04-07 15:54:34 EDT
I have added your rule to /etc/selinux/targeted/src/policy/domains/program/mysqld.te

I rebooted the machine, unfortunately with no success, sorry!

What else do you need to know, to solve this problem?



Btw. I have worked on another FC3 machine today. I installed mysql and
mysql-server (with some perl packages, which these packages depend on) and it
worked !! Is it possible, that I have misconfigured something?

greets Boris
Comment 9 Daniel Walsh 2005-04-07 16:00:14 EDT
Well the AVC message you gave above says that mysql went searching for something
in the root directory.  So it is likely that something is screwed up.
Is mysqld attempting to run as root?

Comment 10 Boris Glawe 2005-04-07 16:26:03 EDT
Here is my /etc/my.cf

According to this file, it tries to run as user mysql. How can I check, which
user it actually uses?

# cat /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

[mysql.server]
user=mysql
basedir=/var/lib

[safe_mysqld]
err-log=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
Comment 11 Boris Glawe 2005-04-11 12:07:30 EDT
Hi,

I have spent some time learning a bit about selinux.

The proble here seems to be the directory /var/lib:

$ ls -lZ /var | grep lib
drwxr-xr-x  root     root     system_u:object_r:home_root_t    lib

This directory also has the inode numer 240482, which is mentioned in the "avc
denied" output:
$ ls -li /var | grep lib
240482 drwxr-xr-x  29 root    root    4096  5. Apr 23:11 lib

I have found the following line in
/etc/selinux/targeted/src/policy/file_contexts/file_contexts:
/var/lib            -d      system_u:object_r:home_root_t

The strange thing is the home_root_t domain, this directory belongs to!? Is this
normal? I have never changed anything in this file.

There are many other processes, which cannot access this directory,too: dhcpd
and ntpd

What would you suggest to do?

greets Boris
Comment 12 Boris Glawe 2005-04-11 12:09:55 EDT
Btw.: The fact that mysql worked on another fc3 machine, as I've reported above,
is only the have truth. The machine I was talking about has selinux disabled.
Comment 13 Daniel Walsh 2005-04-11 12:28:24 EDT
Ok the problem is that you have a user account that is in the /var directory. 
This is screwing up the labeling.   /var should be var_t.  In FC3 we have a
problem handling useraccounts with UID > 500 in the /var directory.

Could you check if this is the case.

Dan
Comment 14 Boris Glawe 2005-04-11 12:40:05 EDT
I have I user "ogo", for opengroupware.org with the uid 500 and the home
directory /var/lib/opengroupware.org

This might be the reason!!

Is it enough to set the line in
/etc/selinux/targeted/src/policy/file_contexts/file_contexts to the correct
value, or do I have to "compile" the rules first?

thanks Boris
Comment 15 Boris Glawe 2005-04-11 12:43:28 EDT
There's another strange thing:

I understand, that the labeling is wrong, but how can the settings in
/etc/selinux/targeted/src/policy/file_contexts/file_contexts be screwed up? Are
these files edited automatically by any instance?
Comment 16 Daniel Walsh 2005-04-11 13:51:20 EDT
Yes, genhomedircon generates these entries.  Do you need to login or does the
app need a login to ogo.  IE Can you change the shell to /sbin/nologin
Comment 17 Boris Glawe 2005-04-21 14:27:17 EDT
I set ogo's home to /home/ogo (and created this dir before of course) and ran 

restorecon -R /

It worked then for me. It's obviously not a mysql bug, but it's an ugly selinux bug.
What's the procedure now in this case!

greets Boris
Comment 18 Daniel Walsh 2005-04-21 14:56:17 EDT
policycoreutils-1.18.1-2.12 Just went out to fedora-testing should fix this problem.

Available on ftp://people.redhat.com:dwalsh/SELinux/FC3

Note You need to log in before you can comment on or make changes to this bug.