Bug 1430402

Can you please attach the whole /var/log/audit/audit.log?

Also, are there functional problems as a result (for example, is the system not usable), or is this noise?

I'm also confused as to why Nova is trying to read mysql's configuration files...

Created attachment 1267031 [details]
audit.log

Attaching audit.log. 

I couldn't determine any functional issues caused by these during my tests. 

The mysql conf files seem to be set in nova.conf:

[root@overcloud-controller-0 heat-admin]# grep sql /etc/nova/nova.conf
connection=mysql+pymysql://nova_api:R4TNM9XvM3Ptg7GcK3dnnHR4W.0.15/nova_api?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo
connection=mysql+pymysql://nova:R4TNM9XvM3Ptg7GcK3dnnHR4W.0.15/nova?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo
connection=mysql+pymysql://nova_placement:R4TNM9XvM3Ptg7GcK3dnnHR4W.0.15/nova_placement?read_default_file=/etc/my.cnf.d/tripleo.cnf&read_default_group=tripleo

[root@overcloud-controller-0 heat-admin]# cat /etc/my.cnf.d/tripleo.cnf
[tripleo]
bind-address=10.0.0.23

[heat-admin@overcloud-controller-1 ~]$ cat /etc/my.cnf.d/tripleo.cnf
[tripleo]
bind-address=10.0.0.13

[root@overcloud-controller-2 heat-admin]# cat /etc/my.cnf.d/tripleo.cnf
[tripleo]
bind-address=10.0.0.22

These are the local addresses on the internal_api network.

This appears to be related to https://launchpad.net/tripleo/+bug/1643487

[root@overcloud-controller-2 heat-admin]# ls -lZ /etc/my.cnf.d
-rw-r--r--. root root unconfined_u:object_r:mysqld_etc_t:s0 client.cnf
-rw-r--r--. root root system_u:object_r:mysqld_etc_t:s0 galera.cnf
-rw-r--r--. root root unconfined_u:object_r:mysqld_etc_t:s0 mysql-clients.cnf
-rw-r--r--. root root unconfined_u:object_r:mysqld_etc_t:s0 server.cnf
-rw-r--r--. root root system_u:object_r:mysqld_etc_t:s0 tripleo.cnf

Um...

      def __init__(self, host=None, user=None, password="",
                   database=None, port=0, unix_socket=None,
                   charset='', sql_mode=None,
                   read_default_file=None, conv=None, use_unicode=None,
                   client_flag=0, cursorclass=Cursor, init_command=None,
                   connect_timeout=10, ssl=None, read_default_group=None,
                   compress=None, named_pipe=None, no_delay=None,
                   autocommit=False, db=None, passwd=None, local_infile=False,
                   max_allowed_packet=16*1024*1024, defer_connect=False,
                   auth_plugin_map={}, read_timeout=None, write_timeout=None,
                   bind_address=None):
                   ^^^^^^^^^^^^^^^^^

Damien, why was placing the bind_address in a file and using read_default_file used instead of just using bind_address=...?

The location of the file (/etc/my.cnf.d) makes this pretty ugly to work around in selinux policies since we'd need to allow every OpenStack service to read all MySQL configuration files or change this file to something that can be read by any context.

(That's the __init__ function from connections.py in PyMySQL)

Lon,
 
Up to Ocata we used to set the bind_address argument directly [1], by appending the according option to the DBURI of all OpenStackService, e.g.:

   connection=mysql+pymysql://nova_api:R4TNM9XvM3Ptg7GcK3dnnHR4W.0.15/nova_api?bind_address=1.2.3.4

Unfortunately, this breaks Nova's cells v2 design, which stores this DBURI into the database itself and use it to create connection to cells database. By doing so, only the controller which holds the bind_address is able to connect to cells database (1 controllers among the 3), so this breaks HA and we needed to revert that setting [2].

The only other means we have to pass that setting to PyMySQL is to make it read it from file. We couldn't use the regular /etc/my.cnf.d/* ones due to limitation 
in PyMysql parsing [3], so we needed to come up with this dedicated tripleo.cnf [4]

Knowing that this file does not contain any secret (just client connection configuration), I think it's safe to make it unconfined for SELinux.

[1] https://review.openstack.org/#/c/414629/
[2] https://review.openstack.org/#/c/430183/
[3] https://github.com/PyMySQL/PyMySQL/issues/548
[4] https://review.openstack.org/436192

Also, as noted in comment #6, /etc/my.cnf.d/client.cnf already has unconfined_u property; we just went for a dedicated tripleo.cnf instead of client.cnf due to PyMySQL's parser limitation.

So having the same selinux context for tripleo.cnf and client.cnf seems to be the way to go.

As for the current impact: with those SELinux denials, the bind_address setting is not read by PyMySQL, but a socket is still created for the DB connection. I.e., it shouldn't block ongoing tests, it's only breaking the original VIP failover bug [1]

[1] https://launchpad.net/tripleo/+bug/1643487

The problem is the label and location:

   mysql_etc_t

We can do:

  mysql_read_config(nova_t)
  mysql_read_config(nova_api_t)
  mysql_read_config(neutron_t)
  ...
  mysql_read_config(glance_t)
  ...

Then, they can read:

  /etc/my.cnf
  /etc/my.cnf.d/*

It actually looks okay to do this, it's just messy.  However, it appears other system daemons do this, including Neutron:

  quantum.te:	mysql_read_config(neutron_t)

Presently:

[lhh@localhost selinux-policy-contrib]$ grep mysql_read_config * | grep -v ^mysql
apache.te:	mysql_read_config(httpd_t)
apache.te:	mysql_read_config(httpd_php_t)
apache.te:	mysql_read_config(httpd_suexec_t)
apache.te:	mysql_read_config(httpd_sys_script_t)
cron.te:	mysql_read_config(system_cronjob_t)
dovecot.te:	mysql_read_config(dovecot_auth_t)
dspam.te:	mysql_read_config(dspam_t)
logrotate.te:	mysql_read_config(logrotate_t)
munin.te:	mysql_read_config(munin_t)
munin.te:	mysql_read_config(services_munin_plugin_t)
mythtv.te:	mysql_read_config(mythtv_script_t)
nagios.te:      mysql_read_config(nagios_services_plugin_t)
pdns.te:	mysql_read_config(pdns_t)
quantum.te:	mysql_read_config(neutron_t)
radius.te:	mysql_read_config(radiusd_t)

Note: not posting upstream since the log was captured in enforcing mode instead of permissive. There might be other issues.

Created attachment 1267537 [details]
Patch for observed AVCs.

If this doesn't resolve it, we'll need an upgrade run in permissive to see the new AVCs that pop up. Not all services will require access to mysql_etc_t since many are run via Apache, which already has access to read mysql_etc_t.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245