Bug 1118859

What is happening here is that when a node joins the cluster and needs sync, the SST is called. We use wsrep_sst_rsync, although other methods exist. Currently we don't other SST methods.

This wsrep_sst_script does a number of things, like call lsof. Clearly it is being prevented from doing this. I believe this audit.log was partially captured while in permissive mode.

I believe that this BZ is blocking the scenario of Staypuft HA with SELinux enabled.   As a workaround - In QE we're disabling SELinux on the "Discovered-Hosts" by changing the staypuft Provision-templates.

These should take care of it.

domain_read_all_domains_state(mysqld_t)
files_search_pids(mysqld_t)
files_getattr_all_sockets(mysqld_t)

Try with the new policy and if things break, attach your log from a permissive test.

If I run with selinux in permissive mode, I still get this denial:

type=AVC msg=audit(1405371699.582:201): avc:  denied  { getattr } for  pid=14592 comm="lsof" path="/usr/bin/mysqld_safe" dev="dm-0" ino=17369661 scontext=system_u:system_r:mysqld_t:s0 tcontext=system_u:object_r:mysqld_safe_exec_t:s0 tclass=file

I don't think we need to transition it is just a read.

allow mysqld_t mysqld_safe_exec_t:file getattr

Created attachment 918272 [details]
audit.log with 'semanage dontaudit off'

I'm using this policy and galera's node cannot join a cluster when running in enforcing mode, but no AVCs are logged. Setting to permissive works, but still no AVCs logged. I used 'semanage dontaudit off' to finally get some AVCs, seen here.

I could be wrong, but the last AVC in comment #10 seems bad. Since mysqld forks the wsrep_rsync_sst, which runs rsync, it needs access to port 4444. Is there a quick/dirty way to give mysql_t access to that port so I can see if that is indeed the problem?

Ryan you can use this for you test:

allow mysqld_t kerberos_port_t:tcp_socket name_bind;

Ryan O'Hara is this a standard config with mariadb listening on port 4444?

grep 4444 /etc/services 
krb524          4444/tcp        nv-video        # Kerberos 5 to 4 ticket xlator
krb524          4444/udp        nv-video        # Kerberos 5 to 4 ticket xlator


A quick fix would be to execute

semanage port -m -t mysqld_port_t -p tcp 4444

Which will relabel port 4444 as a mysqld_port_t port.

If this is standard config we could either change this port permanantly to myqsqld_port_t since not many people are using KRB4 (Probably for 20 years)

Or we could allow mysqld_d to bind to kerberos_port_t

(In reply to Daniel Walsh from comment #13)
> Ryan O'Hara is this a standard config with mariadb listening on port 4444?
> 
> grep 4444 /etc/services 
> krb524          4444/tcp        nv-video        # Kerberos 5 to 4 ticket
> xlator
> krb524          4444/udp        nv-video        # Kerberos 5 to 4 ticket
> xlator
> 
> 
> A quick fix would be to execute
> 
> semanage port -m -t mysqld_port_t -p tcp 4444
> 
> Which will relabel port 4444 as a mysqld_port_t port.
> 
> If this is standard config we could either change this port permanantly to
> myqsqld_port_t since not many people are using KRB4 (Probably for 20 years)
> 
> Or we could allow mysqld_d to bind to kerberos_port_t

Yes. It isn't actually mysqld that listens on port 4444. When a node tries to join a galera cluster, it may need to be sync'd, and we user rsync to do this. mysqld will exec wsrep_sst_rsync, which then starts an rsync daemon listening on port 4444 to receive the sync data. This port is hard-coded in wsrep_sst_rsync. So it is standard if you're using rsync for the SST method.

If we can change port 4444 to mysqld_port_t, that seems fine. I am assuming that the forked wsrep_sst_rsync process will run in a context that has permission to bind to mysqld_port_t. Can we do this in openstack-selinux such that this is isolated to openstack?

Yes, I'll add to the newest build.  We can add it to Fedora also as long as there isn't issue.

semanage port -m -t mysqld_port_t -p tcp 4444

I tested openstack-selinux-0.5.14-3.el7ost this evening in enforcing mode and galera nodes were able join (performing an SST with rsync). No AVCs. Setting the Fixed In Version and moving to MODIFIED.

Note to QE -- an easy way to force an SST is to bring up a bootstrap node and then remove /var/lib/mysql/grastate.dat on the node(s) that will join the cluster. This is because an SST is not always requireded when a node joins. removing this file forces the SST.