Description of problem: With a clean, stock install of Fedora 14 + @clustering +lvm2-cluster, I am unable to start the clvmd service. "#service clvmd start" Will result in I have 2 clustered VG's with 7 clustered lv's configured. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: # getenforce Permissive # setenforce Enforcing # getenforce Enforcing # service clvmd start Actual results: Starting clvmd: clvmd could not connect to cluster manager Consult syslog for more information # clustat Could not connect to CMAN: Connection refused # ls -l /var/lib/corosync/ total 15796 -rw-------. 1 root root 49246208 Jan 30 21:38 core.3222 -rw-------. 1 root root 65900544 Jan 31 15:49 core.4238 -rwx------. 1 root root 4014092 Jan 31 15:49 fdata -rwx------. 1 root root 8 Jan 30 22:21 ringid_10.254.254.210 # date Mon Jan 31 15:49:44 EST 2011 from /var/log/audit/audit.log: type=MAC_STATUS msg=audit(1296506946.362:58530): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 type=SYSCALL msg=audit(1296506946.362:58530): arch=c000003e syscall=1 success=yes exit=1 a0=3 a1=7fff0fd1f280 a2=1 a3=1 items=0 ppid=22185 pid=27714 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts6 ses=4294967295 comm="setenforce" exe="/usr/sbin/setenforce" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) type=AVC msg=audit(1296506963.640:58531): avc: denied { read write } for pid=4238 comm="corosync" name="control_buffer-mmT3wE" dev=tmpfs ino=1946789 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58531): arch=c000003e syscall=2 success=no exit=-13 a0=227fb10 a1=2 a2=180 a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.640:58532): avc: denied { unlink } for pid=4238 comm="corosync" name="control_buffer-mmT3wE" dev=tmpfs ino=1946789 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58532): arch=c000003e syscall=87 success=no exit=-13 a0=227fb10 a1=2 a2=d a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.640:58533): avc: denied { read write } for pid=4238 comm="corosync" name="request_buffer-c0Ed7p" dev=tmpfs ino=1946790 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58533): arch=c000003e syscall=2 success=no exit=-13 a0=2280b10 a1=2 a2=180 a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.640:58534): avc: denied { unlink } for pid=4238 comm="corosync" name="request_buffer-c0Ed7p" dev=tmpfs ino=1946790 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58534): arch=c000003e syscall=87 success=no exit=-13 a0=2280b10 a1=2 a2=d a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.640:58535): avc: denied { read write } for pid=4238 comm="corosync" name="response_buffer-oJHwHb" dev=tmpfs ino=1946791 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58535): arch=c000003e syscall=2 success=no exit=-13 a0=2281b10 a1=2 a2=180 a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.640:58536): avc: denied { unlink } for pid=4238 comm="corosync" name="response_buffer-oJHwHb" dev=tmpfs ino=1946791 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.640:58536): arch=c000003e syscall=87 success=no exit=-13 a0=2281b10 a1=2 a2=d a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.641:58537): avc: denied { read write } for pid=4238 comm="corosync" name="dispatch_buffer-hmWYhX" dev=tmpfs ino=1946792 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.641:58537): arch=c000003e syscall=2 success=no exit=-13 a0=2282b10 a1=2 a2=180 a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=AVC msg=audit(1296506963.641:58538): avc: denied { unlink } for pid=4238 comm="corosync" name="dispatch_buffer-hmWYhX" dev=tmpfs ino=1946792 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:clmvd_tmpfs_t:s0 tclass=file type=SYSCALL msg=audit(1296506963.641:58538): arch=c000003e syscall=87 success=no exit=-13 a0=2282b10 a1=2 a2=d a3=100 items=0 ppid=1 pid=4238 auid=500 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="corosync" exe="/usr/sbin/corosync" subj=unconfined_u:system_r:corosync_t:s0 key=(null) type=ANOM_ABEND msg=audit(1296506963.668:58539): auid=500 uid=0 gid=0 ses=1 subj=unconfined_u:system_r:corosync_t:s0 pid=27741 comm="corosync" sig=11 Expected results: # service clvmd start Activating VG(s): 5 logical volume(s) in volume group "clvm-VG00" now active 2 logical volume(s) in volume group "sharedVG01" now active [ OK ] Additional info: Also not sure why qdiskd needs to what appears to be attempting to access pulseaudio /dev/shm content as well as /dev/hugepages during startup of the cman service, which also causes a different set of avc's: type=AVC msg=audit(1296507103.511:8): avc: denied { getattr } for pid=1992 comm="qdiskd" path="/dev/hugepages" dev=devtmpfs ino=9210 scontext=system_u:system_r:qdiskd_t:s0 tcontext=system_u:object_r:hugetlbfs_t:s0 tclass=dir type=SYSCALL msg=audit(1296507103.511:8): arch=c000003e syscall=6 success=yes exit=0 a0=7fff4b634160 a1=7fff4b6340c0 a2=7fff4b6340c0 a3=1008 items=0 ppid=1 pid=1992 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qdiskd" exe="/usr/sbin/qdiskd" subj=system_u:system_r:qdiskd_t:s0 key=(null) type=AVC msg=audit(1296507103.511:9): avc: denied { read } for pid=1992 comm="qdiskd" name="hugepages" dev=devtmpfs ino=9210 scontext=system_u:system_r:qdiskd_t:s0 tcontext=system_u:object_r:hugetlbfs_t:s0 tclass=dir type=AVC msg=audit(1296507103.511:9): avc: denied { open } for pid=1992 comm="qdiskd" name="hugepages" dev=devtmpfs ino=9210 scontext=system_u:system_r:qdiskd_t:s0 tcontext=system_u:object_r:hugetlbfs_t:s0 tclass=dir type=SYSCALL msg=audit(1296507103.511:9): arch=c000003e syscall=2 success=yes exit=8 a0=7fff4b634160 a1=90800 a2=0 a3=1008 items=0 ppid=1 pid=1992 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="qdiskd" exe="/usr/sbin/qdiskd" subj=system_u:system_r:qdiskd_t:s0 key=(null)
If corosync is coredumping, and all the selinux messages are about corosync, then it seems like this is a corosync problem.
I reported it against the component that, when started, lead to the cluster going offline. If I do not start clvmd, the cluster remains running indefinitely. To me, and I'm at most an SELinux novice, it seems like clvmd uses the clvmd_tempfs_t file context when attempting to communicate with corosync. That isn't what corosync is labeled as, so SELinux denies the read & write operation & prevents the service from connecting. Since rgmanager & gfs2 interfaces w/corosync it seems to me like those services are using the right security context whereas clvmd does not and I thought it logical to indict clvmd as it's actually - in a cluster context - a client (like) service to corosync and should/must adhere to the context corosync expects. There's a secondary (corosync) problem, I agree, in that it core dumps which seems like a bad choice given the situation.
The coredump of corosync when applications with incorrect security context is a known problem addressed in Bug #619918. Since that particular issue is already planned for resolution in an upcoming update of both fedora and rhel, I would close this as a duplicate except for the fact that there is any selinux issue at all. Corosync AVC denials were removed as a result of Bug #631564. It is possible this version has not hit your repository. Could you provide your selinux policy version? According to Bug #631564, AVC denials were fixed in selinux-policy-3.7.19-55 or later. Regards -steve
Sorry, I was originally planning to include this info in the initial report, but spaced it: selinux-policy-3.9.7-24 is installed on this system.
Additionally, the boolean referenced in bug 631564 does not appear to be present on my F14 system: [root ~]# semanage boolean -l | grep -i corosync [root ~]
Adding Dan on needinfo: Dan, I'm a bit lost here - any info your team could add here would be appreciated. Thanks -steve
Looks like an SELinux issue. Need to add fs_list_hugetlbfs(qdiskd_t) Does qdiskd_t need more privs then just listing the contents of hugetlbfs? Miroslav there is a bug in the lvm.if in F14 that has been fixed in Rawhide. Need to back port these fixes to F13/RHEL6 also.
Not sure on qdisk privs - I have added lon as needinfo as he maintains that code. He can tell you what other privs if any are required.
I added fixes to selinux-policy-3.9.7-29.fc14, selinux-policy-3.7.19-87.fc13 and RHEL6.
selinux-policy-3.9.7-29.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/selinux-policy-3.9.7-29.fc14
selinux-policy-3.9.7-29.fc14 has been pushed to the Fedora 14 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update selinux-policy'. You can provide feedback for this update here: https://admin.fedoraproject.org/updates/selinux-policy-3.9.7-29.fc14
selinux-policy-3.9.7-29.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.