Description of problem: Upgrade nfs-ganesha from 3.2 to 3.2 async is breaking due to selinux boolean ganesha_use_fusefs in off state Version-Release number of selected component (if applicable): # rpm -qa | grep ganesha nfs-ganesha-debuginfo-2.4.1-11.el7rhgs.x86_64 nfs-ganesha-2.4.1-11.el7rhgs.x86_64 nfs-ganesha-gluster-2.4.1-11.el7rhgs.x86_64 glusterfs-ganesha-3.8.4-18.5.el7rhgs.x86_64 How reproducible: 1/1 Steps to Reproduce: 1.Create 4 node ganesha cluster up and running and upgrade it to latest async glusterfs-ganesha-3.8.4-18.5.el7rhgs.x86_64 following ganesha upgrade steps in offline mode. After upgrade,before enabling nfs-ganesha,set gluster_use_execmem boolean to on # setsebool -P gluster_use_execmem on [root@dhcp37-58 nfs-ganesha]# semanage boolean -l | grep gluster_use_execmem gluster_use_execmem (on , on) Allow gluster to use execmem Actual results: ganesha service fails to come up and after upgrade ganesha_use_fusefs boolean was in off state. [root@dhcp37-58 ~]# semanage boolean -l | grep ganesha ganesha_use_fusefs (off , off) Allow ganesha to use fusefs Expected results: Ganesha upgrade should not break Additional info: ================ # ausearch -m avc -m user_avc -m selinux_err -i -ts recent | grep ganesha type=SYSCALL msg=audit(07/11/2017 18:13:12.309:6326) : arch=x86_64 syscall=open success=no exit=EACCES(Permission denied) a0=0x7f57d4f9b490 a1=O_RDONLY a2=0x1b6 a3=0x24 items=0 ppid=1 pid=2852 auid=unset uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none) ses=unset comm=ganesha.nfsd exe=/usr/bin/ganesha.nfsd subj=system_u:system_r:ganesha_t:s0 key=(null) type=AVC msg=audit(07/11/2017 18:13:12.309:6326) : avc: denied { search } for pid=2852 comm=ganesha.nfsd name=/ dev="fuse" ino=1 scontext=system_u:system_r:ganesha_t:s0 tcontext=system_u:object_r:fusefs_t:s0 tclass=dir ============== # tailf /var/log/ganesha.log 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[Admin] destroy_fsals :FSAL :EVENT :Exports for FSAL GLUSTER shut down 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[Admin] destroy_fsals :FSAL :CRIT :Extra references (5) hanging around to FSAL GLUSTER 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[Admin] destroy_fsals :FSAL :EVENT :Unloading FSAL GLUSTER 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[Admin] destroy_fsals :FSAL :EVENT :FSAL GLUSTER unloaded 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[Admin] do_shutdown :MAIN :EVENT :FSAL system destroyed. 11/07/2017 17:26:19 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-7322[main] nfs_start :MAIN :EVENT :NFS EXIT: regular exit 11/07/2017 18:13:12 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-2851[main] main :MAIN :EVENT :ganesha.nfsd Starting: Ganesha Version /builddir/build/BUILD/nfs-ganesha-2.4.1/src, built at Jun 7 2017 01:38:50 on 11/07/2017 18:13:12 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-2852[main] main :NFS STARTUP :CRIT :Error (token scan) while parsing (/etc/ganesha/ganesha.conf) 11/07/2017 18:13:12 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-2852[main] config_errs_to_log :CONFIG :CRIT :Config File (<unknown file>:0): new file (/etc/ganesha/ganesha.conf) open error (Permission denied), ignored 11/07/2017 18:13:12 : epoch 42250000 : dhcp37-58.lab.eng.blr.redhat.com : ganesha.nfsd-2852[main] main :NFS STARTUP :FATAL :Fatal errors. Server exiting... Meanwhile I will check for freash installation of 3.2 async with RHEL 7.4 and will update the bug
Fresh setup installation with 3.2 async works fine ganesha_use_fusefs boolean is set to on while doing gluster nfs-ganesha enable. # semanage boolean -l | grep ganesha ganesha_use_fusefs (on , on) Allow ganesha to use fusefs Issue is only observed in upgrade path
This is what I think is happening during an update: According to the yum logs the system first had selinux-policy-targeted-3.13.1-102.el7_3.16. There is no ganesha_use_fusefs in this package. Then the system was updated to RHEL-7.4. glusterfs-ganesha was updated at 18:00:12. Then at 18:00:37 selinux-policy-targeted was updated to 3.13.1-166.el7. This has ganesha_use_fusefs. ganesha_use_fusefs still wasn't available when glusterfs-ganesha was updated so the semanage command (silently) failed. rpm only allows a 'Requires: selinux-policy-targeted >= NV'. I.e. NV = 3.13.1. It doesn't allow a 'Requires: selinux-policy-targeted >= NVR'. I.e. NVR = 3.13.1-166. Thus, for the purposes of upgrading, 3.13.1-102.el7_3.16 satisfies the Requires: but doesn't have the necessary ganesha_fuse_fusefs for the %post to work. Of course on a fresh install you will get the correct version of selinux-policy-targeted and everything works as expected. Off the top of my head the only way to force selinux-policy-targeted to be updated before glusterfs-ganesha is to explicitly update it first, before applying the rest of the update. IOW this has to be prominently documented in the Release Notes.
Perhaps Lukas can suggest some trick to make selinux-policy-targeted update before glusterfs-ganesha?
Kaleb, Yum update will pull all the packages all at once,ganesha and selinux packages. We cannot only update the selinux package first followed by ganesha package. However after upgrading both selinux and ganesha packages,we can document to enable this boolean manually before doing gluster nfs-ganesha enable. I will verify this steps manually too following the upgrade path. Need your opinion on this...
I believe Lukas' suggestion of using the %trigger to run the semanage command after selinux-policy-targeted is updated will fix this.
Verified the path from 3.2 to 3.2 async. After upgrade from 3.2 to 3.2 async, setting boolean ganesha_use_fusefs to ON manually before enabling ganesha,works fine. Raising the documentation bug for 3.2 async to set this boolean on manually after upgrade
Raised the documentation bug for the same for 3.2 async- https://bugzilla.redhat.com/show_bug.cgi?id=1470146