+++ This bug was initially created as a clone of Bug #1311911 +++ Description of problem: volume not getting exported after setting the option ganesha.enable Version-Release number of selected component (if applicable): glusterfs-ganesha-3.7.8-1.el7.x86_64 nfs-ganesha-2.2.0-12.el6rhs.x86_64 glusterfs-3.7.8-1.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. Setup nfs-ganesha on 4 nodes 2. Create a 2X2 volume. 3. Start the volume 4. set the volume option- ganesha.enable on. it says success, but the volume is actually not exported Export file is present [root@dhcp46-59 ~]# cat /etc/ganesha/exports/export.testvol.conf # WARNING : Using Gluster CLI will overwrite manual # changes made to this file. To avoid it, edit the # file and run ganesha-ha.sh --refresh-config. EXPORT{ Export_Id= 2 ; Path = "/testvol"; FSAL { name = GLUSTER; hostname="localhost"; volume="testvol"; } Access_type = RW; Disable_ACL = true; Squash="No_root_squash"; Pseudo="/testvol"; Protocols = "3", "4" ; Transports = "UDP","TCP"; SecType = "sys"; } Also ganesha.conf file has entry of this config file: [root@dhcp46-59 ~]# cat /etc/ganesha/ganesha.conf ################################################### # # EXPORT # # To function, all that is required is an EXPORT # # Define the absolute minimal export # #EXPORT #{ # Export Id (mandatory, each EXPORT must have a unique Export_Id) # Export_Id = 77; # Exported path (mandatory) # Path = "/testvol"; # Pseudo Path (required for NFS v4) # Pseudo = "/testvol"; # Required for access (default is None) # Could use CLIENT blocks instead # Access_Type = RW; # Allow root access # Squash = No_Root_Squash; # Security flavor supported # SecType = "sys"; # Exporting FSAL # FSAL { # Name = "GLUSTER"; # Hostname = localhost; # Volume = "testvol"; # } #} ####################################################### #Create this export block in a file which has the following parameters #in the global part. Or create a separate file with the export block #and include in the following block. NFS_Core_Param { #Use supplied name other tha IP In NSM operations NSM_Use_Caller_Name = true; #Copy lock states into "/var/lib/nfs/ganesha" dir Clustered = false; #Use a non-privileged port for RQuota Rquota_Port = 4501; MNT_Port = 20048; NLM_Port = 32000; } %include "/etc/ganesha/exports/export.vol.conf But showmount does not show that volume is exported Actual results:showmount does not show that volume is exported Expected results: on setting ganesha.enable option volume should get exported Additional info: --- Additional comment from Apeksha on 2016-02-25 05:23:18 EST --- After Restarting the nfs-ganesha service on all the nodes, the volume is getting exported --- Additional comment from Jiffin on 2016-02-26 14:10:14 EST --- IMO the issue may be related to selinux policies, in the audit log the following logs can found while enable and disabling the ganesha.enable option type=USER_AVC msg=audit(1456522097.022:4933): pid=902 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_call interface=org.ganesha.nfsd.exportmgr member=AddExport dest=org.ganesha.nfsd spid=10631 tpid=26644 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1456521684.235:4932): pid=902 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_call interface=org.ganesha.nfsd.exportmgr member=RemoveExport dest=org.ganesha.nfsd spid=5403 tpid=26644 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=? I ran the wrapper script (/usr/libexec/dbus-send.sh) used by cli from the terminal with necessary parameters, the volume got exported. for example /usr/libexec/ganesha/dbus-send.sh /etc/ganesha/ <on/off> <volume name> --- Additional comment from Apeksha on 2016-02-29 04:45:27 EST --- **Steps when selinux was in enforcing mode [root@dhcp46-59 ~]# getenforce Enforcing [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# gluster v create rs 10.70.46.59:/root/brick2 force volume create: rs: success: please start the volume to access data [root@dhcp46-59 ~]# gluster v start rs volume start: rs: success [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# #gluster v set rs ganesha.enable on [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# grep -i "avc" /var/log/audit/audit.log [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# gluster v set rs ganesha.enable on volume set: success [root@dhcp46-59 ~]# grep -i "avc" /var/log/audit/audit.log type=USER_AVC msg=audit(1456767046.846:5613): pid=902 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=signal interface=org.ganesha.nfsd.exportmgr member=AddExport dest=org.ganesha.nfsd spid=1613 tpid=26644 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' [root@dhcp46-59 ~]# cat /var/log/audit/audit.log | audit2allow #============= glusterd_t ============== allow glusterd_t initrc_t:dbus send_msg; [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# showmount -e localhost Export list for localhost: /sample (everyone) [root@dhcp46-59 ~]# **Steps when selinux is in permissive mode [root@dhcp46-59 ~]# setenforce 0 [root@dhcp46-59 ~]# gluster v create rs1 10.70.46.59:/root/brick3 force volume create: rs1: success: please start the volume to access data [root@dhcp46-59 ~]# gluster v start rs1 volume start: rs1: success [root@dhcp46-59 ~]# gluster v set rs1 ganesha.enable on volume set: success [root@dhcp46-59 ~]# showmount -e localhost Export list for localhost: /sample (everyone) /rs1 (everyone) [root@dhcp46-59 ~]# grep -i "avc" /var/log/audit/audit.log type=USER_AVC msg=audit(1456767046.846:5613): pid=902 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=signal interface=org.ganesha.nfsd.exportmgr member=AddExport dest=org.ganesha.nfsd spid=1613 tpid=26644 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1456767084.524:5622): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='avc: received setenforce notice (enforcing=0) exe="/usr/lib/systemd/systemd" sauid=0 hostname=? addr=? terminal=?' type=USER_AVC msg=audit(1456767110.891:5623): pid=902 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=signal interface=org.ganesha.nfsd.exportmgr member=AddExport dest=org.ganesha.nfsd spid=2540 tpid=26644 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' [root@dhcp46-59 ~]# cat /var/log/audit/audit.log | audit2allow #============= glusterd_t ============== allow glusterd_t initrc_t:dbus send_msg; [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# [root@dhcp46-59 ~]# rpm -qa | grep selinux-policy selinux-policy-3.13.1-60.el7_2.3.noarch selinux-policy-targeted-3.13.1-60.el7_2.3.noarch [root@dhcp46-59 ~]# --- Additional comment from Shashank Raj on 2016-03-28 11:45:33 EDT --- Observed the same issue with 3.1.3 build (3.7.9-1) as well where in volume doesn't get exported after setting ganesha.enable on [root@dhcp46-247 brick0]# gluster volume set testvol ganesha.enable on volume set: success and it shows below user avc in audit.log type=USER_AVC msg=audit(1459163604.191:3776): pid=654 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=method_call interface=org.ganesha.nfsd.exportmgr member=RemoveExport dest=org.ganesha.nfsd spid=27599 tpid=28904 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?' [root@dhcp46-247 ~]# rpm -qa|grep selinux selinux-policy-targeted-3.13.1-60.el7.noarch selinux-policy-3.13.1-60.el7.noarch --- Additional comment from Shashank Raj on 2016-03-28 12:04:21 EDT --- In permissive mode: [root@dhcp46-247 exports]# setenforce 0 [root@dhcp46-247 exports]# getenforce Permissive [root@dhcp46-247 exports]# gluster volume list gluster_shared_storage newvol [root@dhcp46-247 exports]# gluster volume set newvol ganesha.enable on volume set: success [root@dhcp46-247 exports]# showmount -e localhost Export list for localhost: /newvol (everyone) Below messages in audit.log type=MAC_STATUS msg=audit(1459179476.783:3983): enforcing=0 old_enforcing=1 auid=0 ses=182 type=SYSCALL msg=audit(1459179476.783:3983): arch=c000003e syscall=1 success=yes exit=1 a0=3 a1=7ffe1ea8ef10 a2=1 a3=7ffe1ea8ec90 items=0 ppid=8970 pid=13164 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=182 comm="setenforce" exe="/usr/sbin/setenforce" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null) type=USER_AVC msg=audit(1459179552.967:3984): pid=654 uid=81 auid=4294967295 ses=4294967295 subj=system_u:system_r:system_dbusd_t:s0-s0:c0.c1023 msg='avc: denied { send_msg } for msgtype=signal interface=org.ganesha.nfsd.exportmgr member=AddExport dest=org.ganesha.nfsd spid=13573 tpid=28904 scontext=system_u:system_r:glusterd_t:s0 tcontext=system_u:system_r:initrc_t:s0 tclass=dbus exe="/usr/bin/dbus-daemon" sauid=81 hostname=? addr=? terminal=?'
Verified this bug with the selinux-policy-3.13.1-60.el7_2.4.noarch and nfs-ganesha-2.3.1-6.el7rhgs.x86_64, and its working as expected. After upgrading to the above selinux policy, volumes can be exported through ganesha and no denial AVC's are seen in audit.log Based on the above observation, marking this bug as Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2016:1247