Bug 1466007
Summary: | [Ganesha] : Unable to bring up a Ganesha HA cluster on latest bits. | ||
---|---|---|---|
Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Ambarish <asoman> |
Component: | nfs-ganesha | Assignee: | Jiffin <jthottan> |
Status: | CLOSED ERRATA | QA Contact: | Ambarish <asoman> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | rhgs-3.3 | CC: | amukherj, bturner, dang, ffilz, jthottan, kkeithle, mbenjamin, rcyriac, rhinduja, rhs-bugs, skoduri, storage-qa-internal |
Target Milestone: | --- | Keywords: | Regression |
Target Release: | RHGS 3.3.0 | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | nfs-ganesha-2.4.4-16 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2017-09-21 04:47:57 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1417151 | ||
Attachments: |
Description
Ambarish
2017-06-28 17:00:58 UTC
This is what I see in messages when I try to enable Ganesha : Jun 28 13:00:08 gqas013 systemd: Starting Process NFS-Ganesha configuration... Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount: Directory /var/run/gluster/shared_storage to mount over is not empty, mounting anyway. Jun 28 13:00:08 gqas013 systemd: Mounting /var/run/gluster/shared_storage... Jun 28 13:00:08 gqas013 mount: /sbin/mount.glusterfs: according to mtab, GlusterFS is already mounted on /run/gluster/shared_storage Jun 28 13:00:08 gqas013 systemd: Started Process NFS-Ganesha configuration. Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount mount process exited, code=exited status=32 Jun 28 13:00:08 gqas013 systemd: Failed to mount /var/run/gluster/shared_storage. Jun 28 13:00:08 gqas013 systemd: Dependency failed for NFS-Ganesha file server. Jun 28 13:00:08 gqas013 systemd: Job nfs-ganesha.service/start failed with result 'dependency'. Jun 28 13:00:08 gqas013 systemd: Unit var-run-gluster-shared_storage.mount entered failed state. (In reply to Ambarish from comment #2) > This is what I see in messages when I try to enable Ganesha : > > Jun 28 13:00:08 gqas013 systemd: Starting Process NFS-Ganesha > configuration... > Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount: > Directory /var/run/gluster/shared_storage to mount over is not empty, > mounting anyway. > Jun 28 13:00:08 gqas013 systemd: Mounting /var/run/gluster/shared_storage... > Jun 28 13:00:08 gqas013 mount: /sbin/mount.glusterfs: according to mtab, > GlusterFS is already mounted on /run/gluster/shared_storage > Jun 28 13:00:08 gqas013 systemd: Started Process NFS-Ganesha configuration. > Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount mount > process exited, code=exited status=32 > Jun 28 13:00:08 gqas013 systemd: Failed to mount > /var/run/gluster/shared_storage. > Jun 28 13:00:08 gqas013 systemd: Dependency failed for NFS-Ganesha file > server. > Jun 28 13:00:08 gqas013 systemd: Job nfs-ganesha.service/start failed with > result 'dependency'. > Jun 28 13:00:08 gqas013 systemd: Unit var-run-gluster-shared_storage.mount > entered failed state. So is mounting SS failing on latest bits? (In reply to Ambarish from comment #4) > (In reply to Ambarish from comment #2) > > This is what I see in messages when I try to enable Ganesha : > > > > Jun 28 13:00:08 gqas013 systemd: Starting Process NFS-Ganesha > > configuration... > > Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount: > > Directory /var/run/gluster/shared_storage to mount over is not empty, > > mounting anyway. > > Jun 28 13:00:08 gqas013 systemd: Mounting /var/run/gluster/shared_storage... > > Jun 28 13:00:08 gqas013 mount: /sbin/mount.glusterfs: according to mtab, > > GlusterFS is already mounted on /run/gluster/shared_storage > > Jun 28 13:00:08 gqas013 systemd: Started Process NFS-Ganesha configuration. > > Jun 28 13:00:08 gqas013 systemd: var-run-gluster-shared_storage.mount mount > > process exited, code=exited status=32 > > Jun 28 13:00:08 gqas013 systemd: Failed to mount > > /var/run/gluster/shared_storage. > > Jun 28 13:00:08 gqas013 systemd: Dependency failed for NFS-Ganesha file > > server. > > Jun 28 13:00:08 gqas013 systemd: Job nfs-ganesha.service/start failed with > > result 'dependency'. > > Jun 28 13:00:08 gqas013 systemd: Unit var-run-gluster-shared_storage.mount > > entered failed state. > > So is mounting SS failing on latest bits? [root@gqas013 ganesha]# mount |grep shared gqas013.sbu.lab.eng.bos.redhat.com:/gluster_shared_storage on /run/gluster/shared_storage type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) [root@gqas013 ganesha]# RCA : We had add dependency in nfs-ganesha system file for var-run-gluster-shared_storage.mount . This is service created by systemd automatically based on /etc/fstab entry during reboot. But when enable shared storage for the first time, there is no such service. So nfs-ganesha will fail to start of because of that dependency Workaround: For the time being remove that dependency from nfs-ganesha.service file edit following file /usr/lib/systemd/system/nfs-ganesha.service remove dependency on var-run-gluster-shared_storage.mount from Requires(remove this line completely) and After (In reply to Jiffin from comment #7) > RCA : > We had add dependency in nfs-ganesha system file for > var-run-gluster-shared_storage.mount . This is service created by systemd > automatically based on /etc/fstab entry during reboot. But when enable > shared storage for the first time, there is no such service. So nfs-ganesha > will fail to start of because of that dependency > > Workaround: > For the time being remove that dependency from nfs-ganesha.service file > edit following file /usr/lib/systemd/system/nfs-ganesha.service > remove dependency on var-run-gluster-shared_storage.mount from > Requires(remove this line completely) and After Other workaround which could be applied IMO is #umount /run/gluster/shared_storage #systemctl restart var-run-gluster-shared_storage.mount (make sure status of this service is SUCCESS) #then go ahead with the cluster setup. Created attachment 1297015 [details]
Remove dependency on glusterfssharedstorage for ganesha
Created attachment 1297425 [details]
Second patch , first patch miissed some of my changes
Created attachment 1299823 [details]
Start-nfs-ganesha-only-if-share-storage-mount-got-su.patch
Works fine on nfs-ganesha-2.4.4-16. I am able to bring up a Ganesha cluster,disable Ganesha and re-enable again. Moving to Verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:2779 |