Bug 1463711 - [GANESHA] Volume having ganesha enable on it fails to get re-export on rebooted node
[GANESHA] Volume having ganesha enable on it fails to get re-export on reboot...
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: nfs-ganesha (Show other bugs)
3.3
Unspecified Unspecified
urgent Severity unspecified
: ---
: RHGS 3.3.0
Assigned To: Daniel Gryniewicz
Manisha Saini
3.3.0-devel-freeze-exception
:
Depends On:
Blocks: 1417151
  Show dependency treegraph
 
Reported: 2017-06-21 10:35 EDT by Manisha Saini
Modified: 2017-09-21 00:47 EDT (History)
8 users (show)

See Also:
Fixed In Version: nfs-ganesha-2.4.4-10
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-09-21 00:47:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Manisha Saini 2017-06-21 10:35:42 EDT
Description of problem:
Volume having ganesha enable on it fails to get re-export on rebooted node.

Version-Release number of selected component (if applicable):
# rpm -qa | grep ganesha
nfs-ganesha-2.4.4-8.el7rhgs.x86_64
glusterfs-ganesha-3.8.4-25.el7rhgs.x86_64
nfs-ganesha-debuginfo-2.4.4-8.el7rhgs.x86_64
nfs-ganesha-gluster-2.4.4-8.el7rhgs.x86_64


How reproducible:
2/2

Steps to Reproduce:
1.Create a 4 node ganesha cluster
2.Create 3 volume.Enable ganesha on it.(Export the volume one by one)
3.Stop the 1st two volume.
4.Reboot 1 of the node
5.When the node comes up,Volume does not get re exported on rebooted node

Actual results:
3rd Volume should get re exported on the rebooted node 

Expected results:
Volume failed to export

Additional info:


ganesha.log
============================

21/06/2017 19:51:50 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] init_server_pkgs :NFS STARTUP :EVENT :Initializing ID Mapper.
21/06/2017 19:51:50 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] init_server_pkgs :NFS STARTUP :EVENT :ID Mapper successfully initialized.
21/06/2017 19:51:50 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_create_export :FSAL :EVENT :Volume ganeshavol1 exported at : '/'
21/06/2017 19:51:54 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_get_fs :FSAL :CRIT :Unable to initialize volume. Volume: ganeshavol1
21/06/2017 19:51:55 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on underlying FSAL GLUSTER
21/06/2017 19:51:55 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/ganeshavol1) to (/ganeshavol1)
21/06/2017 19:51:55 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_create_export :FSAL :EVENT :Volume ganeshavol3 exported at : '/'
21/06/2017 19:51:59 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_get_fs :FSAL :CRIT :Unable to initialize volume. Volume: ganeshavol3
21/06/2017 19:52:00 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on underlying FSAL GLUSTER
21/06/2017 19:52:00 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/ganeshavol3) to (/ganeshavol3)
21/06/2017 19:52:00 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_create_export :FSAL :EVENT :Volume ganeshavol2 exported at : '/'
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] fsal_attach_export :CONFIG :CRIT :Attaching export with out holding a reference!. hdl= = 0x0x7f9bf8f683d0
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] glusterfs_create_export :FSAL :CRIT :Unable to attach export. Export: /ganeshavol2
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on underlying FSAL GLUSTER
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/ganeshavol2) to (/ganeshavol2)
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] main :NFS STARTUP :WARN :No export entries found in configuration file !!!
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol1.conf:7): 1 validation errors in block FSAL
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol1.conf:7): Errors processing block (FSAL)
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol1.conf:4): 1 validation errors in block EXPORT
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol1.conf:4): Errors processing block (EXPORT)
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf:7): 1 validation errors in block FSAL
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf:7): Errors processing block (FSAL)
21/06/2017 19:52:05 : epoch 4a170000 : dhcp42-114.lab.eng.blr.redhat.com : ganesha.nfsd-3283[main] config_errs_to_log :CONFIG :CRIT :Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf:4): 1 validation errors in block EXPORT
===================================


==================

Order in which volume were exported-

}
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol1.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol3.conf"
%include "/var/run/gluster/shared_storage/nfs-ganesha/exports/export.ganeshavol2.conf"
========================

Rebooted Node-
[root@dhcp42-114 ~]# showmount -e
Export list for dhcp42-114.lab.eng.blr.redhat.com:


One of the other node-
[root@dhcp42-107 ganesha]# showmount -e
Export list for dhcp42-107.lab.eng.blr.redhat.com:
/ganeshavol2 (everyone)




Will attach the sosreports shortly
Comment 2 Daniel Gryniewicz 2017-06-21 10:51:57 EDT
This is fixed in 2.5.x already, and a backport is going into 2.4.x as I type this.  It's a simple 3-line fix.

I believe this should be included if at all possible.  It seems a problem that a client will probably hit at some point (an export for a volume that's not ready).  And, the problem is severe:  The only possible way out of this situation is to restart Ganesha, and it has the potential to corrupt memory or crash.
Comment 6 Manisha Saini 2017-06-29 09:40:56 EDT
Verified this bug on 

# rpm -qa | grep ganesha
nfs-ganesha-gluster-2.4.4-10.el7rhgs.x86_64
nfs-ganesha-debuginfo-2.4.4-10.el7rhgs.x86_64
glusterfs-ganesha-3.8.4-31.el7rhgs.x86_64
nfs-ganesha-2.4.4-10.el7rhgs.x86_64


Volume get exported on rebooted node.
Hence moving this to verified state.
Comment 8 errata-xmlrpc 2017-09-21 00:47:57 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:2779

Note You need to log in before you can comment on or make changes to this bug.