Bug 1459905 - [Ganesha] : Volume export via Ganesha fails while trying to export a stopped volume,even after volume start.
[Ganesha] : Volume export via Ganesha fails while trying to export a stopped ...
Status: CLOSED WONTFIX
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: nfs-ganesha (Show other bugs)
3.3
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Kaleb KEITHLEY
Ambarish
: ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-08 10:03 EDT by Ambarish
Modified: 2017-08-13 21:59 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
Story Points: ---
Clone Of:
: 1478647 (view as bug list)
Environment:
Last Closed: 2017-08-11 06:11:36 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ambarish 2017-06-08 10:03:06 EDT
Description of problem:
-----------------------

4 node cluster,1 2*2 volume.

Keep volume in Stopped state.

Create Ganesha cluster,export the volume,Check showmount -e.

showmount wont show the export till Ganesha is restarted.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------

nfs-ganesha-debuginfo-2.4.4-8.el7rhgs.x86_64
glusterfs-ganesha-3.8.4-26.el7rhgs.x86_64


How reproducible:
-----------------

Fairly,on my setup.


Additional info:
----------------

Volume Name: testvol
Type: Distributed-Replicate
Volume ID: 230c0729-d2d8-4c86-b696-d60b2472a36e
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: gqas013.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick0
Brick2: gqas005.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick1
Brick3: gqas006.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick2
Brick4: gqas008.sbu.lab.eng.bos.redhat.com:/bricks/testvol_brick3
Options Reconfigured:
ganesha.enable: on
features.cache-invalidation: on
server.allow-insecure: on
performance.stat-prefetch: off
transport.address-family: inet
nfs.disable: on
nfs-ganesha: enable
cluster.enable-shared-storage: enable
Comment 2 Ambarish 2017-06-08 10:10:07 EDT
I see this in Ganesha.log :

<snip>

08/06/2017 05:26:53 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] glusterfs_create_export :FSAL :EVENT :Volume testvol exported at : '/'
08/06/2017 05:26:57 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] glusterfs_get_fs :FSAL :CRIT :Unable to initialize volume. Volume: testvol
08/06/2017 05:26:58 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on underlying FSAL GLUSTER
08/06/2017 05:26:58 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/testvol) to (/testvol)
08/06/2017 05:26:58 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] gsh_export_addexport :EXPORT :CRIT :0 export entries in /var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf added because (export create, block validation) errors
08/06/2017 05:26:58 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] dbus_message_entrypoint :DBUS :MAJ :Method (AddExport) on (org.ganesha.nfsd.exportmgr) failed: name = (org.freedesktop.DBus.Error.InvalidFileContent), message = (0 export entries in /var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf added because (export create, block validation) errors. Details:
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:7): 1 validation errors in block FSAL
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:7): Errors processing block (FSAL)
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:4): 1 validation errors in block EXPORT
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:4): Errors found in configuration block EXPORT
)
08/06/2017 05:27:12 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] glusterfs_create_export :FSAL :EVENT :Volume testvol exported at : '/'
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] fsal_attach_export :CONFIG :CRIT :Attaching export with out holding a reference!. hdl= = 0x0x7f57ade063d0
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] glusterfs_create_export :FSAL :CRIT :Unable to attach export. Export: /testvol
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] mdcache_fsal_create_export :FSAL :MAJ :Failed to call create_export on underlying FSAL GLUSTER
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] fsal_cfg_commit :CONFIG :CRIT :Could not create export for (/testvol) to (/testvol)
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] gsh_export_addexport :EXPORT :CRIT :0 export entries in /var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf added because (export create, block validation) errors
08/06/2017 05:27:17 : epoch 12520000 : gqas013.sbu.lab.eng.bos.redhat.com : ganesha.nfsd-24063[dbus_heartbeat] dbus_message_entrypoint :DBUS :MAJ :Method (AddExport) on (org.ganesha.nfsd.exportmgr) failed: name = (org.freedesktop.DBus.Error.InvalidFileContent), message = (0 export entries in /var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf added because (export create, block validation) errors. Details:
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:7): 1 validation errors in block FSAL
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:7): Errors processing block (FSAL)
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:4): 1 validation errors in block EXPORT
Config File (/var/run/gluster/shared_storage/nfs-ganesha/exports/export.testvol.conf:4): Errors found in configuration block EXPORT

</snip>
Comment 4 Daniel Gryniewicz 2017-06-08 10:35:29 EDT
I believe this is correct.  Ganesha can only export active volumes.  You *may* be able to start the volume, and re-export it, but that may not work.

The way Ganesha is currently architected, it is not possible to export an inactive volume, start it, and expect it to show up with no further steps.
Comment 5 Ambarish 2017-06-08 10:37:05 EDT

(In reply to Daniel Gryniewicz from comment #4)
> I believe this is correct.  Ganesha can only export active volumes.  You
> *may* be able to start the volume, and re-export it, but that may not work.
> 
> The way Ganesha is currently architected, it is not possible to export an
> inactive volume, start it, and expect it to show up with no further steps.

This seems to work on Manisha's 3.2 setup,though.without a service restart.

We just tried this together on her setup,before reporting this.
Comment 6 Frank Filz 2017-06-08 12:14:54 EDT
Is there a difference in how Gluster presents a stopped volume?

If FSAL_GLUSTER is unable to do what it needs in the create_export method, and returns an error, the upper layers of Ganesha have no way to know that the volume might be available again. Add export or update export after the volume has been started should result in success (the export will not have been added even though it was in the config file originally).

I really don't see any way this is other than working as designed.
Comment 7 Soumya Koduri 2017-06-08 13:42:43 EDT
As part of starting the volume, glusterd (Gluster management process) invoked a hook script which sends dbus signal to export the volume. Maybe the volume was still being intialized by the time dbus signal was sent.

@Ambarish,

Did you try the same test case in any other 3.3 setup (using VMs)?
Comment 8 Ambarish 2017-06-11 23:31:52 EDT
(In reply to Soumya Koduri from comment #7)
> As part of starting the volume, glusterd (Gluster management process)
> invoked a hook script which sends dbus signal to export the volume. Maybe
> the volume was still being intialized by the time dbus signal was sent.
> 
> @Ambarish,
> 
> Did you try the same test case in any other 3.3 setup (using VMs)?

Cannot repro on 3.3 VMs.
Comment 9 Soumya Koduri 2017-06-15 03:36:36 EDT
Ambarish,

Could you please share the setup as we are unable to reproduce this issue.

Note You need to log in before you can comment on or make changes to this bug.