Bug 1903582

Summary: [nfs-ganesha] [rhel 8] showmount -e command failing with error "rpc mount export: RPC: Timed out"
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Arun Kumar <arukumar>
Component: nfs-ganeshaAssignee: Frank Filz <ffilz>
Status: CLOSED ERRATA QA Contact: Upasana <ubansal>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.5CC: arjsharm, dang, ffilz, grajoria, kkeithle, mbenjamin, pprakash, prprakas, puebele, rhs-bugs, shilpsha, skoduri, ssivakum, storage-qa-internal, ubansal
Target Milestone: ---Keywords: Regression, ZStream
Target Release: RHGS 3.5.z Batch Update 4   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: nfs-ganesha-3.4-1.el8rhgs (rhgs-3.5.4) Doc Type: Bug Fix
Doc Text:
Cause: This was only seen in internal testing. Engineering was unable to reproduce or determine the root cause. Consequence: tests would hang. Fix: ganesha was rebased to nfs-ganesha 3.x. Result: there has been no recurrence of the hang since the rebase.
Story Points: ---
Clone Of:
: 1903976 (view as bug list) Environment:
Last Closed: 2021-04-29 07:21:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1901599, 1903976    

Comment 1 Frank Filz 2020-12-02 14:54:31 UTC
What was done prior to this? Ganesha must be hung somewhere. We probably need a core dump.

Comment 2 Frank Filz 2020-12-02 21:50:20 UTC
I've hopped on the system and looked at the Ganesha process with gdb. Nothing seems to be wrong, yet Ganesha doesn't process any RPC calls. I'm not sure how to figure out what might be wrong.

We may need a set of repeatable steps to re-create so we can enable various debug while recreating.

Comment 3 Frank Filz 2020-12-02 22:23:37 UTC
Just to be clear, we definitely need the set of steps to re-create and verification that it is indeed reproducible.

We will have to enable debug etc. while re-creating in order to have any hope of determining the cause and thence how to fix it.

Comment 10 Frank Filz 2020-12-03 18:18:43 UTC
I have brought up Ganesha on a clean system and do not see this issue.

We need a clean set of instructions that reproduce the problem from a clean system so that we can then enable various debug during Ganesha startup to track down the issue. Looking at Ganeaha after the fact doesn't show any problems, so something went haywire during startup, and it may be caused by something leftover from the previous crash or something.

Thanks

Frank

Comment 11 Frank Filz 2020-12-03 18:39:14 UTC
Also, if we could add debug on the Ganesha restart:

In NFS_CORE_PARAM block add:

RPC_Debug_Flags = 0xffffffff;

In the LOG { COMPONENTS { } } block add:

ALL=FULL_DEBUG;

Or change command line to -N NIV_FULL_DEBUG instead of -N NIV_EVENT

That will spit out a lot of logging, but if the recreate is just restart Ganesha and then issue showmount -e that won't be awful.

Make sure to rotate the log or something so the log file JUST contains the final instance of Ganesha.

Comment 15 Yaniv Kaul 2020-12-07 13:04:51 UTC
Can we get the details that Frank asked in comment 11 and clear the NEEDINFO?

Comment 26 errata-xmlrpc 2021-04-29 07:21:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nfs-ganesha bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1463