Bug 1565015

Summary: [Ganesha] File Locking test is failing on ganesha v3 protocol
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Manisha Saini <msaini>
Component: nfs-ganeshaAssignee: Jiffin <jthottan>
Status: CLOSED ERRATA QA Contact: Manisha Saini <msaini>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: rhgs-3.4CC: amukherj, bkunal, ffilz, jthottan, rhinduja, rhs-bugs, storage-qa-internal
Target Milestone: ---Keywords: Regression
Target Release: RHGS 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.12.2-9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1573065 (view as bug list) Environment:
Last Closed: 2018-09-04 06:46:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1480138, 1503137, 1573065    

Description Manisha Saini 2018-04-09 07:04:30 UTC 
Description of problem:

File Locking test is failing on ganesha V3.While verifying BZ-1480138,hit this issue.Even cthon and file locking test is failing with vers=3


Version-Release number of selected component (if applicable):

# rpm -qa | grep ganesha
glusterfs-ganesha-3.12.2-7.el7rhgs.x86_64
nfs-ganesha-gluster-2.5.5-4.el7rhgs.x86_64
nfs-ganesha-2.5.5-4.el7rhgs.x86_64



How reproducible:
3/3

Steps to Reproduce:
1.CReate 4 node ganesha cluster
2.Create 4 x 3 = 12 dist-replicate volume
3.Export the volume via ganesha
4.Run cthon test suit and file locking test

Actual results:

Cthon test is failing-

---------------------------
Test #1 - Test regions of an unlocked file.
	Parent: 1.1  - F_TEST  [               0,               1] PASSED.
	Parent: 1.2  - F_TEST  [               0,          ENDING] PASSED.
	Parent: 1.3  - F_TEST  [               0,7fffffffffffffff] PASSED.
	Parent: 1.4  - F_TEST  [               1,               1] PASSED.
	Parent: 1.5  - F_TEST  [               1,          ENDING] PASSED.
	Parent: 1.6  - F_TEST  [               1,7fffffffffffffff] PASSED.
	Parent: 1.7  - F_TEST  [7fffffffffffffff,               1] PASSED.
	Parent: 1.8  - F_TEST  [7fffffffffffffff,          ENDING] PASSED.
	Parent: 1.9  - F_TEST  [7fffffffffffffff,7fffffffffffffff] PASSED.

Test #2 - Try to lock the whole file.
	Parent: 2.0  - F_TLOCK [               0,          ENDING] FAILED!
	Parent: **** Expected success, returned errno=37...
	Parent: **** Probably implementation error.

** PARENT pass 1 results: 9/9 pass, 0/0 warn, 1/1 fail (pass/total).

**  CHILD pass 1 results: 0/0 pass, 0/0 warn, 0/0 fail (pass/total).
lock tests failed
-------------------------------

File locking test-

--------
# ./a.out /mnt/ganesha_setup2/1G 
opening /mnt/ganesha_setup2/1G
opened; hit Enter to lock... 
locking
fcntl failed (No locks available)

-----------


Expected results:
Locking test should pass with ganesha v3


Additional info:

Messages in ganesha.log-

09/04/2018 01:48:53 : epoch 10430000 : dhcp37-121.lab.eng.blr.redhat.com : ganesha.nfsd-9845[work-214] nsm_monitor :NLM :CRIT :Can not monitor dhcp46-125.lab.eng.blr.redhat.com SM_MON status 1

/var/log/messages-

Apr  9 01:46:02 dhcp37-121 rpc.statd[9837]: Failed to insert: creating /var/lib/nfs/statd/sm/dhcp46-125.lab.eng.blr.redhat.com: Permission denied
Apr  9 01:46:02 dhcp37-121 rpc.statd[9837]: STAT_FAIL to dhcp37-121.lab.eng.blr.redhat.com for SM_MON of dhcp46-125.lab.eng.blr.redhat.com
Comment 10 Frank Filz 2018-05-10 13:46:15 UTC 
Hmm, that's a bit concerning the cthon04 lock tests didn't cause the same log messages... On the other hand, chton04 does NOT have proper synchronization for it's blocked lock tests and it can end up never actually blocking on the lock...

Your multi-client lock test, by requiring manual intervention, assures that synchronization. 

You might also want to look in the Ganesha tree in src/tools/multilock. That is a tool that allows automation of something like the multi-client lock tests. It uses a tcp connection to communicate between the test driver and the lock client processes (which may run on the same host or multiple hosts).
Comment 12 errata-xmlrpc 2018-09-04 06:46:01 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607