Bug 1397846 - [Compound FOPS]: seeing lot of brick log errors saying matching lock not found for unlock
Summary: [Compound FOPS]: seeing lot of brick log errors saying matching lock not foun...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: replicate
Version: rhgs-3.2
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: RHGS 3.2.0
Assignee: Krutika Dhananjay
QA Contact: Nag Pavan Chilakam
URL:
Whiteboard:
Depends On:
Blocks: 1351528
TreeView+ depends on / blocked
 
Reported: 2016-11-23 12:57 UTC by Nag Pavan Chilakam
Modified: 2017-03-23 06:21 UTC (History)
4 users (show)

Fixed In Version: glusterfs-3.8.4-6
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-03-23 06:21:24 UTC
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0486 0 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.2.0 security, bug fix, and enhancement update 2017-03-23 09:18:45 UTC

Description Nag Pavan Chilakam 2016-11-23 12:57:50 UTC 
When I do file operations with compound fops enabled, I see lot of brick error messsages as below:
[2016-11-23 12:51:37.918491] E [inodelk.c:304:__inode_unlock_lock] 0-cfops-locks:  Matching lock not found for unlock 0-9223372036854775807, by 6c10e675c57f0000 on 0x7faf7000cb30
[2016-11-23 12:51:37.918537] E [MSGID: 136002] [decompounder.c:370:dc_finodelk_cbk] 0-cfops-decompounder: fop number 2 failed. Unwinding. [Invalid argument]
[2016-11-23 12:51:37.918593] E [MSGID: 115090] [server-rpc-fops.c:2087:server_compound_cbk] 0-cfops-server: 526497: COMPOUND2 (0ea50530-9adc-424b-beb0-95f52027b671) ==> (Invalid argument) [Invalid argument]


steps to reproduce
1)create a 1x2 vol
2)fuse mount on two different fuse clients
3)untar kernel images into two different directories (one from each client)
4)while untar is happening just bring down one brick
5)check the brick log messsages
bring back the brick online
check brick logs

we can see these errors continuosly
Comment 4 Krutika Dhananjay 2016-11-28 06:23:11 UTC 
Patch https://code.engineering.redhat.com/gerrit/91332 fixes this bug and is merged now. Moving this bug to MODIFIED state.
Comment 6 Nag Pavan Chilakam 2016-11-29 06:57:07 UTC 
hit this while validating RFE 1360978 - [RFE]Reducing number of network round trips
Comment 7 Krutika Dhananjay 2016-11-30 11:31:23 UTC 
(In reply to Krutika Dhananjay from comment #4)
> Patch https://code.engineering.redhat.com/gerrit/91332 fixes this bug and is
> merged now. Moving this bug to MODIFIED state.

Just wanted to clear one thing - when a brick is brought down and then brought back online (irrespective of whether compound-fops is enabled or not on the volume), the 'matching lock not found' failure log messages will be seen. And the appearance of these log messages is valid in that case.

However, if all replicas are up and still during IO these logs appear on the brick logs (which is what you observed), only then it is a bug.

-Krutika
Comment 8 Nag Pavan Chilakam 2016-12-13 13:31:39 UTC 
I am not seeing these errors anymore on 3.8.4-8 with the same steps.
hence moving to verified
Comment 10 errata-xmlrpc 2017-03-23 06:21:24 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0486.html
Note You need to log in before you can comment on or make changes to this bug.