Bug 1340362

Summary:	Call_bail of a frame due to not able to find a saved frame in reply
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Raghavendra G <rgowdapp>
Component:	core	Assignee:	Bug Updates Notification Mailing List <rhs-bugs>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Anoop <annair>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.1	CC:	bugs, rhs-bugs
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1340361	Environment:
Last Closed:	2018-02-07 04:22:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1341183
Bug Blocks:

Description Raghavendra G 2016-05-27 07:34:59 UTC

+++ This bug was initially created as a clone of Bug #1340361 +++

Description of problem:

This was observed on one of user's production setup.

[raghu@unused 01610290]$ grep "cannot lookup the saved" <client-log>
[2016-02-15 22:40:02.575925] C [rpc-clnt.c:452:rpc_clnt_fill_request_info] <client-log>: cannot lookup the saved frame corresponding to xid (14161323)

Above is the log-message indicating that a reply from server couldn't lookup a saved-frame to unwind the response. The xid (0xd815ab) matches to one of the unaccounted call-bails. Similar msg was seen from other log.

As for the RCA for not able to lookup a saved frame for response, I saw in rpc-clnt:

1. Submit the request to transport for transmission to brick.
2. Save the frame for future reference while processing reply.

Now, if we get a response between 1 and 2 (before we were able to save the frame), we would be saving the frame of a request whose reply is already received. This can result in call-bail.


Version-Release number of selected component (if applicable):
Zero day bug, present in all releases

How reproducible:
Racy. Not consistent

Steps to Reproduce:
1.
2.
3.

Actual results:
A reply didn't have an associated saved-frame in rpc-clnt layer.

Expected results:
A reply should always have a saved-frame in rpc-clnt layer.


Additional info:

Comment 2 Amar Tumballi 2018-02-07 04:22:14 UTC

Thank you for the bug report. 

This particular bug was fixed and a update package was published (RHGS 3.3.1+). Please feel free to report any further bugs you find, or make further reports if this bug is not fixed after you install the update.