Bug 1341183 - Call_bail of a frame due to not able to find a saved frame in reply
Summary: Call_bail of a frame due to not able to find a saved frame in reply
Alias: None
Product: GlusterFS
Classification: Community
Component: rpc
Version: 3.7.11
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
Depends On: 1340361 1421937
Blocks: 1340362
TreeView+ depends on / blocked
Reported: 2016-05-31 12:27 UTC by Niels de Vos
Modified: 2017-09-06 10:48 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1340361
Last Closed: 2017-03-08 10:49:06 UTC
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:

Attachments (Terms of Use)

Description Niels de Vos 2016-05-31 12:27:38 UTC
+++ This bug was initially created as a clone of Bug #1340361 +++

Description of problem:

This was observed on one of user's production setup.

[raghu@unused 01610290]$ grep "cannot lookup the saved" <client-log>
[2016-02-15 22:40:02.575925] C [rpc-clnt.c:452:rpc_clnt_fill_request_info] <client-log>: cannot lookup the saved frame corresponding to xid (14161323)

Above is the log-message indicating that a reply from server couldn't lookup a saved-frame to unwind the response. The xid (0xd815ab) matches to one of the unaccounted call-bails. Similar msg was seen from other log.

As for the RCA for not able to lookup a saved frame for response, I saw in rpc-clnt:

1. Submit the request to transport for transmission to brick.
2. Save the frame for future reference while processing reply.

Now, if we get a response between 1 and 2 (before we were able to save the frame), we would be saving the frame of a request whose reply is already received. This can result in call-bail.

Version-Release number of selected component (if applicable):
Zero day bug, present in all releases

How reproducible:
Racy. Not consistent

Steps to Reproduce:

Actual results:
A reply didn't have an associated saved-frame in rpc-clnt layer.

Expected results:
A reply should always have a saved-frame in rpc-clnt layer.

Additional info:

--- Additional comment from Vijay Bellur on 2016-05-27 09:36:29 CEST ---

REVIEW: http://review.gluster.org/14547 (rpc-clnt: save the frame before submitting request to transport) posted (#1) for review on master by Raghavendra G (rgowdapp@redhat.com)

Comment 1 Kaushal 2017-03-08 10:49:06 UTC
This bug is getting closed because GlusteFS-3.7 has reached its end-of-life.

Note: This bug is being closed using a script. No verification has been performed to check if it still exists on newer releases of GlusterFS.
If this bug still exists in newer GlusterFS releases, please reopen this bug against the newer release.

Note You need to log in before you can comment on or make changes to this bug.