Bug 803663 - Sever failed to create reply frame because of connection object being NULL
Sever failed to create reply frame because of connection object being NULL
Status: CLOSED WONTFIX
Product: GlusterFS
Classification: Community
Component: protocol (Show other bugs)
mainline
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Amar Tumballi
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-15 07:44 EDT by Shwetha Panduranga
Modified: 2015-12-01 11:45 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-04-17 14:11:51 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Brick Log File (27.20 KB, text/x-log)
2012-03-15 07:44 EDT, Shwetha Panduranga
no flags Details

  None (edit)
Description Shwetha Panduranga 2012-03-15 07:44:07 EDT
Created attachment 570265 [details]
Brick Log File

Description of problem:

Server failed to create frame because of connection object being NULL and doesn't send reply back to client . Hence client application hangs waiting for a response from server. 

[2012-03-14 23:33:43.510756] E [server-helpers.c:735:server_alloc_frame] (-->/usr/local/lib/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x360) [0x7f679d2970a9] (-->/usr/local/
lib/glusterfs/3.3.0qa27/xlator/protocol/server.so(server_finodelk+0xcb) [0x7f6793bd53db] (-->/usr/local/lib/glusterfs/3.3.0qa27/xlator/protocol/server.so(get_frame_from
_request+0x89) [0x7f6793bc11c5]))) 0-server: invalid argument: conn
[2012-03-14 23:33:43.510836] W [rpcsvc.c:1111:rpcsvc_error_reply] (-->/usr/local/lib/libgfrpc.so.0(rpc_transport_notify+0x130) [0x7f679d29cda8] (-->/usr/local/lib/libgf
rpc.so.0(rpcsvc_notify+0x181) [0x7f679d29744c] (-->/usr/local/lib/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x37b) [0x7f679d2970c4]))) 0-: sending a RPC error reply
[2012-03-14 23:33:43.511039] E [server-helpers.c:735:server_alloc_frame] (-->/usr/local/lib/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x360) [0x7f679d2970a9] (-->/usr/local/
lib/glusterfs/3.3.0qa27/xlator/protocol/server.so(server_finodelk+0xcb) [0x7f6793bd53db] (-->/usr/local/lib/glusterfs/3.3.0qa27/xlator/protocol/server.so(get_frame_from
_request+0x89) [0x7f6793bc11c5]))) 0-server: invalid argument: conn
[2012-03-14 23:33:43.511129] W [rpcsvc.c:1111:rpcsvc_error_reply] (-->/usr/local/lib/libgfrpc.so.0(rpc_transport_notify+0x130) [0x7f679d29cda8] (-->/usr/local/lib/libgf
rpc.so.0(rpcsvc_notify+0x181) [0x7f679d29744c] (-->/usr/local/lib/libgfrpc.so.0(rpcsvc_handle_rpc_call+0x37b) [0x7f679d2970c4]))) 0-: sending a RPC error reply
[2012-03-14 23:33:43.511596] I [socket.c:2377:socket_submit_reply] 0-tcp.dstore1-server: not connected (priv->connected = 255)
[2012-03-14 23:33:43.511648] E [rpcsvc.c:1078:rpcsvc_submit_generic] 0-rpc-service: failed to submit message (XID: 0x233872x, Program: GlusterFS 3.3.0qa27, ProgVers: 330, Proc: 30) to rpc-transport (tcp.dstore1-server)
[2012-03-14 23:33:43.512071] I [socket.c:2377:socket_submit_reply] 0-tcp.dstore1-server: not connected (priv->connected = 255)
[2012-03-14 23:33:43.512104] E [rpcsvc.c:1078:rpcsvc_submit_generic] 0-rpc-service: failed to submit message (XID: 0x23261x, Program: GlusterFS 3.3.0qa27, ProgVers: 330, Proc: 27) to rpc-transport (tcp.dstore1-server)
[2012-03-14 23:33:43.518857] E [server.c:166:server_submit_reply] (-->/usr/local/lib/libglusterfs.so.0(default_finodelk_cbk+0x14d) [0x7f679d4c97ca] (-->/usr/local/lib/glusterfs/3.3.0qa27/xlator/debug/io-stats.so(io_stats_finodelk_cbk+0x23a) [0x7f6793df2f2b] (-->/usr/local/lib/glusterfs/3.3.0qa27/xlator/protocol/server.so(server_finodelk_cbk+0x25e) [0x7f6793bc4f6a]))) 0-: Reply submission failed


Version-Release number of selected component (if applicable):
3.3.0qa27

How reproducible:
often

Steps to Reproduce:
1.create a replicate volume (1 X 3)
2.create fuse mounts from client
3.run ping_pong test from fuse mount: ping_pong f13 -rw 100 50 300
4.bring down a brick while ping_pong is running
5.bring back the brick (changes the graph)

Actual results:
ping_pong test hangs on fuse mount when a brick goes down and come back online.
Comment 1 Amar Tumballi 2012-04-17 14:11:51 EDT
Now visiting this bug again, I found that there is no issues in the logs, and server side behavior. This can happen because connection is not established when the reply was about to be sent. The ping-pong hanging issue should be related to ping-timeout not properly done or something else. Surely not an issue on the server side.

Re-open with 'ping-pong tests hang on fuse mount' summary if the behavior persist on latest mountpoint. Moving to 'wontfix' status for now.

Note You need to log in before you can comment on or make changes to this bug.