Bug 1387510 - Getting continuous warning and error messages when one of the replica brick goes down during rebalance
Summary: Getting continuous warning and error messages when one of the replica brick g...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate
Version: rhgs-3.1
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: ---
Assignee: Pranith Kumar K
QA Contact: Nag Pavan Chilakam
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-10-21 07:06 UTC by Byreddy
Modified: 2016-11-17 06:06 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-17 06:06:16 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Byreddy 2016-10-21 07:06:03 UTC
Description of problem:
=======================
Getting warning and error messages when one of the replica pair brick goes down during rebalance operation.


[2016-10-21 06:21:24.506578] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.506650] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169755: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]
[2016-10-21 06:21:24.529369] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.529450] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169776: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]
[2016-10-21 06:21:24.593487] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.593557] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169807: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]
[2016-10-21 06:21:24.614531] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.614596] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169822: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]
[2016-10-21 06:21:24.650385] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.650500] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169833: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]
[2016-10-21 06:21:24.666088] W [MSGID: 115009] [server-resolve.c:543:server_resolve] 0-Dis-Rep-server: no resolution type for (null) (LOOKUP)
[2016-10-21 06:21:24.666153] E [MSGID: 115050] [server-rpc-fops.c:159:server_lookup_cbk] 0-Dis-Rep-server: 169849: LOOKUP (null) (00000000-0000-0000-0000-000000000000) ==> (Invalid argument) [Invalid argument]




Version-Release number of selected component (if applicable):
=============================================================
glusterfs-3.7.1-11.el7rhgs.5.sfdc01636297 ( RHGS 3.1 )


How reproducible:
=================
Always


Steps to Reproduce:
===================
1. Have 3 cluster nodes having above mentioned bits
2. Have  2 * 2 volume  //Distributed-replicate volume
3. Fuse mount the volume on two diff clients
4. Write enough data from both the clients to take rebalance operation more time in the below steps
5. Add two new bricks
6. Trigger the rebalance  // gluster volume rebalance <vol-name> start
7. During rebalance in progress, down one of the replica pair brick ( kill -15 <pidof brick> ) // Do it on one of old the replica pair of bricks.
8. Check the brick logs of the other brick part of replica pair 


Actual results:
===============
Getting continuous warning and error messages when one of the replica brick goes down during rebalance


Expected results:
=================
No Warning and error messages should throw while rebalance in progress.

Additional info:
=================
Once rebalance is complete, stopped getting these messages in the the brick log.

Comment 3 Byreddy 2016-10-21 07:12:56 UTC
I think we get these warning and error messages in the current 3.8.4-2 build as well based on discussion with dev team so setting the Internal White board to 3.2.0

Comment 6 Byreddy 2016-11-17 06:06:16 UTC
This issue consistently reproducible only in the the hot fix and not reproducing in RHGS3.2 current build 

Closing this bug as works in the current release.


Note You need to log in before you can comment on or make changes to this bug.