Bug 1388010

Summary: [Eventing]: 'VOLUME_REBALANCE' event messages have an incorrect volume name
Product: [Community] GlusterFS Reporter: Nithya Balachandran <nbalacha>
Component: distributeAssignee: Nithya Balachandran <nbalacha>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: mainlineCC: amukherj, bugs, rhinduja, sanandpa, storage-qa-internal, vbellur
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.10.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1386472
: 1388563 (view as bug list) Environment:
Last Closed: 2017-03-06 17:30:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1386472, 1388563    

Comment 1 Nithya Balachandran 2016-10-24 08:28:26 UTC
Description of problem:
======================
Have a 4 node cluster with eventing enabled. VOLUME_REBALANCE_COMPLETE and VOLUME_REBALANCE_FAILED event messages have the attribute 'volume' with the first letter missing. This event no longer remains a distinguishable event, as the consumer (in this case, USM) will not be able to act/respond when it receives one with an incorrect volume name.


Version-Release number of selected component (if applicable):
============================================================
3.8.4-2


How reproducible:
=================
Always


Steps to Reproduce:
==================
1. Have a 4 node cluster with eventing enabled.
2. Create a disperse volume 'disp' and attach a 2*2 hot tier
3. Perform a tier_detach

Step3 internally triggers rebalance, prompting an event to be generated. 


Actual results:
===============
tier_detach_force:{u'message': {u'volume': u'isp'}, u'event': u'VOLUME_REBALANCE_FAILED', u'ts': 1476851588, u'nodeid': u'72c4f894-61f7-433e-a546-4ad2d7f0a176'}

tier_detach_start_after_stop:{u'message': {u'volume': u'isp'}, u'event': u'VOLUME_REBALANCE_COMPLETE', u'ts': 1476850670, u'nodeid': u'ed362eb3-421c-4a25-ad0e-82ef157ea328'}

Comment 2 Worker Ant 2016-10-24 08:30:05 UTC
REVIEW: http://review.gluster.org/15712 (cluster/dht: Incorrect volname in rebalance events) posted (#1) for review on master by N Balachandran (nbalacha)

Comment 3 Nithya Balachandran 2016-10-25 10:31:47 UTC
RCA:

Gluster translators do not store the actual volume name anywhere. Each translator appends a specific string to the volume name and stores this value in this->name.
For dht, the suffix is "-dht" so this->name actually contains <volname>-dht.

The event framework requires the actual volume name to be sent. The rebalance code incorrectly used strtok to parse the volume name by using "-dht" as the delimiter. strtok () treats every char in the delim string as a delimiter. So the parsing fails for a volume which contains 'd', 'h', or 't' in its name.

Fix:
The code was rewritten to use strstr instead.

Comment 4 Worker Ant 2016-10-25 11:09:26 UTC
COMMIT: http://review.gluster.org/15712 committed in master by Raghavendra G (rgowdapp) 
------
commit f4efbf0e3092bd2181f62be1e1f30f202678c866
Author: N Balachandran <nbalacha>
Date:   Mon Oct 24 13:54:41 2016 +0530

    cluster/dht: Incorrect volname in rebalance events
    
    The rebalance event code was using strtok to parse the
    volume name which is incorrect.
    Reworked the code to get the correct volume name using
    strstr.
    
    Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
    BUG: 1388010
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: http://review.gluster.org/15712
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Raghavendra G <rgowdapp>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 5 Shyamsundar 2017-03-06 17:30:52 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.10.0, please open a new bug report.

glusterfs-3.10.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://lists.gluster.org/pipermail/gluster-users/2017-February/030119.html
[2] https://www.gluster.org/pipermail/gluster-users/