Bug 1388563 - [Eventing]: 'VOLUME_REBALANCE' event messages have an incorrect volume name
Summary: [Eventing]: 'VOLUME_REBALANCE' event messages have an incorrect volume name
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: distribute
Version: 3.9
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Nithya Balachandran
QA Contact:
URL:
Whiteboard:
Depends On: 1388010
Blocks: 1386472
TreeView+ depends on / blocked
 
Reported: 2016-10-25 16:27 UTC by Nithya Balachandran
Modified: 2016-12-06 05:59 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.9.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1388010
Environment:
Last Closed: 2016-12-06 05:59:10 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Comment 1 Nithya Balachandran 2016-10-25 16:28:42 UTC
Description of problem:
======================
Have a 4 node cluster with eventing enabled. VOLUME_REBALANCE_COMPLETE and VOLUME_REBALANCE_FAILED event messages have the attribute 'volume' with the first letter missing. This event no longer remains a distinguishable event, as the consumer (in this case, USM) will not be able to act/respond when it receives one with an incorrect volume name.


Version-Release number of selected component (if applicable):
============================================================
3.8.4-2


How reproducible:
=================
Always


Steps to Reproduce:
==================
1. Have a 4 node cluster with eventing enabled.
2. Create a disperse volume 'disp' and attach a 2*2 hot tier
3. Perform a tier_detach

Step3 internally triggers rebalance, prompting an event to be generated. 


Actual results:
===============
tier_detach_force:{u'message': {u'volume': u'isp'}, u'event': u'VOLUME_REBALANCE_FAILED', u'ts': 1476851588, u'nodeid': u'72c4f894-61f7-433e-a546-4ad2d7f0a176'}

tier_detach_start_after_stop:{u'message': {u'volume': u'isp'}, u'event': u'VOLUME_REBALANCE_COMPLETE', u'ts': 1476850670, u'nodeid': u'ed362eb3-421c-4a25-ad0e-82ef157ea328'}

Comment 2 Nithya Balachandran 2016-10-25 16:29:54 UTC
RCA:

Gluster translators do not store the actual volume name anywhere. Each translator appends a specific string to the volume name and stores this value in this->name.
For dht, the suffix is "-dht" so this->name actually contains <volname>-dht.

The event framework requires the actual volume name to be sent. The rebalance code incorrectly used strtok to parse the volume name by using "-dht" as the delimiter. strtok () treats every char in the delim string as a delimiter. So the parsing fails for a volume which contains 'd', 'h', or 't' in its name.

Fix:
The code was rewritten to use strstr instead.

--- Additional comment from Worker Ant on 2016-10-25 07:09:26 EDT ---

COMMIT: http://review.gluster.org/15712 committed in master by Raghavendra G (rgowdapp) 
------
commit f4efbf0e3092bd2181f62be1e1f30f202678c866
Author: N Balachandran <nbalacha>
Date:   Mon Oct 24 13:54:41 2016 +0530

    cluster/dht: Incorrect volname in rebalance events
    
    The rebalance event code was using strtok to parse the
    volume name which is incorrect.
    Reworked the code to get the correct volume name using
    strstr.
    
    Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
    BUG: 1388010
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: http://review.gluster.org/15712
    NetBSD-regression: NetBSD Build System <jenkins.org>
    Smoke: Gluster Build System <jenkins.org>
    Reviewed-by: Raghavendra G <rgowdapp>
    CentOS-regression: Gluster Build System <jenkins.org>

Comment 3 Worker Ant 2016-10-25 16:40:20 UTC
REVIEW: http://review.gluster.org/15725 (cluster/dht: Incorrect volname in rebalance events) posted (#1) for review on release-3.9 by N Balachandran (nbalacha)

Comment 4 Worker Ant 2016-10-26 02:57:14 UTC
COMMIT: http://review.gluster.org/15725 committed in release-3.9 by Pranith Kumar Karampuri (pkarampu) 
------
commit 3c27ddac02366c6e5b015c832d4656f520680ced
Author: N Balachandran <nbalacha>
Date:   Tue Oct 25 22:02:16 2016 +0530

    cluster/dht: Incorrect volname in rebalance events
    
    The rebalance event code was using strtok to parse the
    volume name which is incorrect.
    Reworked the code to get the correct volume name using
    strstr.
    
    > Change-Id: Ib5f3305a34e6bf1ecfef677d87c5aff96bdeb0e6
    > BUG: 1388010
    > Signed-off-by: N Balachandran <nbalacha>
    > Reviewed-on: http://review.gluster.org/15712
    > NetBSD-regression: NetBSD Build System <jenkins.org>
    > Smoke: Gluster Build System <jenkins.org>
    > Reviewed-by: Raghavendra G <rgowdapp>
    > CentOS-regression: Gluster Build System <jenkins.org>
    (cherry picked from commit f4efbf0e3092bd2181f62be1e1f30f202678c866)
    
    Change-Id: Ia22d6a8ce23e9dcab8281b4995277cbb3a196fda
    BUG: 1388563
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: http://review.gluster.org/15725
    Smoke: Gluster Build System <jenkins.org>
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Pranith Kumar Karampuri <pkarampu>

Comment 5 Aravinda VK 2016-10-27 05:22:22 UTC
glusterfs-3.9.0rc2 is released[1] and packages are available for different distributions[2] to test.

[1] http://www.gluster.org/pipermail/maintainers/2016-October/001601.html
[2] http://www.gluster.org/pipermail/maintainers/2016-October/001605.html and http://www.gluster.org/pipermail/maintainers/2016-October/001606.html

Comment 6 Aravinda VK 2016-12-06 05:59:10 UTC
Gluster 3.9 GA is released http://blog.gluster.org/2016/11/announcing-gluster-3-9/


Note You need to log in before you can comment on or make changes to this bug.