Bug 1378300 - Modifications to AFR Events
Summary: Modifications to AFR Events
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate
Version: rhgs-3.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: RHGS 3.2.0
Assignee: Ravishankar N
QA Contact: Sweta Anandpara
Depends On: 1378072 1379028
Blocks: 1351528
TreeView+ depends on / blocked
Reported: 2016-09-22 04:20 UTC by Ravishankar N
Modified: 2017-03-23 05:48 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.8.4-2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1378072
Last Closed: 2017-03-23 05:48:06 UTC

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0486 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.2.0 security, bug fix, and enhancement update 2017-03-23 09:18:45 UTC

Description Ravishankar N 2016-09-22 04:20:40 UTC
+++ This bug was initially created as a clone of Bug #1378072 +++


On Wednesday 21 September 2016 03:53 PM, Ravishankar N wrote:
> On 09/21/2016 03:34 PM, Aravinda wrote:
>> Hi,
>> We have following SPLIT_BRAIN events
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=file type mismatch;gfid=%s;ia_type-%d=%s;ia_type-%d=%s"
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=gfid mismatch. Skipping conservative merge.;file=<gfid:%s>/%s>;count=2;child-%d=%s;gfid-%d=%s;child-%d=%s;gfid-%d=%s",
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=file type mismatch. Skipping conservative merge;file=<gfid:%s>/%s>;count=2;child-%d=%s;type-%d=%s;child-%d=%s;type-%d=%s",
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=file type mismatch;file=<gfid:%s>/%s;count=2;child-%d=%s;type-%d=%s;child-%d=%s;type-%d=%s"
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=gfid mismatch;file=<gfid:%s>/%s;count=2;child-%d=%s;gfid-%d=%s;child-%d=%s;gfid-%d=%s"
>> Message keys are not same even though Split brain type is same. For example,
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=file type mismatch;gfid=%s;ia_type-%d=%s;ia_type-%d=%s"
>> EVENT_AFR_SPLIT_BRAIN, "subvol=%s;msg=file type mismatch;file=<gfid:%s>/%s;count=2;child-%d=%s;type-%d=%s;child-%d=%s;type-%d=%s"
>> We can split these events into two types.(Message included as EVENT name itself, so that no separate msg is required.)
>> EVENT_AFR_SPLIT_BRAIN_FILE_TYPE_MISMATCH, "subvol=%s;file=<gfid:%s>/%s;count=2;child-%d=%s;type-%d=%s;child-%d=%s;type-%d=%s"
>> EVENT_AFR_SPLIT_BRAIN_GFID_MISMATCH, "subvol=%s;file=<gfid:%s>/%s>;count=2;child-%d=%s;gfid-%d=%s;child-%d=%s;gfid-%d=%s"
>> Let me know your thoughts.
> I think it is better to retain one EVENT type for split-brains. Otherwise we need to keep on adding different EVENT types for example when we need to propagate data split-brain or metadata split-brain etc.
> The msg=<message> also gives us a way to specific a more verbose message that is immediately available to the consumer, should they decide to parse it. Also, for all types of split-brains, there has to be a remedial action (i.e. resolving the split-brains) required by the admin. Having to monitor only one EVENT type for all split-brains would make  it easier is what I feel.
If it is one event type, then I would prefer "type" instead of msg. (type can be gfid|file|data|meta etc). Descriptive message is not required since these APIs are consumed programatically, free text and non uniform key names breaks the parsing.
> -Ravi

Comment 2 Atin Mukherjee 2016-09-24 08:11:05 UTC
Upstream mainline patch : http://review.gluster.org/15550
Upstream 3.9 patch : http://review.gluster.org/15565

Comment 3 Ravishankar N 2016-09-26 05:33:13 UTC
Downstream patch: https://code.engineering.redhat.com/gerrit/#/c/85605/

Comment 6 Sweta Anandpara 2016-12-19 09:39:33 UTC
Tested and verified this on the build 3.8.4-8. 

Created different scenarios so that I hit AFR_SPLIT_BRAIN events with respect to all types - data/metadata/gfid/type-mismatch.

The 'type' field which is newly introduced in AFR_SPLIT_BRAIN correctly displays the value. Moving this blanket BZ to verified in 3.2

{u'message': {u'count': u'2', u'subvol': u'ozone-replicate-0', u'gfid-1': u'4ad208ac-7558-45ba-a807-c09f5f7339a5', u'gfid-0': u'373c519b-c93f-4e5f-9203-817a00198ad7', u'child-0': u'ozone-client-0', u'child-1': u'ozone-client-1', u'file': u'<gfid:00000000-0000-0000-0000-000000000001>/a>', u'type': u'gfid'}, u'event': u'AFR_SPLIT_BRAIN', u'ts': 1482139686, u'nodeid': u'8d1aaf3a-059e-41c2-871b-6c7f5c0dd90b'}

{u'message': {u'subvol': u'ozone-replicate-1', u'type': u'metadata', u'file': u'3012b1b1-4db6-4044-8455-d0ecd9f5e33f'}, u'event': u'AFR_SPLIT_BRAIN', u'ts': 1482139671, u'nodeid': u'95c24075-02aa-49c1-a1e4-c7e0775e7128'}

{u'message': {u'subvol': u'ozone-replicate-0', u'type': u'data', u'file': u'd3902718-cf0d-44b9-be72-c1c1a75bdde7'}, u'event': u'AFR_SPLIT_BRAIN', u'ts': 1482139664, u'nodeid': u'8d3ab9e3-c086-4474-9db2-677d5018d055'}

Comment 8 errata-xmlrpc 2017-03-23 05:48:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.