Bug 1600459
Summary: | Error handling in /usr/sbin/gluster-eventsapi produces IndexError: tuple index out of range | |||
---|---|---|---|---|
Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Daniel Horák <dahorak> | |
Component: | eventsapi | Assignee: | Srijan Sivakumar <ssivakum> | |
Status: | CLOSED ERRATA | QA Contact: | Leela Venkaiah Gangavarapu <lgangava> | |
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | rhgs-3.4 | CC: | puebele, rhs-bugs, rkothiya, sheggodu, ssivakum, sunkumar | |
Target Milestone: | --- | Keywords: | EasyFix, Rebase, ZStream | |
Target Release: | RHGS 3.5.z Batch Update 4 | Flags: | lgangava:
needinfo-
|
|
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | glusterfs-6.0-52 | Doc Type: | No Doc Update | |
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1685027 (view as bug list) | Environment: | ||
Last Closed: | 2021-04-29 07:20:37 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1685027, 1687248, 1687249 |
Description
Daniel Horák
2018-07-12 10:09:49 UTC
Possible Reproduction scenario might be, to remove (rename) /var/lib/glusterd/events/ directory on one Gluster Storage Node and try to add webhook from another storage node: On Gluster node 5: # mv /var/lib/glusterd/events/ /var/lib/glusterd/events_BACKUP On Gluster node 1: # gluster-eventsapi webhook-add http://0.0.0.0:8697/test Traceback (most recent call last): File "/usr/sbin/gluster-eventsapi", line 666, in <module> runcli() File "/usr/lib/python2.7/site-packages/gluster/cliutils/cliutils.py", line 224, in runcli cls.run(args) File "/usr/sbin/gluster-eventsapi", line 329, in run sync_to_peers(args) File "/usr/sbin/gluster-eventsapi", line 177, in sync_to_peers "{1}".format(e[0], e[2]), IndexError: tuple index out of range Patch sent to upstream to fix the issue https://review.gluster.org/#/c/glusterfs/+/22294/ Hi, Proceeding with the same reproducer the fix is not working as expected: 1. Remove /var/lib/glusterd/events folder from one of the nodes 2. Perform `gluster-eventsapi webhook-add http://0.0.0.0:8697/test` on any node expect above node Actual: # gluster-eventsapi webhook-add http://0.0.0.0:8697/test Traceback (most recent call last): File "/usr/sbin/gluster-eventsapi", line 174, in sync_to_peers sync_file_to_peers(WEBHOOKS_FILE_TO_SYNC) File "/usr/lib/python3.6/site-packages/gluster/cliutils/cliutils.py", line 172, in sync_file_to_peers raise GlusterCmdException((rc, out, err)) gluster.cliutils.cliutils.GlusterCmdException: (1, '', 'Commit failed on 10.70.46.154. Error: Unable to open /var/lib/glusterd//events/webhooks.json\n') During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/sbin/gluster-eventsapi", line 670, in <module> runcli() File "/usr/lib/python3.6/site-packages/gluster/cliutils/cliutils.py", line 225, in runcli cls.run(args) File "/usr/sbin/gluster-eventsapi", line 333, in run sync_to_peers(args) File "/usr/sbin/gluster-eventsapi", line 177, in sync_to_peers errmsg = e.message[2] if e.message[2] else e.message[1] AttributeError: 'GlusterCmdException' object has no attribute 'message' Expected: Correct error message Additional Info: # rpm -qa glusterfs-events glusterfs-events-6.0-51.el8rhgs.x86_64 # /usr/libexec/platform-python -V Python 3.6.8 From pdb, there's is no attr `message`: # gluster-eventsapi webhook-add http://0.0.0.0:8697/test3 > /usr/sbin/gluster-eventsapi(179)sync_to_peers() -> errmsg = e.message[2] if e.message[2] else e.message[1] (Pdb) e GlusterCmdException((1, '', 'Commit failed on 10.70.46.154. Error: Unable to open /var/lib/glusterd//events/webhooks.json\n'),) (Pdb) dir(e) ['__cause__', '__class__', '__context__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__suppress_context__', '__traceback__', '__weakref__', 'args', 'with_traceback'] (Pdb) c Traceback (most recent call last): ... [Above Traceback] AttributeError: 'GlusterCmdException' object has no attribute 'message' Possible fix: Replace `message` with `args[0]` in [0] and [1] [0] https://github.com/gluster/glusterfs/blob/devel/events/src/peer_eventsapi.py#L175-L181 [1] https://github.com/gluster/glusterfs/blob/devel/events/src/peer_eventsapi.py#L188-L192 With above fix: # gluster-eventsapi webhook-add http://0.0.0.0:8697/test6 Failed to sync Webhooks file: [Error: 1]Commit failed on 10.70.46.154. Error: Unable to open /var/lib/glusterd//events/webhooks.json Another obeservation: ------------------ Even the webhook-add operation failed, status states the creation of webhooks (below shouldn't have registered afaik) # gluster-eventsapi status Webhooks: http://0.0.0.0:8697/test http://0.0.0.0:8697/test2 http://0.0.0.0:8697/test3 http://0.0.0.0:8697/test4 http://0.0.0.0:8697/test5 http://0.0.0.0:8697/test6 Thanks, Leela. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (glusterfs bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1462 |