Bug 1378030 - glusterd fails to start without installing glusterfs-events package
Summary: glusterd fails to start without installing glusterfs-events package
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterd
Version: rhgs-3.2
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: RHGS 3.2.0
Assignee: Aravinda VK
QA Contact: Byreddy
URL:
Whiteboard:
Depends On:
Blocks: 1351528 1378057 1380252
TreeView+ depends on / blocked
 
Reported: 2016-09-21 11:07 UTC by SATHEESARAN
Modified: 2017-03-23 05:47 UTC (History)
8 users (show)

Fixed In Version: glusterfs-3.8.4-2
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1378057 (view as bug list)
Environment:
Last Closed: 2017-03-23 05:47:49 UTC


Attachments (Terms of Use)
glusterd log file with debug loglevel enabled (6.86 KB, text/plain)
2016-09-21 11:15 UTC, SATHEESARAN
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:0486 normal SHIPPED_LIVE Moderate: Red Hat Gluster Storage 3.2.0 security, bug fix, and enhancement update 2017-03-23 09:18:45 UTC

Description SATHEESARAN 2016-09-21 11:07:16 UTC
Description of problem:
-----------------------
I installed glusterfs-server-3.8.4-1.el7rhgd and glusterd failed to start.
After install glusterfs-events package, glusterd started normally

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
RHGS 3.2.0 ( interim build - glusterfs-3.8.4-1.el7rhgs )

How reproducible:
-----------------
Always

Steps to Reproduce:
-------------------
1. Install glusterfs-server package
2. Start glusterd

Actual results:
---------------
glusterd failed to start

Expected results:
-----------------
glusterfs-server package should pull glusterfs-events package as as dependency


Additional info:

Comment 1 SATHEESARAN 2016-09-21 11:13:42 UTC
Error from glusterd.log

<snip>
[2016-09-21 09:59:42.606761] D [MSGID: 0] [glusterd-utils.c:6373:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-09-21 09:59:42.606776] D [MSGID: 0] [glusterd.c:1721:init] 0-management: cannot get run-with-valgrind value
[2016-09-21 09:59:42.665672] E [MSGID: 106229] [glusterd.c:456:glusterd_check_gsync_present] 0-glusterd: geo-replication module not working as desired
[2016-09-21 09:59:42.665784] D [MSGID: 0] [glusterd.c:466:glusterd_check_gsync_present] 0-glusterd: Returning -1
[2016-09-21 09:59:42.665819] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2016-09-21 09:59:42.665833] E [MSGID: 101066] [graph.c:324:glusterfs_graph_init] 0-management: initializing translator failed
[2016-09-21 09:59:42.665845] E [MSGID: 101176] [graph.c:673:glusterfs_graph_activate] 0-graph: init failed
[2016-09-21 09:59:42.666266] D [logging.c:1765:gf_log_flush_extra_msgs] 0-logging-infra: Log buffer size reduced. About to flush 5 extra log messages
[2016-09-21 09:59:42.666289] D [logging.c:1768:gf_log_flush_extra_msgs] 0-logging-infra: Just flushed 5 extra log messages
[2016-09-21 09:59:42.666501] W [glusterfsd.c:1288:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x7f8b772d169d] -->/usr/sbin/glusterd(glusterfs_process_volfp+0x172) [0x7f8b772d1542] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x7f8b772d0abb] ) 0-: received signum (1), shutting down
</snip>

Comment 2 SATHEESARAN 2016-09-21 11:15:12 UTC
Created attachment 1203219 [details]
glusterd log file with debug loglevel enabled

Comment 3 SATHEESARAN 2016-09-21 11:18:14 UTC
One more issue seen in a scenario, where 'glusterd' is not running and 'glusterfs-events' package is not installed.

I see a traceback in the following scenario :

# gluster pe s
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 29, in <module>
    from syncdutils import FreeObject, norm, grabpidfile, finalize
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 28, in <module>
    from events import eventtypes
ImportError: No module named events
peer status: failed
Connection failed. Please check if gluster daemon is operational.

Comment 4 SATHEESARAN 2016-09-21 11:20:51 UTC
As a workaround to this, one need to install glusterfs-events package along with python-gluster package.

Comment 5 Aravinda VK 2016-09-21 11:33:08 UTC
RCA:

Geo-rep/Snap scheduler imports eventtypes.py but that file is packaged under glusterfs-events rpm. During glusterd/gluster cli invocation, call to `/usr/libexec/glusterfs/gsyncd --version` is failing with Python traceback.

To fix the issue, conditionally enable flag in Geo-rep code if events is not installed by guarding the import.

EVENTS_ENABLED = True
try:
    from events import eventtypes
except ImportError:
    EVENTS_ENABLED = False

And check this flag while calling gf_event.

Comment 6 SATHEESARAN 2016-09-21 11:46:19 UTC
I have raised this bug with the understanding that glusterfs-events package is required for glusterfs-server package. That is the reason in comment0, I expected that glusterfs-server package to glusterfs-events.

Later understood the fact that the glusterfs-events will be installed only if there is a requirement.

So the expected result is that glusterd should start without installing glusterfs-events package

Comment 7 Aravinda VK 2016-09-21 12:32:11 UTC
Posted the patch to upstream to fix the issue
http://review.gluster.org/15539

Comment 9 Aravinda VK 2016-09-29 06:33:12 UTC
Downstream patch sent
https://code.engineering.redhat.com/gerrit/85934

Comment 10 Atin Mukherjee 2016-09-29 11:05:36 UTC
upstream mainline patch: http://review.gluster.org/15539
upstream 3.9 patch : http://review.gluster.org/#/c/15594
downstream patch : https://code.engineering.redhat.com/gerrit/85934

All the patches are merged now.

Comment 12 Byreddy 2016-09-30 04:32:49 UTC
Verified this issue using the Build : glusterfs-3.8.4-2.

With out installing the glusterfs-events package, able to start the glusterd with out any issue.


Before Update, Package Version:
===============================
~]# rpm -qa |grep gluster
glusterfs-3.7.9-12.el7rhgs.x86_64
glusterfs-server-3.7.9-12.el7rhgs.x86_64
gluster-nagios-addons-0.2.7-1.el7rhgs.x86_64
glusterfs-client-xlators-3.7.9-12.el7rhgs.x86_64
glusterfs-geo-replication-3.7.9-12.el7rhgs.x86_64
python-gluster-3.7.9-12.el7rhgs.noarch
glusterfs-api-3.7.9-12.el7rhgs.x86_64
vdsm-gluster-4.17.33-1.el7rhgs.noarch
glusterfs-libs-3.7.9-12.el7rhgs.x86_64
glusterfs-cli-3.7.9-12.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-fuse-3.7.9-12.el7rhgs.x86_64


After Update, Package Version:
==============================
~]# rpm -qa |grep gluster
glusterfs-geo-replication-3.8.4-2.el7rhgs.x86_64
gluster-nagios-addons-0.2.7-1.el7rhgs.x86_64
glusterfs-fuse-3.8.4-2.el7rhgs.x86_64
glusterfs-3.8.4-2.el7rhgs.x86_64
glusterfs-cli-3.8.4-2.el7rhgs.x86_64
vdsm-gluster-4.17.33-1.el7rhgs.noarch
glusterfs-client-xlators-3.8.4-2.el7rhgs.x86_64
glusterfs-server-3.8.4-2.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-api-3.8.4-2.el7rhgs.x86_64
glusterfs-libs-3.8.4-2.el7rhgs.x86_64
python-gluster-3.8.4-2.el7rhgs.noarch

"Glusterd Status:"
==================
~]# systemctl status   glusterd
● glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2016-09-30 09:51:10 IST; 5s ago
  Process: 19043 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 19044 (glusterd)
   CGroup: /system.slice/glusterd.service
           └─19044 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO

Sep 30 09:51:05 dhcp43-238.lab.eng.blr.redhat.com systemd[1]: Starting GlusterFS, a clustered file-system server...
Sep 30 09:51:10 dhcp43-238.lab.eng.blr.redhat.com systemd[1]: Started GlusterFS, a clustered file-system server.


Moving to verified state.

Comment 14 errata-xmlrpc 2017-03-23 05:47:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0486.html


Note You need to log in before you can comment on or make changes to this bug.