Bug 1378030

Error from glusterd.log

<snip>
[2016-09-21 09:59:42.606761] D [MSGID: 0] [glusterd-utils.c:6373:glusterd_sm_tr_log_init] 0-glusterd: returning 0
[2016-09-21 09:59:42.606776] D [MSGID: 0] [glusterd.c:1721:init] 0-management: cannot get run-with-valgrind value
[2016-09-21 09:59:42.665672] E [MSGID: 106229] [glusterd.c:456:glusterd_check_gsync_present] 0-glusterd: geo-replication module not working as desired
[2016-09-21 09:59:42.665784] D [MSGID: 0] [glusterd.c:466:glusterd_check_gsync_present] 0-glusterd: Returning -1
[2016-09-21 09:59:42.665819] E [MSGID: 101019] [xlator.c:433:xlator_init] 0-management: Initialization of volume 'management' failed, review your volfile again
[2016-09-21 09:59:42.665833] E [MSGID: 101066] [graph.c:324:glusterfs_graph_init] 0-management: initializing translator failed
[2016-09-21 09:59:42.665845] E [MSGID: 101176] [graph.c:673:glusterfs_graph_activate] 0-graph: init failed
[2016-09-21 09:59:42.666266] D [logging.c:1765:gf_log_flush_extra_msgs] 0-logging-infra: Log buffer size reduced. About to flush 5 extra log messages
[2016-09-21 09:59:42.666289] D [logging.c:1768:gf_log_flush_extra_msgs] 0-logging-infra: Just flushed 5 extra log messages
[2016-09-21 09:59:42.666501] W [glusterfsd.c:1288:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xfd) [0x7f8b772d169d] -->/usr/sbin/glusterd(glusterfs_process_volfp+0x172) [0x7f8b772d1542] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x7f8b772d0abb] ) 0-: received signum (1), shutting down
</snip>

Created attachment 1203219 [details]
glusterd log file with debug loglevel enabled

One more issue seen in a scenario, where 'glusterd' is not running and 'glusterfs-events' package is not installed.

I see a traceback in the following scenario :

# gluster pe s
Traceback (most recent call last):
  File "/usr/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 29, in <module>
    from syncdutils import FreeObject, norm, grabpidfile, finalize
  File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 28, in <module>
    from events import eventtypes
ImportError: No module named events
peer status: failed
Connection failed. Please check if gluster daemon is operational.

As a workaround to this, one need to install glusterfs-events package along with python-gluster package.

RCA:

Geo-rep/Snap scheduler imports eventtypes.py but that file is packaged under glusterfs-events rpm. During glusterd/gluster cli invocation, call to `/usr/libexec/glusterfs/gsyncd --version` is failing with Python traceback.

To fix the issue, conditionally enable flag in Geo-rep code if events is not installed by guarding the import.

EVENTS_ENABLED = True
try:
    from events import eventtypes
except ImportError:
    EVENTS_ENABLED = False

And check this flag while calling gf_event.

I have raised this bug with the understanding that glusterfs-events package is required for glusterfs-server package. That is the reason in comment0, I expected that glusterfs-server package to glusterfs-events.

Later understood the fact that the glusterfs-events will be installed only if there is a requirement.

So the expected result is that glusterd should start without installing glusterfs-events package

Posted the patch to upstream to fix the issue
http://review.gluster.org/15539

Downstream patch sent
https://code.engineering.redhat.com/gerrit/85934

upstream mainline patch: http://review.gluster.org/15539
upstream 3.9 patch : http://review.gluster.org/#/c/15594
downstream patch : https://code.engineering.redhat.com/gerrit/85934

All the patches are merged now.

Verified this issue using the Build : glusterfs-3.8.4-2.

With out installing the glusterfs-events package, able to start the glusterd with out any issue.


Before Update, Package Version:
===============================
~]# rpm -qa |grep gluster
glusterfs-3.7.9-12.el7rhgs.x86_64
glusterfs-server-3.7.9-12.el7rhgs.x86_64
gluster-nagios-addons-0.2.7-1.el7rhgs.x86_64
glusterfs-client-xlators-3.7.9-12.el7rhgs.x86_64
glusterfs-geo-replication-3.7.9-12.el7rhgs.x86_64
python-gluster-3.7.9-12.el7rhgs.noarch
glusterfs-api-3.7.9-12.el7rhgs.x86_64
vdsm-gluster-4.17.33-1.el7rhgs.noarch
glusterfs-libs-3.7.9-12.el7rhgs.x86_64
glusterfs-cli-3.7.9-12.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-fuse-3.7.9-12.el7rhgs.x86_64


After Update, Package Version:
==============================
~]# rpm -qa |grep gluster
glusterfs-geo-replication-3.8.4-2.el7rhgs.x86_64
gluster-nagios-addons-0.2.7-1.el7rhgs.x86_64
glusterfs-fuse-3.8.4-2.el7rhgs.x86_64
glusterfs-3.8.4-2.el7rhgs.x86_64
glusterfs-cli-3.8.4-2.el7rhgs.x86_64
vdsm-gluster-4.17.33-1.el7rhgs.noarch
glusterfs-client-xlators-3.8.4-2.el7rhgs.x86_64
glusterfs-server-3.8.4-2.el7rhgs.x86_64
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-api-3.8.4-2.el7rhgs.x86_64
glusterfs-libs-3.8.4-2.el7rhgs.x86_64
python-gluster-3.8.4-2.el7rhgs.noarch

"Glusterd Status:"
==================
~]# systemctl status   glusterd
● glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2016-09-30 09:51:10 IST; 5s ago
  Process: 19043 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 19044 (glusterd)
   CGroup: /system.slice/glusterd.service
           └─19044 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO

Sep 30 09:51:05 dhcp43-238.lab.eng.blr.redhat.com systemd[1]: Starting GlusterFS, a clustered file-system server...
Sep 30 09:51:10 dhcp43-238.lab.eng.blr.redhat.com systemd[1]: Started GlusterFS, a clustered file-system server.


Moving to verified state.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2017-0486.html