Bug 1527615

Summary: tendrl-monitoring-integration service fails to start due to python import error of cliutils
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Daniel Messer <dmesser>
Component: web-admin-tendrl-monitoring-integrationAssignee: Shubhendu Tripathi <shtripat>
Status: CLOSED DUPLICATE QA Contact: sds-qe-bugs
Severity: high Docs Contact:
Priority: unspecified    
Version: rhgs-3.3CC: dmesser, mbukatov, rhs-bugs, shtripat
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-20 09:39:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
sosreport from system RHGS1 where tendrl-server should be deployed none

Description Daniel Messer 2017-12-19 16:21:03 UTC 
Created attachment 1370076 [details]
sosreport from system RHGS1 where tendrl-server should be deployed

Description of problem:

A RHEL 7.4 server with all current packages and the RHGS Web Admin repo enabled fails to run the monitoring-integration service. The service fails to start after completing the tendrl-server role while doing a systemctl start tendrl-monitoring-integration.service:

Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: Load definitions (.yml) for namespace.tendrl.objects.NodeContext
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: Load definitions (.yml) for namespace.tendrl.objects.TendrlContext
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: Traceback (most recent call last):
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/bin/tendrl-monitoring-integration", line 9, in <module>
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: load_entry_point('tendrl-monitoring-integration==1.5.4', 'console_scripts', 'tendrl-moni
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib/python2.7/site-packages/tendrl/monitoring_integration/manager/__init__.py
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: monitoring_integration_manager = MonitoringIntegrationManager()
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib/python2.7/site-packages/tendrl/monitoring_integration/manager/__init__.py
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: self.webhook_receiver = webhook_receiver.WebhookReceiver()
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib/python2.7/site-packages/tendrl/monitoring_integration/grafana/webhook_rec
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: self.alert_handler = AlertHandlerManager()
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib/python2.7/site-packages/tendrl/monitoring_integration/alert/handlers/__in
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: self._load_handlers()
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib/python2.7/site-packages/tendrl/monitoring_integration/alert/handlers/__in
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: mod = importlib.import_module(handler_fqdn)
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: __import__(name)
Dec 19 16:05:01 RHGS1 tendrl-monitoring-integration[4852]: ImportError: No module named cliutils.cliutils

Version-Release number of selected component (if applicable):

tendrl-monitoring-integration-1.5.4-14.el7rhgs.noarch

How reproducible:

Always


Steps to Reproduce:
1. Configure a 3 node RHGS cluster
2. Install tendrl bits from internal RHT repos (http:////pulp.dist.prod.ext.phx2.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhgs-webadmin/3.1/os/)
3. configure inventory / site.yml
4. run playbook

Actual results:

Playbook fails at running handler to restart tendrl-monitoring-service

Expected results:

Playbook succeeds

Additional info:

RHEL is updated from pulp.dist.prod.ext.phx2.redhat.com as well.

sosreport from the system attached.
Comment 2 Shubhendu Tripathi 2017-12-20 05:53:56 UTC 
Daniel,

Looking at `/etc/yum.repos.d` contents in attached SOS report, I see below repo files

rh-gluster-3-web-admin-server-for-rhel-7-server-rpms.repo
rh-gluster-3-web-admin-agent-for-rhel-7-server-rpms.repo
rh-gluster-3-samba-for-rhel-7-server-rpms.repo
rh-gluster-3-nfs-for-rhel-7-server-rpms.repo
rh-gluster-3-for-rhel-7-server-rpms.repo
rhel-ha-for-rhel-7-server-rpms.repo

I understand with this, that gluster bits as well are installed on the tendrl server node. Tendrl server colocation with gluster nodes is not supported and you need a separate node for tendrl server.

If you have sat a 3 node gluster cluster, you need a 4th node for tendrl server to get installed.

With suggested scenario it should work fine.
Comment 3 Shubhendu Tripathi 2017-12-20 05:57:10 UTC 
Daniel,

Below QSG and Admin guide would be helpful

https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/quick_start_guide/

https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.3/html-single/monitoring_guide/
Comment 4 Daniel Messer 2017-12-20 09:21:52 UTC 
I can confirm that with a separate node for tendrl-server it works. We however should not depend on this separation. Will file an RFE. Thanks for the hint. You can close this.
Comment 5 Shubhendu Tripathi 2017-12-20 09:39:41 UTC 
Based on this I would close this as NOT_A_BUG. Thanks
Comment 6 Martin Bukatovic 2018-06-18 12:10:08 UTC 
Marking as a duplicate of BZ 1572216.

This BZ was reported earlier, but it was closed as not a bug. On the other hand
the other BZ is already verified.

*** This bug has been marked as a duplicate of bug 1572216 ***