Bug 1919984

Summary:	engine-setup failse to deploy the grafana service in an external DWH server
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Miguel Martin <mmartinv>
Component:	ovirt-engine-dwh	Assignee:	Yedidyah Bar David <didi>
Status:	CLOSED ERRATA	QA Contact:	Guilherme Santos <gdeolive>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.4.3	CC:	dfodor, gdeolive, lsurette, mavital, mhicks, mperina, mtessun, sbonazzo, srevivo
Target Milestone:	ovirt-4.4.6
Target Release:	4.4.6
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovirt-engine-dwh-4.4.6.2	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-06-01 13:22:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Integration	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Miguel Martin 2021-01-25 14:05:33 UTC

Description of problem:
engine-setup failse to deploy the grafana service in an external DWH server

Version-Release number of selected component (if applicable):
4.4.3

How reproducible:
In CU environment

Steps to Reproduce:
1. setup the engine in machine A
2. setup the dwh on machine B without grafana
3. run "engine-setup --reconfigure-optional-components" on machine B and configure grafana

Actual results:
The setup fails with the following error:
~~~
Failed to execute stage 'Environment customization': str, bytes or bytearray expected, not NoneType
~~~


Expected results:
The setup finishes successfuly and the grafana service is deployed.

Comment 2 Yedidyah Bar David 2021-01-25 14:16:53 UTC

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-grafana-dwh/pki/apache.py", line 126, in _customization
    self._enrolldata.enroll_cert()
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/remote_engine.py", line 403, in enroll_cert
    self._remote_engine.configure(fqdn=self._engine_fqdn)
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/remote_engine.py", line 144, in configure
    self._style.configure(fqdn=fqdn)
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/remote_engine/remote_engine_root_ssh.py", line 176, in configure
    self._ssh_get_port()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-common/base/remote_engine/remote_engine_root_ssh.py", line 81, in _ssh_get_port
    paramiko.Transport((self._fqdn, port))
  File "/usr/lib/python3.6/site-packages/paramiko/transport.py", line 373, in __init__
    retry_on_signal(lambda: sock.connect((hostname, port)))
  File "/usr/lib/python3.6/site-packages/paramiko/util.py", line 280, in retry_on_signal
    return function()
  File "/usr/lib/python3.6/site-packages/paramiko/transport.py", line 373, in <lambda>
    retry_on_signal(lambda: sock.connect((hostname, port)))
TypeError: str, bytes or bytearray expected, not NoneType

Seems like we call paramiko.Transport((self._fqdn, port)) with self._fqdn=None, because we never get the engine fqdn, because OVESETUP_DWH_CONFIG/remoteEngineConfigured is True from a previous run.

So a workaround might be to edit /etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf and change OVESETUP_DWH_CONFIG/remoteEngineConfigured to 'bool:False' and then retry.

Comment 3 Miguel Martin 2021-01-26 11:03:11 UTC

After reproducing the problem in the lab the actual error is:
~~~
[ ERROR ] Failed to execute stage 'Environment customization': 'NoneType' object has no attribute 'execute_on_engine'
~~~

Within the log file the following errors can be found:
~~~
...
...
...
2021-01-26 05:59:02,536-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.getCredentials:1343 dbenv: {'OVESETUP_DWH_DB/host': 'localhost', 'OVESETUP_DWH_DB/port': '5432', 'OVESETUP_DWH_DB/secured': False, 'OVESETUP_DWH_D
B/securedHostValidation': False, 'OVESETUP_DWH_DB/database': 'ovirt_engine_history', 'OVESETUP_GRAFANA_DB/user': 'ovirt_engine_history_grafana', 'OVESETUP_GRAFANA_DB/password': '**FILTERED**'}
2021-01-26 05:59:02,536-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:234 Database: 'None', Statement: '
                select count(*) as count
                from pg_catalog.pg_tables
                where schemaname = 'public';
            ', args: {}
2021-01-26 05:59:02,537-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:239 Creating own connection
2021-01-26 05:59:02,559-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.getCredentials:1375 database connection failed
Traceback (most recent call last):
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 1373, in getCredentials
    ] = self.isNewDatabase()
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 428, in isNewDatabase
    transaction=False,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 248, in execute
    database=database,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 181, in connect
    sslmode=sslmode,
  File "/usr/lib64/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL:  password authentication failed for user "ovirt_engine_history_grafana"
FATAL:  password authentication failed for user "ovirt_engine_history_grafana"

2021-01-26 05:59:02,561-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:234 Database: 'None', Statement: 'SHOW server_version', args: {}
2021-01-26 05:59:02,561-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.execute:239 Creating own connection
2021-01-26 05:59:02,583-0500 DEBUG otopi.ovirt_engine_setup.engine_common.database database.getCredentials:1382 database version check failed
Traceback (most recent call last):
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 1380, in getCredentials
    ] = self.checkDBMSUpgrade()
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 497, in checkDBMSUpgrade
    database,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 455, in checkServerVersion
    transaction=False,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 248, in execute
    database=database,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/engine_common/database.py", line 181, in connect
    sslmode=sslmode,
  File "/usr/lib64/python3.6/site-packages/psycopg2/__init__.py", line 130, in connect
    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: FATAL:  password authentication failed for user "ovirt_engine_history_grafana"
FATAL:  password authentication failed for user "ovirt_engine_history_grafana"
...
...
...
2021-01-26 05:59:02,624-0500 DEBUG otopi.context context._executeMethod:145 method exception
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine-grafana-dwh/core/config.py", line 200, in _customization_sso
    cmd=cmd,
  File "/usr/share/ovirt-engine/setup/ovirt_engine_setup/remote_engine.py", line 65, in execute_on_engine
    return self._style.execute_on_engine(
AttributeError: 'NoneType' object has no attribute 'execute_on_engine'
2021-01-26 05:59:02,625-0500 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Environment customization': 'NoneType' object has no attribute 'execute_on_engine'
...
...
...
~~~

Using the workaround at #2 seems to work

Comment 5 Guilherme Santos 2021-05-05 15:29:02 UTC

Verified on:
ovirt-engine-dwh-4.4.6.2-1.el8ev.noarch
ovirt-engine-4.4.6.6-0.10.el8ev.noarch

Steps:
description's

Results
grafana successfully deployed in machine B

Comment 9 errata-xmlrpc 2021-06-01 13:22:11 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: RHV Manager security update (ovirt-engine) [ovirt-4.4.6]), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2179

Comment 10 meital avital 2022-08-08 19:58:56 UTC

Due to QE capacity, we are not going to cover this issue in our automation