Bug 1756320

Summary: RHVH 4.4: Failed to register 4.4 host to 4.3 engine
Product: Red Hat Enterprise Virtualization Manager Reporter: cshao <cshao>
Component: ovirt-host-deployAssignee: Yedidyah Bar David <didi>
Status: CLOSED ERRATA QA Contact: cshao <cshao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.4.0CC: cshao, didi, dougsland, lleistne, lsurette, lsvaty, mavital, nlevy, peyu, qiyuan, rdlugyhe, Rhev-m-bugs, sbonazzo, srevivo, tburke, weiwang, yaniwang, ycui, yturgema
Target Milestone: ovirt-4.3.7Keywords: Regression, TestBlocker, ZStream
Target Release: 4.3.7   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-12-12 10:37:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1762752    
Attachments:
Description Flags
register failed
none
engine log
none
RHVH log
none
ovirt-host-deploy log
none
437.log none

Description cshao 2019-09-27 11:36:32 UTC
Created attachment 1620026 [details]
register failed

Description of problem:
RHVH 4.4: Failed to register to engine.

An error has occurred during installation of Host test: Failed to execute stage 'Environment customization': No module named 'rpmUtils'.

Version-Release number of selected component (if applicable):
redhat-virtualization-host-4.4.0-20190926.1.el8_1
vdsm-4.40.0-107.gite72720e.el8ev.x86_64
engine: 4.3.4.3-0.1.el7

How reproducible:
100%

Steps to Reproduce:
1. Install RHVH-4.4-20190926.3-RHVH-x86_64-dvd1.iso via anaconda GUI.
2. Register to engine.
3.

Actual results:
RHVH 4.4: Failed to register to engine.

Expected results:
Register to engine can succeed.

Additional info:

Comment 1 cshao 2019-09-27 11:42:31 UTC
Created attachment 1620028 [details]
engine log

Comment 2 cshao 2019-09-27 11:43:54 UTC
Created attachment 1620029 [details]
RHVH log

Comment 6 Douglas Schilling Landgraf 2019-09-30 17:51:37 UTC
It seems related to otopi, not register.py from vdsm. Sending to the proper component.

Comment 7 Yedidyah Bar David 2019-10-06 12:23:08 UTC
Isn't this a duplicate of bug 1724056? Can you check what version of ovirt-host-deploy you have? Probably the one from 4.3, which is 1.8, based on your engine. I do not think we are going to support adding an EL8 (or Fedora) host to a 4.3 engine, so should be safe to close as duplicate.

Comment 8 cshao 2019-10-07 00:05:16 UTC
(In reply to Yedidyah Bar David from comment #7)
> Isn't this a duplicate of bug 1724056? Can you check what version of
> ovirt-host-deploy you have? Probably the one from 4.3, which is 1.8, based
> on your engine. I do not think we are going to support adding an EL8 (or
> Fedora) host to a 4.3 engine, so should be safe to close as duplicate.

# rpm -qa | grep ovirt-host-deploy
ovirt-host-deploy-java-1.9.0-0.0.master.20190905084200.git267f406.el7ev.noarch
python2-ovirt-host-deploy-1.9.0-0.0.master.20190905084200.git267f406.el7ev.noarch
ovirt-host-deploy-common-1.9.0-0.0.master.20190905084200.git267f406.el7ev.noarch

If we don't support adding an EL8 host to a 4.3 engine, then this is a duplicate of bug 1724056.

Thanks.

Comment 9 Yedidyah Bar David 2019-10-07 06:55:49 UTC
(In reply to cshao from comment #8)
> (In reply to Yedidyah Bar David from comment #7)
> > Isn't this a duplicate of bug 1724056? Can you check what version of
> > ovirt-host-deploy you have? Probably the one from 4.3, which is 1.8, based
> > on your engine. I do not think we are going to support adding an EL8 (or
> > Fedora) host to a 4.3 engine, so should be safe to close as duplicate.
> 
> # rpm -qa | grep ovirt-host-deploy
> ovirt-host-deploy-java-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch
> python2-ovirt-host-deploy-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch
> ovirt-host-deploy-common-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch

OK, that's more interesting. Patch for that bug was merged on July, so should
have been available by 20190905. If you still have the host-deploy logs, please
attach them, from /var/log/ovirt-engine/host-deploy. Thanks.

> 
> If we don't support adding an EL8 host to a 4.3 engine, then this is a
> duplicate of bug 1724056.

I think we don't, but will ask around and keep open for now.

Comment 10 cshao 2019-10-07 12:07:50 UTC
Created attachment 1623113 [details]
ovirt-host-deploy log

Comment 11 cshao 2019-10-07 12:09:08 UTC
(In reply to Yedidyah Bar David from comment #9)
> (In reply to cshao from comment #8)
> > (In reply to Yedidyah Bar David from comment #7)
> > > Isn't this a duplicate of bug 1724056? Can you check what version of
> > > ovirt-host-deploy you have? Probably the one from 4.3, which is 1.8, based
> > > on your engine. I do not think we are going to support adding an EL8 (or
> > > Fedora) host to a 4.3 engine, so should be safe to close as duplicate.
> > 
> > # rpm -qa | grep ovirt-host-deploy
> > ovirt-host-deploy-java-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> > python2-ovirt-host-deploy-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> > ovirt-host-deploy-common-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> 
> OK, that's more interesting. Patch for that bug was merged on July, so should
> have been available by 20190905. If you still have the host-deploy logs,
> please
> attach them, from /var/log/ovirt-engine/host-deploy. Thanks.

Thank you.
Please see attachment "ovirt-host-deploy log" for more info.


> 
> > 
> > If we don't support adding an EL8 host to a 4.3 engine, then this is a
> > duplicate of bug 1724056.
> 
> I think we don't, but will ask around and keep open for now.

Comment 12 Yedidyah Bar David 2019-10-10 06:16:52 UTC
(In reply to cshao from comment #8)
> # rpm -qa | grep ovirt-host-deploy
> ovirt-host-deploy-java-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch
> python2-ovirt-host-deploy-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch
> ovirt-host-deploy-common-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> noarch

But:

(In reply to cshao from comment #10)
> Created attachment 1623113 [details]
> ovirt-host-deploy log

has:

2019-10-07 12:03:38,180+0000 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND       ### Version: ovirt-host-deploy-1.8.0 (ovirt-host-deploy-1.8.0-1.el7ev)

So either there is some mismatch/confusion somewhere, or perhaps you have a stale bundle cached. Please try to remove it, /var/cache/ovirt-engine/ovirt-host-deploy.tar , and try again. Thanks.

Comment 13 Yuval Turgeman 2019-10-10 07:19:47 UTC
Didi, I think that Chen specified the version on the host - shouldn't we check the versions on the engine ?

Comment 14 Yedidyah Bar David 2019-10-10 07:51:15 UTC
(In reply to Yuval Turgeman from comment #13)
> Didi, I think that Chen specified the version on the host - shouldn't we
> check the versions on the engine ?

Of course. Sorry I wasn't clear. The host does not need host-deploy, and if it's installed there, in whatever version, it's irrelevant for host-deploy itself.

It's only installed, these days, because hosted-engine uses some library code from it. Perhaps we should move this code to ovirt-setup-lib and drop host-deploy from the host, to lower the confusion. We'll probably have to do this anyway once host-deploy is fully replaced with ansible.

Comment 15 Yuval Turgeman 2019-10-10 08:02:17 UTC
Thanks, Didi - Chen, can you share the versions of host-deploy on the engine's side ?

Comment 16 cshao 2019-10-10 08:47:33 UTC
(In reply to Yuval Turgeman from comment #15)
> Thanks, Didi - Chen, can you share the versions of host-deploy on the
> engine's side ?

engine 4.3
# rpm -qa| grep host-deploy
python2-ovirt-host-deploy-1.8.0-1.el7ev.noarch
ovirt-host-deploy-common-1.8.0-1.el7ev.noarch
ovirt-host-deploy-java-1.8.0-1.el7ev.noarch

Comment 17 cshao 2019-10-10 08:51:51 UTC
(In reply to Yedidyah Bar David from comment #12)
> (In reply to cshao from comment #8)
> > # rpm -qa | grep ovirt-host-deploy
> > ovirt-host-deploy-java-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> > python2-ovirt-host-deploy-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> > ovirt-host-deploy-common-1.9.0-0.0.master.20190905084200.git267f406.el7ev.
> > noarch
> 
> But:
> 
> (In reply to cshao from comment #10)
> > Created attachment 1623113 [details]
> > ovirt-host-deploy log
> 
> has:
> 
> 2019-10-07 12:03:38,180+0000 DEBUG otopi.plugins.otopi.dialog.machine
> dialog.__logString:204 DIALOG:SEND       ### Version:
> ovirt-host-deploy-1.8.0 (ovirt-host-deploy-1.8.0-1.el7ev)
> 

This version info was from host side.

> So either there is some mismatch/confusion somewhere, or perhaps you have a
> stale bundle cached. Please try to remove it,
> /var/cache/ovirt-engine/ovirt-host-deploy.tar , and try again. Thanks.

Comment 18 Yedidyah Bar David 2019-10-10 08:58:15 UTC
(In reply to cshao from comment #16)
> (In reply to Yuval Turgeman from comment #15)
> > Thanks, Didi - Chen, can you share the versions of host-deploy on the
> > engine's side ?
> 
> engine 4.3
> # rpm -qa| grep host-deploy
> python2-ovirt-host-deploy-1.8.0-1.el7ev.noarch
> ovirt-host-deploy-common-1.8.0-1.el7ev.noarch
> ovirt-host-deploy-java-1.8.0-1.el7ev.noarch

OK. This makes sense. So comment 7 applies. Sandro - do we want/need to support adding 4.4 host to 4.3 engine? Cherry-picking bug 1724056 to 4.3 should be easy, but I have no idea if this is enough.

Comment 19 Sandro Bonazzola 2019-10-15 11:53:14 UTC
(In reply to Yedidyah Bar David from comment #18)

> OK. This makes sense. So comment 7 applies. Sandro - do we want/need to
> support adding 4.4 host to 4.3 engine? Cherry-picking bug 1724056 to 4.3
> should be easy, but I have no idea if this is enough.

Yes, got confirmation that 4.3 engine should be able to add 4.4 hosts.

Comment 20 cshao 2019-10-16 01:22:37 UTC
(In reply to Sandro Bonazzola from comment #19)
> (In reply to Yedidyah Bar David from comment #18)
> 
> > OK. This makes sense. So comment 7 applies. Sandro - do we want/need to
> > support adding 4.4 host to 4.3 engine? Cherry-picking bug 1724056 to 4.3
> > should be easy, but I have no idea if this is enough.
> 
> Yes, got confirmation that 4.3 engine should be able to add 4.4 hosts.

Thanks Sandro for confirmation.

Comment 21 Sandro Bonazzola 2019-10-22 08:24:30 UTC
4.3.7 engine with updated host deploy should allow to register the host.

Comment 22 cshao 2019-11-11 09:58:33 UTC
Register RHVH 4.4 to 4.3.7 engine still got Failed.

Test version:
redhat-virtualization-host-4.4.0-20191107.1.el8_1
ovirt-engine-4.3.7.1-0.1.el7.noarch

Detail info please refer the "437.log"

Comment 23 cshao 2019-11-11 09:59:20 UTC
Created attachment 1634803 [details]
437.log

Comment 24 Yedidyah Bar David 2019-11-12 07:41:25 UTC
The bug in comment 22 affected also 4.4, and was fixed during development without opening a bug in bugzilla. The patch was: https://gerrit.ovirt.org/101425 . The bug is not related to oVirt/RHV version, but to OS version - the code didn't support el8, and the fix adds that. for reference, the error from the log, that this patch fixes, is:

2019-11-11 17:45:26,526+0800 DEBUG otopi.context context._executeMethod:127 Stage customization METHOD otopi.plugins.ovirt_host_deploy.kdump.packages.Plugin._customization
2019-11-11 17:45:26,526+0800 DEBUG otopi.plugins.otopi.dialog.machine dialog.__logString:204 DIALOG:SEND       **%EventStart STAGE customization METHOD otopi.plugins.ovirt_host_deploy.kdump.packages.Plugin._customization (None)
2019-11-11 17:45:26,526+0800 DEBUG otopi.context context._executeMethod:145 method exception
Traceback (most recent call last):
  File "/tmp/ovirt-8sdTGxVM5v/pythonlib/otopi/context.py", line 132, in _executeMethod
    method['method']()
  File "/tmp/ovirt-8sdTGxVM5v/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 222, in _customization
    self._kexec_tools_version_supported()
  File "/tmp/ovirt-8sdTGxVM5v/otopi-plugins/ovirt-host-deploy/kdump/packages.py", line 151, in _kexec_tools_version_supported
    min_hdr = self._create_hdr(*min_version)
TypeError: _create_hdr() argument after * must be an iterable, not NoneType

It should be fixed in ovirt-host-deploy-1.8.4.

Comment 26 cshao 2019-11-14 12:33:54 UTC
I will verify this bug after upgrade ovirt-host-deploy to 1.8.4.

Comment 27 cshao 2019-11-18 07:48:12 UTC
Test version:
ovirt-host-deploy-1.8.4
rhvh-4.4.0.8-0.20191107.0+1

# rpm -qa| grep ovirt-host-deploy
python2-ovirt-host-deploy-1.8.4-1.el7ev.noarch
ovirt-host-deploy-java-1.8.4-1.el7ev.noarch
ovirt-host-deploy-common-1.8.4-1.el7ev.noarch


Test result:
Register RHVH 4.4 host to 4.3 engine can successful. So the bug is fixed, change bug status to VERIFIED.

Comment 31 errata-xmlrpc 2019-12-12 10:37:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:4233