Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1434728

Summary: RHEV-H 6.4 -> 6.6 upgrade fails but eventually it works properly.
Product: Red Hat Enterprise Virtualization Manager Reporter: Olimp Bockowski <obockows>
Component: rhev-hypervisorAssignee: Douglas Schilling Landgraf <dougsland>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.4.5CC: gklein, lsurette, pstehlik, srevivo, ycui, ykaul
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-22 21:14:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Olimp Bockowski 2017-03-22 09:04:29 UTC
Description of problem:

According to RHEV-M: Upgrade failed
When RHEV-M updates hypervisor node, it seems that hypervisor node reboots on its own. Manager loses connectivity and error is displayed. 
However, upgrade is successful and, after the reboot, one can activate the node and it works properly. 

Version-Release number of selected component (if applicable):

RHEV Hypervisor - 6.4 - 20130501.0.el6_4
->
RHEV Hypervisor - 6.6 - 20150603.0.el6ev

How reproducible:
100%

Steps to Reproduce:
1. Using Admin Portal trigger upgrade of RHEV-H 6.4 into RHEV-H 6.6.  
2. Wait 
3. Upgrade failed, please refer to logs for further information.


Actual results:
Failed

Expected results:
Successful 

Additional info:

The exact reason of failed installation message is in engine.log:
 
2016-04-11 11:25:29,911 ERROR [org.ovirt.engine.core.bll.InstallerMessages] (OVirtNodeUpgrade) Installation of Host. Received illegal XML from Host. Message: <BSTRAP component='RHEL_INSTALL'
 status='OK'message='vdsm daemon stopped for upgrade process!'/>: java.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 45; Element type "BSTRAP" must be fo
llowed by either attribute specifications, ">" or "/>".
        at org.ovirt.engine.core.bll.InstallerMessages._internalPostOldXmlFormat(InstallerMessages.java:96) [bll.jar:]
        at org.ovirt.engine.core.bll.InstallerMessages.postOldXmlFormat(InstallerMessages.java:74) [bll.jar:]
        at org.ovirt.engine.core.bll.OVirtNodeUpgrade._threadMain(OVirtNodeUpgrade.java:60) [bll.jar:]
        at org.ovirt.engine.core.bll.OVirtNodeUpgrade.access$000(OVirtNodeUpgrade.java:25) [bll.jar:]
        at org.ovirt.engine.core.bll.OVirtNodeUpgrade$1.run(OVirtNodeUpgrade.java:96) [bll.jar:]
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_95]
Caused by: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 45; Element type "BSTRAP" must be followed by either attribute specifications, ">" or "/>".
        at org.apache.xerces.parsers.DOMParser.parse(DOMParser.java:244)
        at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:285)
        at org.ovirt.engine.core.bll.InstallerMessages._internalPostOldXmlFormat(InstallerMessages.java:94) [bll.jar:]
        ... 5 more

Hypervisor doesn't report it, because actually it is error related to RHEV-M which is not able to understand syntax of XML message hypervisor sends. 
Deeply looking into it I think that below string is the problem:

Received illegal XML from Host. Message: <BSTRAP component='RHEL_INSTALL'
 status='OK'message='vdsm daemon stopped for upgrade process!'/>

there is no space between 'status' and 'message'

Apart from error message I believe installation is successful. 
After update in hypervisor's general tab I can see OS Version: 'RHEV Hypervisor - 6.6 - 20150603.0.el6ev'

Comment 1 Douglas Schilling Landgraf 2017-03-22 21:14:08 UTC
This is an old issue, the workaround:

After upgrade from rhev-hypervisor with this bug, move host to maintain once and
then activate.  The issue shouldn't be present in hosts with vdsm-4.10.2-12.0 or higher.

*** This bug has been marked as a duplicate of bug 920671 ***