Bug 1699913 - [Upgrade] Upgrade from 4.2 to 4.3 passed, but the engine failed to get started and UI is unavailable
Summary: [Upgrade] Upgrade from 4.2 to 4.3 passed, but the engine failed to get starte...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.3.0
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ovirt-4.3.3-1
: ---
Assignee: Nobody
QA Contact: Nikolai Sednev
URL:
Whiteboard:
Depends On:
Blocks: 1672859 1686445
TreeView+ depends on / blocked
 
Reported: 2019-04-15 12:29 UTC by Nikolai Sednev
Modified: 2019-05-31 09:06 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-31 09:06:02 UTC
oVirt Team: Virt
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
engine logs (10.87 MB, application/x-xz)
2019-04-15 12:29 UTC, Nikolai Sednev
no flags Details
sosreport from alma04 (10.70 MB, application/x-xz)
2019-04-15 12:34 UTC, Nikolai Sednev
no flags Details
sosreport from alma03 (18.58 MB, application/x-xz)
2019-04-15 12:35 UTC, Nikolai Sednev
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 99456 0 ovirt-engine-4.3 ABANDONED upgrade: enforce we are upgrading from 4.2.8.6 or newer 2020-03-02 19:54:38 UTC

Description Nikolai Sednev 2019-04-15 12:29:15 UTC
Created attachment 1555208 [details]
engine logs

Description of problem:
Upgrade from 4.2 to 4.3 passed, but the engine failed to get started and UI is unavailable, although HE-VM is accessible through ssh.

Version-Release number of selected component (if applicable):
rhvm-appliance-4.2-20190411.1.el7.noarch
ovirt-hosted-engine-ha-2.2.19-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.34-1.el7ev.noarch
Linux 3.10.0-957.10.1.el7.x86_64 #1 SMP Thu Feb 7 07:12:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.6 (Maipo)

In engine log I see:

2019-04-15 15:10:37,913+03 ERROR [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) [] Error during
 initialization: javax.ejb.EJBException: java.lang.IllegalStateException: WFLYEE0042: Failed to construct component in
stance
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:246) [wildfly-ejb3-7.2.0.GA-redha
t-00005.jar:7.2.0.GA-redhat-00005]
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:362) [wildfly-ejb3-7.2.0.GA-redhat-000
05.jar:7.2.0.GA-redhat-00005]
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:144) [wildfly-ejb3-7.2.0.GA-r
edhat-00005.jar:7.2.0.GA-redhat-00005]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509)
        at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScope
ActivationInterceptor.java:72) [weld-ejb.jar:3.0.5.Final-redhat-00001]
        at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInter
ceptor.java:89)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvoc
ationContextInterceptor.java:41) [wildfly-ejb3-7.2.0.GA-redhat-00005.jar:7.2.0.GA-redhat-00005]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.jav
a:47) [wildfly-ejb3-7.2.0.GA-redhat-00005.jar:7.2.0.GA-redhat-00005]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100
) [wildfly-ejb3-7.2.0.GA-redhat-00005.jar:7.2.0.GA-redhat-00005]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422)
        at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.j


How reproducible:
100%

Steps to Reproduce:
1.
Installed Software Version:4.2.8.6-0.1.el7ev engine on pair of hosts and attached one data storage domain over NFS.
2.
I manually set CPU type of host cluster to Intel Conroe Family (the oldest family for the Intel CPU type supported in 4.2).
3.
Upgraded the engine to latest bits.
4.
Try to disable global amintenance and power-off the engine-VM and see that engine's UI won't be reachable even after restarting engine's VM.
Actual results:
Engine UI can't be reached after successful upgrade of the engine.

Expected results:
UI should be reachable.

Additional info:
Logs from engine attached.

Comment 1 Nikolai Sednev 2019-04-15 12:34:10 UTC
Created attachment 1555209 [details]
sosreport from alma04

Comment 2 Nikolai Sednev 2019-04-15 12:35:11 UTC
Created attachment 1555211 [details]
sosreport from alma03

Comment 3 Simone Tiraboschi 2019-04-15 12:40:16 UTC
Caused by: java.lang.RuntimeException: Error during JSON deserialization of org.ovirt.engine.core.dao.VdsStaticDaoImpl.KernelCmdlineColumn
...
Caused by: org.codehaus.jackson.map.exc.UnrecognizedPropertyException: Unrecognized field "smtDisabled" (Class org.ovirt.engine.core.dao.VdsStaticDaoImpl$KernelCmdlineColumn), not marked as ignorable

Moving to virt for further investigation

Comment 4 Michal Skrivanek 2019-04-15 12:55:04 UTC
can you please update to a 4.3.3.4 instead of 4.3.3.3? It seems to be caused by the fact you're upgrading to "old" 4.3 build not having the new patch merged to 4.2.8.6

Comment 5 Simone Tiraboschi 2019-04-15 13:07:09 UTC
So we should probably enforce the right upgrade path within engine-setup.

Comment 12 Nikolai Sednev 2019-04-21 14:41:57 UTC
Works just fine with ovirt-engine-setup-4.3.3.5-0.1.el7.noarch.

Tested upgrade on hosts and engine from 4.2:
ovirt-engine-setup-4.2.8.7-0.1.el7ev.noarch
ovirt-hosted-engine-ha-2.2.19-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.34-1.el7ev.noarch

To 4.3:
ovirt-engine-setup-4.3.3.5-0.1.el7.noarch
ovirt-hosted-engine-ha-2.3.1-1.el7ev.noarch
ovirt-hosted-engine-setup-2.3.7-1.el7ev.noarch

Moving to verified.


Note You need to log in before you can comment on or make changes to this bug.