Description of problem: The first hosted-engine host fails to go up at engine eyes due to: 2017-12-18 22:57:18,019-05 INFO [org.ovirt.engine.core.bll.AddUnmanagedVmsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [149fd43] Running command: AddUnmanagedVmsCommand internal: true. 2017-12-18 22:57:18,021-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [149fd43] START, FullListVDSCommand(HostName = dhcp-8-176.nay.redhat.com, FullListVDSCommandParameters:{hostId='c316efb7-b613-406b-b3b7-c6893e9fcbf3', vmIds='[10d65864-1721-4eec-a2c5-d6344467d111]'}), log id: 4ba3e3cc 2017-12-18 22:57:19,027-05 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [149fd43] FINISH, FullListVDSCommand, return: [{xml=<domain type='kvm' id='1'> <name>HostedEngine</name> <uuid>10d65864-1721-4eec-a2c5-d6344467d111</uuid> <metadata xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <ovirt-tune:qos/> <ovirt-vm:vm xmlns:ovirt-vm="http://ovirt.org/vm/1.0"> <ovirt-vm:destroy_on_reboot type="bool">False</ovirt-vm:destroy_on_reboot> <ovirt-vm:memGuaranteedSize type="int">0</ovirt-vm:memGuaranteedSize> <ovirt-vm:startTime type="float">1513654775.44</ovirt-vm:startTime> <ovirt-vm:device mac_address="00:16:3e:18:f9:d4"> <ovirt-vm:network>ovirtmgmt</ovirt-vm:network> <ovirt-vm:specParams/> <ovirt-vm:vm_custom/> </ovirt-vm:device> <ovirt-vm:device devtype="disk" name="vda"> <ovirt-vm:domainID>dea166f4-7109-47e4-8baa-15302e6eb1bf</ovirt-vm:domainID> <ovirt-vm:imageID>226bf258-d40a-471f-aa52-ffc84eb55130</ovirt-vm:imageID> <ovirt-vm:poolID>00000000-0000-0000-0000-000000000000</ovirt-vm:poolID> <ovirt-vm:volumeID>82216ddd-4c7e-4c76-8845-0dbb058750e9</ovirt-vm:volumeID> <ovirt-vm:specParams/> <ovirt-vm:vm_custom/> <ovirt-vm:volumeChain> <ovirt-vm:volumeChainNode> <ovirt-vm:domainID>dea166f4-7109-47e4-8baa-15302e6eb1bf</ovirt-vm:domainID> <ovirt-vm:imageID>226bf258-d40a-471f-aa52-ffc84eb55130</ovirt-vm:imageID> <ovirt-vm:leaseOffset type="int">0</ovirt-vm:leaseOffset> <ovirt-vm:leasePath>/rhev/data-center/mnt/10.66.148.11:_home_yzhao_nfs1/dea166f4-7109-47e4-8baa-15302e6eb1bf/images/226bf258-d40a-471f-aa52-ffc84eb55130/82216ddd-4c7e-4c76-8845-0dbb058750e9.lease</ovirt-vm:leasePath> <ovirt-vm:path>/rhev/data-center/mnt/10.66.148.11:_home_yzhao_nfs1/dea166f4-7109-47e4-8baa-15302e6eb1bf/images/226bf258-d40a-471f-aa52-ffc84eb55130/82216ddd-4c7e-4c76-8845-0dbb058750e9</ovirt-vm:path> <ovirt-vm:volumeID>82216ddd-4c7e-4c76-8845-0dbb058750e9</ovirt-vm:volumeID> </ovirt-vm:volumeChainNode> </ovirt-vm:volumeChain> </ovirt-vm:device> </ovirt-vm:vm> </metadata> <memory unit='KiB'>15278080</memory> <currentMemory unit='KiB'>15278080</currentMemory> <vcpu placement='static'>4</vcpu> <cputune> <shares>1020</shares> </cputune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>Red Hat</entry> <entry name='product'>RHEV Hypervisor</entry> <entry name='version'>7.4-0.6.el7</entry> <entry name='serial'>928EA9B0-6656-11E4-8784-D04CC4AC1500</entry> <entry name='uuid'>10d65864-1721-4eec-a2c5-d6344467d111</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.3.0'>hvm</type> <smbios mode='sysinfo'/> </os> <features> <acpi/> </features> <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Haswell-noTSX</model> <feature policy='require' name='vme'/> <feature policy='require' name='f16c'/> <feature policy='require' name='rdrand'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='arat'/> <feature policy='require' name='xsaveopt'/> <feature policy='require' name='abm'/> </cpu> <clock offset='variable' adjustment='-1' basis='utc'> <timer name='rtc' tickpolicy='catchup'/> <timer name='pit' tickpolicy='delay'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/tmp/tmpsZbGZe/seed.iso' startupPolicy='optional'/> <backingStore/> <target dev='hdc' bus='ide'/> <readonly/> <alias name='ide0-1-0'/> <address type='drive' controller='0' bus='1' target='0' unit='0'/> </disk> <disk type='file' device='disk' snapshot='no'> <driver name='qemu' type='raw' cache='none' error_policy='stop' io='threads'/> <source file='/var/run/vdsm/storage/dea166f4-7109-47e4-8baa-15302e6eb1bf/226bf258-d40a-471f-aa52-ffc84eb55130/82216ddd-4c7e-4c76-8845-0dbb058750e9'/> <backingStore/> <target dev='vda' bus='virtio'/> <serial>226bf258-d40a-471f-aa52-ffc84eb55130</serial> <boot order='1'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/> </disk> <controller type='scsi' index='0' model='virtio-scsi'> <alias name='scsi0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </controller> <controller type='usb' index='0' model='piix3-uhci'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <controller type='ide' index='0'> <alias name='ide'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/> </controller> <controller type='virtio-serial' index='0'> <alias name='virtio-serial0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </controller> <lease> <lockspace>dea166f4-7109-47e4-8baa-15302e6eb1bf</lockspace> <key>82216ddd-4c7e-4c76-8845-0dbb058750e9</key> <target path='/rhev/data-center/mnt/10.66.148.11:_home_yzhao_nfs1/dea166f4-7109-47e4-8baa-15302e6eb1bf/images/226bf258-d40a-471f-aa52-ffc84eb55130/82216ddd-4c7e-4c76-8845-0dbb058750e9.lease'/> </lease> <interface type='bridge'> <mac address='00:16:3e:18:f9:d4'/> <source bridge='ovirtmgmt'/> <target dev='vnet0'/> <model type='virtio'/> <link state='up'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <console type='unix'> <source mode='bind' path='/var/run/ovirt-vmconsole-console/10d65864-1721-4eec-a2c5-d6344467d111.sock'/> <target type='virtio' port='0'/> <alias name='console0'/> </console> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channels/10d65864-1721-4eec-a2c5-d6344467d111.com.redhat.rhevm.vdsm'/> <target type='virtio' name='com.redhat.rhevm.vdsm' state='connected'/> <alias name='channel0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channels/10d65864-1721-4eec-a2c5-d6344467d111.org.qemu.guest_agent.0'/> <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/> <alias name='channel1'/> <address type='virtio-serial' controller='0' bus='0' port='2'/> </channel> <channel type='unix'> <source mode='bind' path='/var/lib/libvirt/qemu/channels/10d65864-1721-4eec-a2c5-d6344467d111.org.ovirt.hosted-engine-setup.0'/> <target type='virtio' name='org.ovirt.hosted-engine-setup.0' state='disconnected'/> <alias name='channel2'/> <address type='virtio-serial' controller='0' bus='0' port='3'/> </channel> <input type='mouse' bus='ps2'> <alias name='input0'/> </input> <input type='keyboard' bus='ps2'> <alias name='input1'/> </input> <graphics type='vnc' port='5900' autoport='yes' listen='0' passwdValidTo='2017-12-19T06:39:38'> <listen type='address' address='0'/> </graphics> <video> <model type='vga' vram='32768' heads='1' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='none'/> <rng model='virtio'> <backend model='random'>/dev/urandom</backend> <alias name='rng0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </rng> </devices> <seclabel type='dynamic' model='selinux' relabel='yes'> <label>system_u:system_r:svirt_t:s0:c138,c280</label> <imagelabel>system_u:object_r:svirt_image_t:s0:c138,c280</imagelabel> </seclabel> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+107:+107</label> <imagelabel>+107:+107</imagelabel> </seclabel> </domain> , status=Up, guestDiskMapping={226bf258-d40a-471f-a={name=/dev/vda}, QEMU_DVD-ROM_QM00003={name=/dev/sr0}}, vmName=HostedEngine, vmId=10d65864-1721-4eec-a2c5-d6344467d111, devices=[Ljava.lang.Object;@3d16752d, smp=4, vmType=kvm, external=false, memSize=14920, clientIp=, statusTime=4296633730, arch=x86_64}], log id: 4ba3e3cc 2017-12-18 22:57:19,029-05 ERROR [org.ovirt.engine.core.bll.AddUnmanagedVmsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [149fd43] Command 'org.ovirt.engine.core.bll.AddUnmanagedVmsCommand' failed: No enum constant org.ovirt.engine.core.common.businessentities.network.VmInterfaceType.virtio 2017-12-18 22:57:19,029-05 ERROR [org.ovirt.engine.core.bll.AddUnmanagedVmsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [149fd43] Exception: java.lang.IllegalArgumentException: No enum constant org.ovirt.engine.core.common.businessentities.network.VmInterfaceType.virtio at java.lang.Enum.valueOf(Enum.java:238) [rt.jar:1.8.0_151] at org.ovirt.engine.core.common.businessentities.network.VmInterfaceType.valueOf(VmInterfaceType.java:6) [common.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerObjectsBuilder.buildVmNetworkInterfacesFromDevices(VdsBrokerObjectsBuilder.java:232) [vdsbroker.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.importHostedEngineVm(AddUnmanagedVmsCommand.java:181) [bll.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.convertVm(AddUnmanagedVmsCommand.java:111) [bll.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.executeCommand(AddUnmanagedVmsCommand.java:97) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1205) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1345) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1987) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:164) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:103) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1405) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:412) [bll.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:509) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:491) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:697) [bll.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80) [wildfly-weld-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) [wildfly-weld-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl.jar:2.3.3.Final-redhat-1] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:263) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:374) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:243) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:66) [wildfly-ejb3-7.0.0.GA-redhat-2.jar:7.0.0.GA-redhat-2] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:195) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view4.runInternalAction(Unknown Source) [bll.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_151] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_151] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_151] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_151] at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:433) [weld-core-impl.jar:2.3.3.Final-redhat-1] at org.jboss.weld.bean.proxy.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:128) [weld-core-impl.jar:2.3.3.Final-redhat-1] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-impl.jar:2.3.3.Final-redhat-1] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:100) [weld-core-impl.jar:2.3.3.Final-redhat-1] at org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runInternalAction(Unknown Source) [bll.jar:] at org.ovirt.engine.core.bll.VdsEventListener.addUnmanagedVms(VdsEventListener.java:473) [bll.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring.addUnmanagedVms(VmsMonitoring.java:326) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring.perform(VmsMonitoring.java:96) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher.poll(PollVmStatsRefresher.java:43) [vdsbroker.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_151] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_151] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent.jar:1.0.0.redhat-1] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent.jar:1.0.0.redhat-1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_151] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent.jar:1.0.0.redhat-1] Version-Release number of selected component (if applicable): Seen on https://bugzilla.redhat.com/show_bug.cgi?id=1522641#c48 with rhvm-appliance-4.2-20171207.0.el7.ova which contains: ovirt-engine-4.2.0-0.6.el7.noarch vdsm-jsonrpc-java-1.4.9-1.el7ev.noarch How reproducible: 5 times in a row Steps to Reproduce: 1. deploy hosted-engine using rhvm-appliance-4.2-20171207.0.el7.ova 2. 3. Actual results: hosted-engine-setup fails with [ ERROR ] Unable to add {host} to the manager in engine.log we see: Exception: java.lang.IllegalArgumentException: No enum constant org.ovirt.engine.core.common.businessentities.network.VmInterfaceType.virtio Expected results: hosted-engine successfully deploys Additional info:
Created attachment 1369833 [details] engine.log
I saw this exception, but not that it failed the flow. Gil, did QE see this?
Created attachment 1369880 [details] setup and engine logs I ran HE deployment on ISCS and deployment passed, but I still can see the exception in the engine.log Versions: rhvm-4.2.0-0.6.el7.noarch vdsm-jsonrpc-java-1.4.9-1.el7ev.noarch
Passed on both Gluster and NFS. I also see the exception.
(In reply to Yaniv Kaul from comment #2) > I saw this exception, but not that it failed the flow. Gil, did QE see this? We are not failing the setup if the host is not up in the engine and this can hide the issue, see https://bugzilla.redhat.com/show_bug.cgi?id=1522641#c52
(In reply to Simone Tiraboschi from comment #6) > (In reply to Yaniv Kaul from comment #2) > > I saw this exception, but not that it failed the flow. Gil, did QE see this? > > We are not failing the setup if the host is not up in the engine and this > can hide the issue, > see https://bugzilla.redhat.com/show_bug.cgi?id=1522641#c52 https://bugzilla.redhat.com/show_bug.cgi?id=1527394 might also be related.
Cleaning the need info based on comment #4 and #5 (We see the exception - it does not fail the deployment)
Milan, it seems that when we create the nic device in vdsm from the engine xml, we take the model as is from the <model type='..'> sub-element of the interface device element, but that should be mapped to 'pv' for the engine, otherwise the engine won't recognize it (you can see the opposite direction of the mapping in [1]). [1] https://github.com/oVirt/vdsm/blob/master/lib/vdsm/virt/vmdevices/network.py#L135
(In reply to Arik from comment #10) Actually, I'm not that sure that this is a result of the engine XML since I don't see a reverse mapping of 'virtio' to 'pv' anywhere in vdsm when the devices are reported back to the engine as part of the FullList. Simone, did you guys maybe change the model of interfaces of the hosted-engine VM to 'virtio' by any chance? if not, are we sure that it is a regression?
And the code cited by Arik in comment #10 is not new, so if a reverse conversion is missing, it has been missing for long time and is unrelated to Engine XML.
(In reply to Arik from comment #11) > Simone, did you guys maybe change the model of interfaces of the > hosted-engine VM to 'virtio' by any chance? if not, are we sure that it is a > regression? On hosted-engine-setup side it's still 'pv': https://github.com/oVirt/ovirt-hosted-engine-setup/blob/master/src/ovirt_hosted_engine_setup/mixins.py#L240 But in comment 1 I see that vdsm reports it as virtio: <source bridge='ovirtmgmt'/> <target dev='vnet0'/> <model type='virtio'/> In the past we never saw this.
The log @ https://bugzilla.redhat.com/show_bug.cgi?id=1527866 has a somewhat different stack: 2017-12-21 12:45:56,184+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Received a console Device without an address when processing VM 1664ad88-c76e-42fc-a e03-26ba642513e1 devices, skipping device: {device=console, alias=serial0, vmid=1664ad88-c76e-42fc-ae03-26ba642513e1, type=console, specParams={consoleType=serial, enableSocket=false}} 2017-12-21 12:45:56,185+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Received a spice Device without an address when processing VM 1664ad88-c76e-42fc-ae0 3-26ba642513e1 devices, skipping device: {device=spice, specParams={fileTransferEnable=true, copyPasteEnable=true, displayIp=127.0.0.1}, vmid=1664ad88-c76e-42fc-ae03-26ba642513e1, port=5900, type=spice} 2017-12-21 12:45:56,185+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Received a pci Device without an address when processing VM 1664ad88-c76e-42fc-ae03- 26ba642513e1 devices, skipping device: {device=pci, alias=pci.0, model=pci-root, type=pci, index=0} 2017-12-21 12:45:56,186+02 ERROR [org.ovirt.engine.core.bll.AddUnmanagedVmsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Command 'org.ovirt.engine.core.bll.AddUnmanagedVmsCommand' failed: java.lang.Boolean cannot be cast to java.lang.String 2017-12-21 12:45:56,186+02 ERROR [org.ovirt.engine.core.bll.AddUnmanagedVmsCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Exception: java.lang.ClassCastException: java.lang.Boolean cannot be cast to java.lang.String at org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring.buildNewVmDevice(VmDevicesMonitoring.java:652) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring.processVmDevices(VmDevicesMonitoring.java:498) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring.access$600(VmDevicesMonitoring.java:53) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring$Change.processFullList(VmDevicesMonitoring.java:210) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.monitoring.VmDevicesMonitoring$Change.updateVmFromFullList(VmDevicesMonitoring.java:193) [vdsbroker.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.addDevices(AddUnmanagedVmsCommand.java:228) [bll.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.convertVm(AddUnmanagedVmsCommand.java:134) [bll.jar:] at org.ovirt.engine.core.bll.AddUnmanagedVmsCommand.executeCommand(AddUnmanagedVmsCommand.java:97) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1209) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1349) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1991) [bll.jar:]
(In reply to Yaniv Kaul from comment #14) That exception happens in a different flow where the VM is detected as an 'external VM' (rather than the one related to this bz where we have a VM in the database with origin=HOSTED_ENGINE).
The posted patch is not enough, seems like we have more issues with disk devices due to 4.2.1 changes that need to be addressed - the proposed patch solves only the exception related to the conversion of NIC model.
oops, that was incorrect - the fixes on the engine side allow new engine work with old vdsm, but Milan's patch is needed for old engine to work with new vdsm.
Works for me on these components: rhvm-appliance-4.2-20180202.0.el7.noarch ovirt-hosted-engine-ha-2.2.4-1.el7ev.noarch ovirt-hosted-engine-setup-2.2.9-1.el7ev.noarch Linux 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64 x86_64 x86_64 GNU/Linux Successfully deployed SHE on FC. Moving to verified.
This bugzilla is included in oVirt 4.2.1 release, published on Feb 12th 2018. Since the problem described in this bug report should be resolved in oVirt 4.2.1 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.