Bug 1364132 - Once the engine imports the hosted-engine VM we lose the console device
Summary: Once the engine imports the hosted-engine VM we lose the console device
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.HostedEngine
Version: 4.0.2
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.1.1
: 4.1.1
Assignee: Jenny Tokar
QA Contact: Nikolai Sednev
URL:
Whiteboard: PM-10
Depends On:
Blocks: 1370120
TreeView+ depends on / blocked
 
Reported: 2016-08-04 14:14 UTC by Simone Tiraboschi
Modified: 2018-06-07 14:44 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The console device was not added to the engine and thus was not written to the generated ovf. Consequence: When hosted engine was started with the generated ovf it was impossible to connect to the vm via the console. Fix: 1. Import the console to the engine if it exists in the initial configuration. 2. When reading the ovf in the hosted engine agent always add a console device to the generated vm.conf. Result: It is possible to connect to the hosted engine vm via console device.
Clone Of:
Environment:
Last Closed: 2017-04-21 09:41:56 UTC
oVirt Team: SLA
Embargoed:
rule-engine: ovirt-4.1+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1368027 0 high CLOSED Hosted Engine VM needs RNG for entropy 2021-02-22 00:41:40 UTC
Red Hat Bugzilla 1373484 0 high CLOSED Initial HE VM config must include maxVCpus parameter in order to support CPU hotplug 2021-02-22 00:41:40 UTC
oVirt gerrit 70594 0 'None' 'MERGED' 'core: Add console device' 2019-12-05 14:13:59 UTC
oVirt gerrit 71263 0 'None' 'MERGED' 'Always add console device' 2019-12-05 14:13:59 UTC
oVirt gerrit 71472 0 'None' 'MERGED' 'Always add console device' 2019-12-05 14:13:59 UTC
oVirt gerrit 71757 0 'None' 'MERGED' 'core: Add console device' 2019-12-05 14:13:59 UTC

Internal Links: 1368027 1373484

Description Simone Tiraboschi 2016-08-04 14:14:18 UTC
Description of problem:
In the initial vm.conf we have 
devices={device:console,specParams:{},type:console,deviceId:1e69c76c-4928-44e9-bc5f-2d7e90141cc3,alias:console0}

But we don't have any console device in the vm.conf we get back from the OVF_STORE.

so 
virsh console is failing with

Connected to domain HostedEngine
Escape character is ^]
error: internal error: cannot find character device <null>

while also
socat UNIX-CONNECT:/var/run/ovirt-vmconsole-console/{vmuuid}.sock,user=ovirt-vmconsole STDIO,raw,echo=0,escape=1
is failing since we don't have any unix domain socket under /var/run/ovirt-vmconsole-console/


Version-Release number of selected component (if applicable):
ovirt-engine.noarch                 4.0.2.4-1.el7.centos

How reproducible:
100%

Steps to Reproduce:
1. deploy hosted-engine
2. add the first regular storage domain
3. wait for the engine VM to be imported and for the OVF_STORE to be generated (by default OvfUpdateIntervalInMinutes is at 60 minutes)

Actual results:
the generated vm.conf doesn't contain
devices={device:console,specParams:{},type:console,deviceId:1e69c76c-4928-44e9-bc5f-2d7e90141cc3,alias:console0}
or something similar

Expected results:
we have a console device

Additional info:

Comment 1 Simone Tiraboschi 2016-08-04 14:14:56 UTC
The console device is neither in the ovf inside the OVF_STORE

Comment 2 Sandro Bonazzola 2016-08-05 14:23:14 UTC
Targeting to 4.0.4 since it's blocking a 4.0.4 targeted bug

Comment 3 Simone Tiraboschi 2016-08-24 16:42:05 UTC
With rhbz#1333449 we are going to rewrote 'hosted-engine --console' command to use the serial connection though virsh instead of a graphical connection via virt-viewer, this to avoid requiring a graphical stack on the host.

Due to to this bug the serial console is only available in the bootstrap phase but not when the engine import the engine vm so at that point
'hosted-engine --console' will not work.
'hosted-engine --console' is primarily used for bootstrapping task but it can be also really useful for troubleshooting activities on existing systems.

Since this is not really blocking rhbz#1333449 we removed the dependency but this is still worth to be fixed to provide a working troubleshooting console with 'hosted-engine --console'.

Comment 4 Doron Fediuck 2016-09-07 10:57:45 UTC
Based on comment 3 we will reevaluate this for the next 4.x release.

Comment 5 Yaniv Lavi 2016-09-14 13:21:45 UTC
Changing to RFE to tackle some similar issues we have. Currently around devices.

Comment 6 Jenny Tokar 2017-01-09 17:15:21 UTC
After some investigation, the console device is not being added by the engine since it lacks the address attribute. 
I'm not sure where (or if) this attribute should be created for this device.

Comment 7 Tomas Jelinek 2017-01-11 14:24:18 UTC
@Jenny - the console device should not have an address. We don't even send it to VDSM (similar to graphics device). So the fact the console device does not have an address is expected and should be handled properly by export/import.

Comment 8 Sandro Bonazzola 2017-02-01 16:01:16 UTC
oVirt 4.1.0 GA has been released, re-targeting to 4.1.1.
Please check if this issue is correctly targeted or already included in 4.1.0.

Comment 9 Jenny Tokar 2017-02-02 12:18:02 UTC
The target is correct, the fix is not included in 4.1.0.

Comment 10 Nikolai Sednev 2017-02-15 07:59:13 UTC
Shouldn't this bug get cloned to 4.1.0?

Comment 11 Nikolai Sednev 2017-02-15 12:48:04 UTC
Works for me for 4.1.0 upgraded engine to 4.1.1-0.1, while hosts are 4.1RHEVHs and auto-import was done in 4.1.0.

Working on 4.1.1-0.1:

#  hosted-engine --console
The engine VM is running on this host
Connected to domain HostedEngine
Escape character is ^]

Red Hat Enterprise Linux Server 7.3 (Maipo)
Kernel 3.10.0-514.6.1.el7.x86_64 on an x86_64

nsednev-he-1 login: 
Password: 

Engine:
rhevm-setup-plugins-4.1.0-1.el7ev.noarch
rhevm-dependencies-4.1.0-1.el7ev.noarch
rhevm-doc-4.1.0-2.el7ev.noarch
rhev-guest-tools-iso-4.1-3.el7ev.noarch
rhevm-4.1.1-0.1.el7.noarch
rhevm-branding-rhev-4.1.0-0.el7ev.noarch
Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.3 (Maipo)

Hosts:
sanlock-3.4.0-1.el7.x86_64
ovirt-node-ng-nodectl-4.1.0-0.20170104.1.el7.noarch
qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64
vdsm-4.19.4-1.el7ev.x86_64
ovirt-imageio-common-1.0.0-0.el7ev.noarch
ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch
ovirt-setup-lib-1.1.0-1.el7ev.noarch
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
mom-0.5.8-1.el7ev.noarch
ovirt-imageio-daemon-1.0.0-0.el7ev.noarch
ovirt-hosted-engine-ha-2.1.0.1-1.el7ev.noarch
ovirt-hosted-engine-setup-2.1.0.1-1.el7ev.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
libvirt-client-2.0.0-10.el7_3.4.x86_64
ovirt-host-deploy-1.6.0-1.el7ev.noarch
Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016
Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux release 7.3


HA logs shows that engine vm.conf being read from shared storage:
MainThread::INFO::2017-02-15 14:44:50,404::config::405::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config
::(_get_vm_conf_content_from_ovf_store) Trying to get a fresher copy of vm configuration from the OVF_STORE
MainThread::INFO::2017-02-15 14:44:52,875::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) F
ound OVF_STORE: imgUUID:3bc8a554-b5eb-4beb-8358-e4600471adc6, volUUID:1389cc6d-df76-444f-bcda-93da27b915b8
MainThread::INFO::2017-02-15 14:44:53,270::ovf_store::103::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) F
ound OVF_STORE: imgUUID:f2f9add7-a814-4054-a8ea-0b66345929c5, volUUID:265ed6e3-9be2-4b96-a2c6-1a3846e2c25a
MainThread::INFO::2017-02-15 14:44:53,349::ovf_store::112::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngi
neVMOVF) Extracting Engine VM OVF from the OVF_STORE
MainThread::INFO::2017-02-15 14:44:53,356::ovf_store::119::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(getEngi
neVMOVF) OVF_STORE volume path: /rhev/data-center/mnt/10.35.110.11:_Compute__NFS_nsednev__he__1/ffb43f0a-1e02-4b8e-a4d
b-bc89e47fef20/images/f2f9add7-a814-4054-a8ea-0b66345929c5/265ed6e3-9be2-4b96-a2c6-1a3846e2c25a 
MainThread::INFO::2017-02-15 14:44:53,372::config::424::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config
::(_get_vm_conf_content_from_ovf_store) Found an OVF for HE VM, trying to convert
MainThread::INFO::2017-02-15 14:44:53,378::config::429::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store) Got vm.conf from OVF_STORE

vm.conf on host, locally stored:
# cat /run/ovirt-hosted-engine-ha/vm.conf
cpuType=SandyBridge
emulatedMachine=pc-i440fx-rhel7.3.0
vmId=8dc1d363-1127-44c5-881a-c0e3a55c69f1
smp=4
memSize=16385
maxVCpus=16
spiceSecureChannels=smain,sdisplay,sinputs,scursor,splayback,srecord,ssmartcard,susbredir
vmName=HostedEngine
display=qxl
devices={index:0,iface:virtio,format:raw,bootOrder:1,address:{slot:0x06,bus:0x00,domain:0x0000,type:pci,function:0x0},volumeID:f99baa82-25be-4659-93c8-759f4ac31309,imageID:895ab3d0-b1bf-4b8f-a554-de6e6c0b9c79,readonly:false,domainID:ffb43f0a-1e02-4b8e-a4db-bc89e47fef20,deviceId:895ab3d0-b1bf-4b8f-a554-de6e6c0b9c79,poolID:00000000-0000-0000-0000-000000000000,device:disk,shared:exclusive,propagateErrors:off,type:disk}
devices={nicModel:pv,macAddr:00:16:3E:7B:B8:53,linkActive:true,network:ovirtmgmt,deviceId:c563ed2c-0280-4c94-84f4-c8998f063d74,address:{slot:0x03,bus:0x00,domain:0x0000,type:pci,function:0x0},device:bridge,type:interface}
devices={index:2,iface:ide,shared:false,readonly:true,deviceId:8c3179ac-b322-4f5c-9449-c52e3665e0ae,address:{controller:0,target:0,unit:0,bus:1,type:drive},device:cdrom,path:,type:disk}
devices={device:virtio-serial,type:controller,deviceId:7ee9b46d-7ddd-413f-a1a0-4d9b5a3448cf,address:{slot:0x05,bus:0x00,domain:0x0000,type:pci,function:0x0}}
devices={alias:rng0,specParams:{source:random},deviceId:d350f421-9faa-46d2-b4ab-cdd0df191d49,address:{slot:0x07,bus:0x00,domain:0x0000,type:pci,function:0x0},device:virtio,model:virtio,type:rng}
devices={device:scsi,model:virtio-scsi,type:controller,deviceId:170fa597-cf9e-4508-8a8e-dc85ed84297c,address:{slot:0x04,bus:0x00,domain:0x0000,type:pci,function:0x0}}
devices={device:usb,type:controller,deviceId:b95479b4-0b02-44cc-bba6-4cea8bf36e1f,address:{slot:0x01,bus:0x00,domain:0x0000,type:pci,function:0x2}}
devices={device:ide,type:controller,deviceId:2c9eb6ce-5513-44a9-a0fe-240e5e5b9cb1,address:{slot:0x01,bus:0x00,domain:0x0000,type:pci,function:0x1}}
devices={device:qxl,alias:video0,type:video,deviceId:adbe726a-9b71-420d-ba59-780e106a3d88,address:{slot:0x02,bus:0x00,domain:0x0000,type:pci,function:0x0}}

Moving to verified.

Comment 12 Yaniv Lavi 2017-02-23 07:56:32 UTC
(In reply to Nikolai Sednev from comment #10)
> Shouldn't this bug get cloned to 4.1.0?

No, cloning is optional in upstream.


Note You need to log in before you can comment on or make changes to this bug.