4.3.0 is tracked on bug #1561964
I fear it's due to a combination of: https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/common/src/main/java/org/ovirt/engine/core/common/action/VmManagementParametersBase.java#L114 with: https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/bll/UpdateVmCommand.java#L1228
ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNofmyenginehere list . . . debug1: Sending command: list 318ab408-b282-4d31-8935-4cf939b144c7 HostedEngine debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 . . . Enabled serial console for the engine and it received and saved within engine's configuration. By default serial console is disabled on engine. ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNofmyenginehere . . ERROR: Console '318ab408-b282-4d31-8935-4cf939b144c7.sock' is not available debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 debug1: client_input_channel_req: channel 0 rtype eow reply 0 Connection to alma04.qa.lab.tlv.redhat.com closed. debug1: channel 0: free: client-session, nchannels 1 Connection to nsednev-he-1.qa.lab.tlv.redhat.com closed. Transferred: sent 3256, received 4224 bytes, in 1.9 seconds Bytes per second: sent 1705.0, received 2211.9 debug1: Exit status 1 ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNofmyenginehere connect . . debug1: Sending command: connect ERROR: No pty support, please enable at client side debug1: client_input_channel_req: channel 0 rtype exit-status reply 0 debug1: client_input_channel_req: channel 0 rtype eow reply 0 On engine I see this: he-1 ~]# systemctl status *console* -l ● systemd-vconsole-setup.service - Setup Virtual Console Loaded: loaded (/usr/lib/systemd/system/systemd-vconsole-setup.service; static; vendor preset: disabled) Active: active (exited) since Wed 2018-07-25 18:22:32 IDT; 1h 35min ago Docs: man:systemd-vconsole-setup.service(8) man:vconsole.conf(5) Process: 503 ExecStart=/usr/lib/systemd/systemd-vconsole-setup (code=exited, status=0/SUCCESS) Main PID: 503 (code=exited, status=0/SUCCESS) CGroup: /system.slice/systemd-vconsole-setup.service Jul 25 18:22:32 nsednev-he-1.qa.lab.tlv.redhat.com systemd[1]: Started Setup Virtual Console. ● ovirt-vmconsole-proxy-sshd.service - oVirt VM Console SSH server daemon Loaded: loaded (/usr/lib/systemd/system/ovirt-vmconsole-proxy-sshd.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2018-07-25 18:22:41 IDT; 1h 35min ago Main PID: 1402 (sshd) CGroup: /system.slice/ovirt-vmconsole-proxy-sshd.service └─1402 /usr/sbin/sshd -f /usr/share/ovirt-vmconsole/ovirt-vmconsole-proxy/ovirt-vmconsole-proxy-sshd/sshd_config -D Jul 25 19:44:45 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5418]: Accepted publickey for ovirt-vmconsole from 10.36.116.164 port 39494 ssh2: RSA SHA256:Dqs2nB/hQG2QfVCJABZbC38q8jZm99Fkw8UEBgCfYLI Jul 25 19:44:52 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5447]: rexec line 22: Deprecated option RSAAuthentication Jul 25 19:44:52 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5447]: ssh_selinux_change_context: setcon system_u:system_r:sshd_net_t:s0-s0:c0.c1023 from system_u:system_r:ovirt_vmconsole_t:s0-s0:c0.c1023 failed with Permission denied [preauth] Jul 25 19:44:53 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5447]: reprocess config line 22: Deprecated option RSAAuthentication Jul 25 19:44:53 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5447]: Accepted publickey for ovirt-vmconsole from 10.36.116.164 port 39510 ssh2: RSA SHA256:Dqs2nB/hQG2QfVCJABZbC38q8jZm99Fkw8UEBgCfYLI Jul 25 19:47:30 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5562]: rexec line 22: Deprecated option RSAAuthentication Jul 25 19:47:30 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5562]: ssh_selinux_change_context: setcon system_u:system_r:sshd_net_t:s0-s0:c0.c1023 from system_u:system_r:ovirt_vmconsole_t:s0-s0:c0.c1023 failed with Permission denied [preauth] Jul 25 19:47:31 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5562]: reprocess config line 22: Deprecated option RSAAuthentication Jul 25 19:47:31 nsednev-he-1.qa.lab.tlv.redhat.com sshd[5562]: Accepted publickey for ovirt-vmconsole from 10.36.116.164 port 39620 ssh2: RSA SHA256:Dqs2nB/hQG2QfVCJABZbC38q8jZm99Fkw8UEBgCfYLI Jul 25 19:47:33 nsednev-he-1.qa.lab.tlv.redhat.com ovirt-vmconsole-proxy-shell[5576]: ERROR No pty support, please enable at client side Moving back to assigned as I was unable to connect over serial console to the engine, although now serial console can be successfully saved in the edit menu for the engine. Tested on these components on hosts: ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch ovirt-hosted-engine-setup-2.2.24-1.el7ev.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-vmconsole-host-1.0.4-1.el7ev.noarch rhvm-appliance-4.2-20180620.0.el7.noarch Linux 3.10.0-862.10.2.el7.x86_64 #1 SMP Wed Jul 4 09:41:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.5 (Maipo) On engine: ovirt-vmconsole-proxy-1.0.5-4.el7ev.noarch ovirt-vmconsole-1.0.5-4.el7ev.noarch ovirt-engine-vmconsole-proxy-helper-4.2.6_SNAPSHOT-84.gad3de30.0.scratch.master.el7ev.noarch ovirt-engine-setup-4.2.6_SNAPSHOT-84.gad3de30.0.scratch.master.el7ev.noarch ovirt-engine-setup-base-4.2.6_SNAPSHOT-84.gad3de30.0.scratch.master.el7ev.noarch Linux 3.10.0-862.10.2.el7.x86_64 #1 SMP Wed Jul 4 09:41:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.5 (Maipo)
(In reply to Nikolai Sednev from comment #7) > Moving back to assigned as I was unable to connect over serial console to > the engine, although now serial console can be successfully saved in the > edit menu for the engine. Did you also tried with hosted-engine --console on host side?
(In reply to Simone Tiraboschi from comment #8) > (In reply to Nikolai Sednev from comment #7) > > Moving back to assigned as I was unable to connect over serial console to > > the engine, although now serial console can be successfully saved in the > > edit menu for the engine. > > Did you also tried with hosted-engine --console on host side? Yes and it looks like nothing is listening on the engine's side and there is no login/password during connection to the engine: alma04 ~]# hosted-engine --console The engine VM is running on this host Escape character is ^] pwd
OK, let's try to recap. downstream rhevm-appliance is not configured to accept logins on the serial console, it's now tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1608733 we can fix for future rhevm-appliance builds but who deployed in the past has to manually fix it and reboot the engine VM. Upstream appliance is fine on this point. Then Nikolai initially deployed with an old appliance without the fix and so the console device got lost as for this bug. Then he upgraded his engine to the latest build and so the console device can be enabled also for the engine VM but nothing is automatically re-enabling it for engine VM deployed in the past without the serial console. More than that, the user has to wait one hour for OVF store regeneration and then reboot the engine VM to make it effective. We can probably forcefully turn on the console device at engine-setup time on upgrades and maybe we can also do something to trigger a quick OVF_STORE refresh. But then we still have to reboot the engine VM to make it effective. Yaniv, do we really want to have this automatically or just warn the user somehow?
Another option is to simply track this is a KBS and have the user enabling the serial console device from the engine rebooting the VM at the end. It could make sense since he has also to manually enable the serial console at OS level on downstream appliance deployed in the past. The bad point is that the serial console is probably going to be used for troubleshooting activities and if you are in trouble you probably don't have a running engine to enable it.
Please check this comment: https://bugzilla.redhat.com/show_bug.cgi?id=1608733#c4 Before using solution: https://access.redhat.com/solutions/3545201
On latest appliance rhvm-appliance-4.2-20180727.1.el7.noarch the serial console comes disabled by default. Tested on these components: ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch ovirt-hosted-engine-setup-2.2.24-1.el7ev.noarch rhvm-appliance-4.2-20180727.1.el7.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-vmconsole-host-1.0.4-1.el7ev.noarch Linux 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.5 (Maipo)
Engine setup within the rhvm-appliance-4.2-20180727.1.el7.noarch is ovirt-engine-setup-4.2.5.2-0.1.el7ev.noarch.
Tested on these components on host: ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch ovirt-hosted-engine-setup-2.2.25-1.el7ev.noarch rhvm-appliance-4.2-20180801.0.el7.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch ovirt-vmconsole-host-1.0.4-1.el7ev.noarch Linux 3.10.0-862.10.2.el7.x86_64 #1 SMP Wed Jul 4 09:41:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.5 (Maipo) Engine within the appliance is: ovirt-engine-setup-4.2.5.2-0.1.el7ev.noarch ovirt-vmconsole-1.0.5-4.el7ev.noarch ovirt-vmconsole-proxy-1.0.5-4.el7ev.noarch The "Enable VirtIO serial console" is not marked in UI, while it should be by default. Serial console is not working by default. Moving back to assigned.
IIUC, this bug doesn't talk about enabling it by default in the UI. The console wasn't available at all previously, even when it was marked in the UI, because of a wrong console= parameter in the appliance kernel boot params. Was this fixed ?
I tried reproducing again and it worked as expected for me with: ovirt-hosted-engine-setup.noarch 2.2.25-1.el7ev @rhv-4.2.5 ovirt-hosted-engine-ha.noarch 2.2.16-1.el7ev @rhv-4.2.5 rhvm-appliance.noarch 2:4.2-20180801.0.el7 installed ansible.noarch 2.6.2-1.el7ae @ansible-nighly-2.5.z [root@r75he20180403h1 ~]# hosted-engine --console The engine VM is running on this host Connected to domain HostedEngine Escape character is ^] Red Hat Enterprise Linux Server 7.5 (Maipo) Kernel 3.10.0-862.9.1.el7.x86_64 on an x86_64 enginevm login:
"hosted-engine --console" from ha-host that is running the engine is not equal to "ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNOFYOURENGINE". In order to make serial console work properly, customer required: 1.Deploy SHE. 2.Copy it's public ssh key to the engine. 3.Enable serial console on engine's VM. 4.Set environment in to global maintenance. 5.Power-off or shutdown the HE-VM. 6.Power-on the HE-VM. 7.Remove global maintenance. 8.Connect from remote laptop over ssh using public ssh key to SHE-VM using "ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNOFYOURENGINE". In such scenario customer exposed to engine's downtime and not easy setup. I'm moving this bug to verified with the exceptions as were described above, due to the fact that engine is not programmed for getting started with enabled by default serial console on it, but to have the option to get serial console enabled only. If steps 1-8 followed in correct order, then serial console is working fine: debug1: Remote: Port forwarding disabled. debug1: Remote: User rc execution disabled. debug1: Remote: X11 forwarding disabled. debug1: Remote: Forced command. debug1: Remote: Agent forwarding disabled. debug1: Remote: Port forwarding disabled. debug1: Remote: User rc execution disabled. debug1: Remote: X11 forwarding disabled. debug1: Sending environment. debug1: Sending env LANG = en_US.UTF-8 debug1: Sending env LANGUAGE = Red Hat Enterprise Linux Server 7.5 (Maipo) Kernel 3.10.0-862.9.1.el7.x86_64 on an x86_64 nsednev-he-1 login:
(In reply to Nikolai Sednev from comment #21) > "hosted-engine --console" from ha-host that is running the engine is not > equal to "ssh -v -i /root/.ssh/id_rsa -p 2222 > ovirt-vmconsole@FQDNOFYOURENGINE". hosted-engine --console is now wrapping socat over the serial console exactly as ovirt-vmconsole > In order to make serial console work properly, customer required: > 3.Enable serial console on engine's VM. > 4.Set environment in to global maintenance. > 5.Power-off or shutdown the HE-VM. > 6.Power-on the HE-VM. > 7.Remove global maintenance. These are not needed, on a clean setup it should be enabled by default as I verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:2349
sync2jira