Bug 1608733

Summary: Serial console connection to HE returns "ERROR: No pty support, please enable at client side" and fails.
Product: Red Hat Enterprise Virtualization Manager Reporter: Nikolai Sednev <nsednev>
Component: rhevm-applianceAssignee: Yuval Turgeman <yturgema>
Status: CLOSED ERRATA QA Contact: Nikolai Sednev <nsednev>
Severity: high Docs Contact:
Priority: urgent    
Version: 4.2.5CC: cshao, dfediuck, huzhao, mkalinin, qiyuan, stirabos, weiwang, yaniwang, yaoxu, ycui, ylavi, yzhao
Target Milestone: ovirt-4.2.5-1Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-06 09:18:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Node RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1590943    

Description Nikolai Sednev 2018-07-26 08:44:31 UTC
Description of problem:
Serial console connection to HE returns "ERROR: No pty support, please enable at client side" and fails.

Version-Release number of selected component (if applicable):
Components on engine:
ovirt-engine-setup-4.2.6_SNAPSHOT-84.gad3de30.0.scratch.master.el7ev.noarch
ovirt-vmconsole-1.0.5-4.el7ev.noarch
ovirt-engine-vmconsole-proxy-helper-4.2.6_SNAPSHOT-84.gad3de30.0.scratch.master.el7ev.noarch
Linux 3.10.0-862.10.2.el7.x86_64 #1 SMP Wed Jul 4 09:41:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

Components on host:
ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.24-1.el7ev.noarch
Linux 3.10.0-862.10.2.el7.x86_64 #1 SMP Wed Jul 4 09:41:38 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

How reproducible:
100%

Steps to Reproduce:
1.Deploy SHE ove any type of storage.
2.Enable serial console on engine VM from UI and save.
3.Check that in UI you see that serial console is enabled for the engine VM.
4.Wait more than 1 hour, so the OVF store will receive the XML changes.
5.Restart the engine VM.
6.Check that XML dump has serial console enabled for the engine on host "virsh -r dumpxml HostedEngine".
7.Connect to HE-VM using serial console from your laptop "ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNoftheHEVMhere connect".

Actual results:
nsednev ~]# ssh -v -i /root/.ssh/id_rsa -p 2222 ovirt-vmconsole@FQDNoftheHEVMhere connect
debug1: Sending env LANGUAGE = 
debug1: Sending command: connect
ERROR: No pty support, please enable at client side
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2932, received 4104 bytes, in 1.3 seconds
Bytes per second: sent 2323.0, received 3251.6
debug1: Exit status 1


Expected results:
Serial connection should succeed.

Additional info:
http://pastebin.test.redhat.com/623637
OVF store has proper serial console enabled XML configuration, engine was restarted and received proper serial console device configuration to it.
 virsh -r dumpxml HostedEngine
    <serial type='unix'>
      <source mode='bind' path='/var/run/ovirt-vmconsole-console/318ab408-b282-4d31-8935-4cf939b144c7.sock'/>
       <target type='isa-serial' port='0'>
         <model name='isa-serial'/>
       </target>
       <alias name='serial0'/>
     </serial>

Comment 1 Simone Tiraboschi 2018-07-26 08:46:11 UTC
In upstream ovirt-engine-appliance nothing special is needed

Comment 2 Simone Tiraboschi 2018-07-26 09:06:22 UTC
Removing console=tty0 and adding console=ttyS0,115200 in GRUB_CMDLINE_LINUX (running "grub2-mkconfig -o /etc/grub2.cfg" after that) as for https://access.redhat.com/solutions/277793 fixes it.

In upstream ovirt-engine-appliace we have:
[root ~]# cat /etc/default/grub
GRUB_TIMEOUT=1
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL="serial console"
GRUB_SERIAL_COMMAND="serial"
GRUB_CMDLINE_LINUX="crashkernel=auto console=ttyS0"
GRUB_DISABLE_RECOVERY="true"

and the serial console works by default.

We can fix on newer rhvm-appliance build but who deployed in the past has to manually fix enabling the serial console at OS level if needed.
Marina, could you please open a KBS for this?

Comment 4 Marina Kalinin 2018-07-27 17:00:55 UTC
I created this KCS to address this bz. Please review:
https://access.redhat.com/solutions/3545201

However, I could not fully verify it because I hit bz#1575996 and could not start HE VM with the serial console device.
I need someone else to confirm my solution before we can set it as verified.

Also, I am not sure I am doing it right - vmconsole proxy is running on the hosted-engine. Maybe we should use screen or something else to connect to HE VM instead of via vmconsole?

Comment 5 Simone Tiraboschi 2018-07-30 11:53:27 UTC
(In reply to Marina from comment #4)
> I created this KCS to address this bz. Please review:
> https://access.redhat.com/solutions/3545201
> 
> However, I could not fully verify it because I hit bz#1575996 and could not
> start HE VM with the serial console device.
> I need someone else to confirm my solution before we can set it as verified.
> 
> Also, I am not sure I am doing it right - vmconsole proxy is running on the
> hosted-engine. Maybe we should use screen or something else to connect to HE
> VM instead of via vmconsole?

Two comments:
- the user has to update ovirt-engine (due to https://bugzilla.redhat.com/show_bug.cgi?id=1590943 ) and ovirt-hosted-engine-setup (on host side)
- 'hosted-engine --console' will open the serial console as well (wrapping socat) and it will also work if the vmconsole proxy or the engine (on auth side) are down while 'ssh -t -p 2222 ovirt-vmconsole@RHVM_FQDN connect --vm-name HostedEngine' requires a working engine VM.

Comment 6 Nikolai Sednev 2018-07-30 12:18:20 UTC
Works for me on latest appliance rhvm-appliance-4.2-20180727.1.el7.noarch, the serial console comes disabled by default, but if enabling it from UI and setting environment in to global maintenance and then powering off the engine and then starting it and then cancelling global maintenance, then serial console is working just fine on engine.
Engine setup within the rhvm-appliance-4.2-20180727.1.el7.noarch is ovirt-engine-setup-4.2.5.2-0.1.el7ev.noarch.
Tested on these components:
ovirt-hosted-engine-ha-2.2.16-1.el7ev.noarch
ovirt-hosted-engine-setup-2.2.24-1.el7ev.noarch
rhvm-appliance-4.2-20180727.1.el7.noarch
ovirt-vmconsole-1.0.4-1.el7ev.noarch
ovirt-vmconsole-host-1.0.4-1.el7ev.noarch
Linux 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux Server release 7.5 (Maipo)

Moving to verified.

Comment 8 errata-xmlrpc 2018-08-06 09:18:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2348