Bug 1215436

Summary: Provide way to connect text only to HE during setup, virsh is failing.
Product: Red Hat Enterprise Virtualization Manager Reporter: Artyom <alukiano>
Component: ovirt-hosted-engine-setupAssignee: Simone Tiraboschi <stirabos>
Status: CLOSED CURRENTRELEASE QA Contact: Artyom <alukiano>
Severity: low Docs Contact:
Priority: low    
Version: 3.6.0CC: aburden, alukiano, bazulay, didi, gklein, jcall, lsurette, pstehlik, rbalakri, Rhev-m-bugs, sbonazzo, stirabos, yeylon, ykaul, ylavi
Target Milestone: ovirt-3.6.1Keywords: Regression
Target Release: 3.6.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
It is now possible to connect to the serial console of the engine virtual machine using a text-only connection when deploying the self-hosted engine. Instructions for connecting to the serial console are provided during the deployment process.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-17 07:23:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Integration RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1279774    
Bug Blocks:    

Description Artyom 2015-04-26 14:58:01 UTC
Description of problem:
Failed to connect to console under HE deployment
# virsh -c qemu+tls://Test/system console HostedEngine
error: failed to connect to the hypervisor
error: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory

Version-Release number of selected component (if applicable):
ovirt-hosted-engine-setup-1.3.0-0.0.master.20150401110307.git9665976.el7.noarch

How reproducible:
Always(deployment on clean host)

Steps to Reproduce:
1. Run hosted-engine --deploy
2. Run deployment until you need to connect to engine vm
[ INFO  ] Creating VM
          You can now connect to the VM with the following command:
                /bin/remote-viewer vnc://localhost:5900
          Use temporary password "4897nvVs" to connect to vnc console.
          Please note that in order to use remote-viewer you need to be able to run graphical applications.
          This means that if you are using ssh you have to supply the -Y flag (enables trusted X11 forwarding).
          Otherwise you can run the command from a terminal in your preferred desktop environment.
          If you cannot run graphical applications you can connect to the graphic console from another host or connect to the console using the following command:
          virsh -c qemu+tls://Test/system console HostedEngine
          If you need to reboot the VM you will need to start it manually using the command:
          hosted-engine --vm-start
          You can then set a temporary password using the command:
          hosted-engine --add-console-password
          The VM has been started.  Install the OS and shut down or reboot it.  To continue please make a selection:
         
          (1) Continue setup - VM installation is complete
          (2) Reboot the VM and restart installation
          (3) Abort setup
          (4) Destroy VM and abort setup

3. Try to connect to console from host via:
virsh -c qemu+tls://Test/system console HostedEngine

Actual results:
Connection failed with error message:
error: failed to connect to the hypervisor
error: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory

Expected results:
Connection success without any errors

Additional info:
Not sure about logs, because message pretty informative
looks like we try to connect via virsh before certificates generation

Comment 2 Yedidyah Bar David 2015-05-10 08:38:21 UTC
Artyom - remote-viewer does work for me. Changed the summary accordingly. Please ack. Thanks.

If it works for you too, that's a workaround, so I do not think it should be a blocker.

Comment 3 Artyom 2015-05-10 10:05:05 UTC
Yes, remote-viewer work fine(VNX and also SPICE), bug only related to terminal console(virsh -c qemu+tls://Test/system console HostedEngine)

Comment 4 Yedidyah Bar David 2015-05-26 05:43:27 UTC
Following an internal discussion, we decided to drop the 'virsh' text from this message, and officially support remote-viewer only.

Comment 8 Yedidyah Bar David 2015-07-01 09:25:30 UTC
Started documenting options in [1]. Need to add more options for local spicec, remote-viewer connecting directly (ssl/tls, perhaps also ssh), etc.

[1] http://www.ovirt.org/Hosted_Engine_Console

Comment 9 Simone Tiraboschi 2015-08-11 13:39:13 UTC
According to this http://www.ovirt.org/Serial_Console_Setup#I_need_to_access_the_console_the_old_way
socat connected to the console unix domain socket could do the job.

Comment 10 Yaniv Lavi 2015-08-16 10:52:09 UTC
(In reply to Simone Tiraboschi from comment #9)
> According to this
> http://www.ovirt.org/
> Serial_Console_Setup#I_need_to_access_the_console_the_old_way
> socat connected to the console unix domain socket could do the job.

So the two options will be virt-viewer for GUI and unix domain socket for text only. Looks good to me.

Comment 11 Yedidyah Bar David 2015-08-16 12:41:46 UTC
Note that it's not merely "text only" - the current patch supports serial console, not a vga text console. For this we'll need to add support for qemu's curses support.

Comment 13 Artyom 2015-09-20 16:31:16 UTC
Hi Simone, in time of verification I encounter number of problems:

1) to use serial console you need to have console=ttyS0 under kernel boot parameters, otherwise it will not work, I believe we need at least inform user about it(in most cases this options disabled), also we need to be sure that this options enable under rhevm-appliance(I know you checked CentOS 7.1 appliance and kernel boot parameters included console=ttyS0)
So Simone inform me if you want additional bug connect to message or you can fix it under this bug.

2) in my case(engine vm RHEL6.7), dumpxml looks like:
<console type='unix'>
      <source mode='bind' path='/var/run/ovirt-vmconsole-console/6aeca138-34c0-4b61-81c4-92ab235bf24b.sock'/>
      <target type='virtio' port='0'/>
      <alias name='console0'/>
    </console>
but I do not see element <serial>(do not sure if it must exist but under http://wiki.libvirt.org/page/Unable_to_connect_to_console_of_a_running_domain it appear)
it can be helpful to look on dumpxml of your vm(CentOS7 appliance) and in case if xml not correct I will additional bug.

Thanks

Comment 14 Simone Tiraboschi 2015-10-06 11:45:39 UTC
(In reply to Artyom from comment #13)

> 1) to use serial console you need to have console=ttyS0 under kernel boot
> parameters, otherwise it will not work, I believe we need at least inform
> user about it(in most cases this options disabled), also we need to be sure
> that this options enable under rhevm-appliance(I know you checked CentOS 7.1
> appliance and kernel boot parameters included console=ttyS0)
> So Simone inform me if you want additional bug connect to message or you can
> fix it under this bug.

I'll fix here

> 2) in my case(engine vm RHEL6.7), dumpxml looks like:
> <console type='unix'>
>       <source mode='bind'
> path='/var/run/ovirt-vmconsole-console/6aeca138-34c0-4b61-81c4-92ab235bf24b.
> sock'/>
>       <target type='virtio' port='0'/>
>       <alias name='console0'/>
>     </console>
> but I do not see element <serial>(do not sure if it must exist but under
> http://wiki.libvirt.org/page/
> Unable_to_connect_to_console_of_a_running_domain it appear)
> it can be helpful to look on dumpxml of your vm(CentOS7 appliance) and in
> case if xml not correct I will additional bug.

We have:
                {
                    'device': 'console',
                    'specParams': {
                        'enableSocket': 'true',
                    },
                    'type': 'console',
                    'deviceId': self.environment[
                        ohostedcons.VMEnv.CONSOLE_UUID
                    ],
                    'alias': 'console0'
                },
and it's working with ovirt engine appliance so I don't think that serial is mandatory here.

Comment 15 Sandro Bonazzola 2015-10-07 13:32:45 UTC
Postponing to 3.6.1, not a blocker for 3.6.0.

Comment 20 Yedidyah Bar David 2016-02-15 07:12:34 UTC
Perhaps somehow make it more explicit that you can connect to the *serial* console of a vm, not its vga one? Many OSes/liveCDs/etc require extra manual settings/configuration to use the serial console, or do not support it at all.

See also bug 505749, bug 501328, https://gerrit.ovirt.org/46700 .

Comment 23 Andrew Burden 2016-02-16 07:20:26 UTC
Thanks for flagging this, didi.
I've updated the doc text to call out the serial console twice, so hopefully that adequately highlights it as suggested.

Also, taking in John's comment, and since the ovirt wiki has an alternative, I've removed the recommendation for using socat and leave this up to user discretion. 

As always, let me know if any of this is in error :)

Cheers

Comment 24 Yedidyah Bar David 2016-02-16 07:36:24 UTC
I actually liked the previous version more I think... Perhaps something like:

It is now possible to connect to the serial console of the engine virtual machine using a text-only connection when deploying the self-hosted engine. Instructions about how to connect are provided during the deploy process.

Notes:
1. Feel free to drop the last sentence altogether. Not sure how important it is to make release notes "self-contained" vs "merely highlight changes/additions".
2. The link [1] is not to the wiki, but to the source code. This is what actually is displayed to the user.
IIRC we do not have it mentioned specifically on the wiki. The new serial console proxy, documented on the wiki at [2], is something different, and actually not really related to current bug: A serial console IIRC was provided also in previous versions (didn't check), and in this bug we merely changed the text telling how to use it. Whereas, [2] talks mainly about the proxy, or how to access this serial console from *remote* using ssh, centrally-managed, etc., which is not relevant during hosted-engine deploy (when you still have no engine, no central management, etc.)

[1] https://gerrit.ovirt.org/gitweb?p=ovirt-hosted-engine-setup.git;a=blob;f=src/ovirt_hosted_engine_setup/mixins.py;h=d6303aeca43d5552596988b04d384ceb10945a65;hb=HEAD#l336
[2] http://www.ovirt.org/Features/Serial_Console

Comment 25 Simone Tiraboschi 2016-02-16 08:54:39 UTC
(In reply to Yedidyah Bar David from comment #24)
> IIRC we do not have it mentioned specifically on the wiki. The new serial
> console proxy, documented on the wiki at [2], is something different, and
> actually not really related to current bug: A serial console IIRC was
> provided also in previous versions (didn't check), and in this bug we merely
> changed the text telling how to use it.

In this bug we just changed the text but the serial console type as changed for the serial console proxy (from VirtIO -> to Serial).

Looks at the first know issue here:
http://www.ovirt.org/Features/Serial_Console#Known_issues

Comment 26 Andrew Burden 2016-02-17 00:55:01 UTC
Aye aye :)

Doc text updated as suggested.
Re: Release Notes: They are more or less a list to highlight changes/additions, but in instances like this where it's low cost (one sentence) and provides potential value for the user (not having to go looking for more details; knowing it's a part of the deployment script) then we may as well include it.

Cheers!

Comment 27 Gil Klein 2016-02-17 07:23:51 UTC
This bug was fixed and is slated to be in the upcoming version. As we
are focusing our testing at this phase on severe bugs, this bug was
closed without going through its verification step. If you think this
bug should be verified by QE, please set its severity to high and move
it back to ON_QA