Bug 2042479 - Can't open guest console which has vnc+vga graphics on rhv4.4
Summary: Can't open guest console which has vnc+vga graphics on rhv4.4
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 4.4.10
Hardware: x86_64
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Liran Rotenberg
QA Contact: meital avital
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-19 15:27 UTC by mxie@redhat.com
Modified: 2022-04-08 10:46 UTC (History)
17 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-01-25 11:06:50 UTC
oVirt Team: Virt
Embargoed:
pm-rhel: blocker?


Attachments (Terms of Use)
cannot-open-vnc+vga-console-rhv4.4.png (189.16 KB, image/png)
2022-01-19 15:27 UTC, mxie@redhat.com
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-44503 0 None None None 2022-01-19 15:29:10 UTC

Description mxie@redhat.com 2022-01-19 15:27:07 UTC
Created attachment 1851864 [details]
cannot-open-vnc+vga-console-rhv4.4.png

Description of problem:
Can't open guest console which has vnc+vga graphics on rhv4.4 

Version-Release number of selected component (if applicable):
RHV-4.4.10.2-0.1.el8ev
RHV node:vdsm-4.40.100.2-1.el8ev.x86_64


How reproducible:
100%

Steps to Reproduce:
Scenario1: open rhv4.4 web on rhel8 or rhel9 server

1.1 Create a guest which has vnc+vga graphics on rhv4.4
1.2 Power on guest,click console option (open with remote-viewer) to connect guest console but the console can't open with error 'the certificate is not trusted', pls refer to screenshot'cannot-open-vnc+vga-console-rhv4.4.png'

Scenario2: open rhv4.4 web on fedora system
2.1 Create a guest which has vnc+vga graphics on rhv4.4
2.2 Power on guest, click console option (open with remote-viewer) to connect guest console but the console will be disappeared immediately

Scenario3: 
3.1 Try to open the console of guest which has vnc+vga graphics on another rhv4.4 env (4.4.10.2-0.1.el8ev), the console can't open with error 'Setting VM ticket failed.', pls refer to screenshot'set-ticket-failed'



Actual results:
As description

Expected results:
Can open guest console which has vnc+vga graphics on rhv4.4 

Additional info:
1.Not long ago, guest console which has vnc+vga can open successfully on rhv4.4,I'm not sure what's wrong, I can't open it now. I didn't find the root cause, as v2v will set vnc+vga as default graphic after converting to rhv, the bug has blocked v2v test, please help to fix the bug asap, thanks!

Comment 3 Francesco Romani 2022-01-19 15:30:36 UTC
ovirt-vmconsole is about the text console, not about the graphical console

Comment 4 Liran Rotenberg 2022-01-20 08:42:28 UTC
Hi Ming,
Can we get more details? On which platforms the engine/hosts are running? EL9, FC is not supported. EL8.6 is not tested yet.
Can you provide engine and vdsm logs, debug output of the remote-viewer, our generated console file from the engine(console.vv) and if the host/cluster is encrypted(in cluster view and in the host can be seen in /etc/libvirt/qemu.conf, vnc_tls=1 ).

Comment 5 Liran Rotenberg 2022-01-20 12:27:02 UTC
Just for the update, we saw today on libvirt 8 that there is BZ 1506689, which throw a warning in case the password of the token is > 8.
We sent 12 chars, the result is an error in our side. The UI shows 'Setting VM ticket failed.'

Comment 6 mxie@redhat.com 2022-01-20 13:38:06 UTC
(In reply to Liran Rotenberg from comment #4)
> Hi Ming,
> Can we get more details? On which platforms the engine/hosts are running?
> EL9, FC is not supported. EL8.6 is not tested yet.
> Can you provide engine and vdsm logs, debug output of the remote-viewer, our
> generated console file from the engine(console.vv) and if the host/cluster
> is encrypted(in cluster view and in the host can be seen in
> /etc/libvirt/qemu.conf, vnc_tls=1 ).

Our RHV4.4 env is based on rhel8.4, the host and cluster are not encrypted, attach console.vv, vdsm.log and engine.log of scenario1/2 rhv4.4 env

Comment 10 Liran Rotenberg 2022-01-20 15:51:01 UTC
I didn't find anything interesting. Everything looks OK to me.
What about the debug output of the remote-viewer?

Comment 11 mxie@redhat.com 2022-01-21 05:52:36 UTC
(In reply to Liran Rotenberg from comment #5)
> Just for the update, we saw today on libvirt 8 that there is BZ 1506689,
> which throw a warning in case the password of the token is > 8.
> We sent 12 chars, the result is an error in our side. The UI shows 'Setting
> VM ticket failed.'

I saw you can reproduce the scenario3 of comment0 on your rhv4.4 env, as the rhv4.4 env of comment0 scenario3 was set up by my collogue recently, I think we can focus on console error 'Setting VM ticket failed" firstly

Because can't download console.vv file of comment0 scenario3, collect remote viewer debug log of comment0 scenario1/2 as below

1.Download console.vv file of comment0 scenario1/2

2.Open the console.vv file on my fedora laptop
# rpm -q virt-viewer
virt-viewer-6.0-4.fc28.x86_64

# remote-viewer console.vv --debug --gtk-vnc-debug 
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.874: Opening display to console.vv
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.874: Guest (null) has a vnc display
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:06.875: vncconnection.c Init VncConnection=0x55e4ee1b1bb0
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:06.875: vncdisplaykeymap.c Using Wayland evdev virtual keycode mapping
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:06.875: vncdisplay.c Grab sequence is now Control_L+Alt_L
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.997: Spice foreign menu updated
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.997: After open connection callback fd=-1
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.997: Opening connection to display at console.vv
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.999: fullscreen display 0: 0
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:06.999: vncconnection.c Open host=10.73.195.48 port=5909
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:06.999: notebook show status 0x55e4ee182340
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.100: vncconnection.c Open coroutine starting
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.100: vncconnection.c Started background coroutine
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.100: vncconnection.c Resolving host 10.73.195.48 5909
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.100: vncconnection.c Trying one socket
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.100: vncconnection.c Socket pending
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.112: vncconnection.c Finally connected
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.112: vncconnection.c Emit main context 13
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.113: vncdisplay.c Grab sequence is now 
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.113: notebook show status 0x55e4ee182340
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.114: Insert display 0 0x55e4edfdbba0
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.114: notebook show status 0x55e4ee182340
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.114: vncdisplay.c Connected to VNC server
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.114: vncconnection.c Protocol initialization
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.114: vncconnection.c Read error Resource temporarily unavailable
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.127: Allocated 1024x740
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.127: Child allocate 1024x640
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.136: vncconnection.c Server version: 3.8
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.136: vncconnection.c Sending full greeting
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.137: vncconnection.c Using version: 3.8
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.137: vncconnection.c Read error Resource temporarily unavailable
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Possible auth 19
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Emit main context 11
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Thinking about auth type 19
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Decided on auth type 19
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Waiting for auth type
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Choose auth 19
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Checking if credentials are needed
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c No credentials required
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.148: vncconnection.c Read error Resource temporarily unavailable
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.158: vncconnection.c Read error Resource temporarily unavailable
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Possible VeNCrypt sub-auth 261
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Emit main context 12
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Requested auth subtype 261
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Waiting for VeNCrypt auth subtype
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Choose auth 261
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Checking if credentials are needed
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c No credentials required
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.169: vncconnection.c Read error Resource temporarily unavailable
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.180: vncconnection.c Do TLS handshake
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.221: vncconnection.c Checking if credentials are needed
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.221: vncconnection.c Want a TLS clientname
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.221: vncconnection.c Requesting missing credentials
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.225: vncconnection.c Emit main context 10
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.225: Got VNC credential request for 1 credential(s)
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.225: vncconnection.c Set credential 2 libvirt
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Searching for certs in /etc/pki
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Searching for certs in /root/.pki
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Failed to find certificate CA/cacert.pem
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c No CA certificate provided, using GNUTLS global trust
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Failed to find certificate CA/cacrl.pem
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Failed to find certificate libvirt/private/clientkey.pem
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Failed to find certificate libvirt/clientcert.pem
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Waiting for missing credentials
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c Got all credentials
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.227: vncconnection.c No CA certificate provided; trying the system trust store instead
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.322: vncconnection.c Using the system trust store and CRL
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.322: vncconnection.c No client cert or key provided
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.322: vncconnection.c No CA revocation list provided
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.322: vncconnection.c Handshake was blocking
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.337: vncconnection.c Handshake was blocking
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.401: vncconnection.c Handshake was blocking
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.402: vncconnection.c Handshake done
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.402: vncconnection.c Validating
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Error: The certificate is not trusted
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Emit main context 16
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncdisplay.c VNC server error
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Auth failed
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Doing final VNC cleanup
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Close VncConnection=0x55e4ee1b1bb0
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncconnection.c Emit main context 15
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.406: vncdisplay.c Disconnected from VNC server
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.406: Not removing main window 0 0x55e4edecc8f0
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncdisplay.c Grab sequence is now 
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.407: Disconnected
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.407: close vnc=0x55e4ee1a8250
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncconnection.c Init VncConnection=0x55e4ee41f050
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncdisplaykeymap.c Using Wayland evdev virtual keycode mapping
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncdisplay.c Grab sequence is now Control_L+Alt_L
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.407: notebook show status 0x55e4ee182340
(remote-viewer:19331): virt-viewer-DEBUG: 13:49:07.407: Guest (null) display has disconnected, shutting down
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncdisplay.c Display destroy, requesting that VNC connection close
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncdisplay.c Releasing VNC widget
(remote-viewer:19331): gtk-vnc-DEBUG: 13:49:07.407: vncconnection.c Finalize VncConnection=0x55e4ee41f050

Comment 12 Liran Rotenberg 2022-01-23 09:26:46 UTC
Based on the message it's the exact same we once had in the user-list[1].
This happened when encryption is on. I also saw a previous bug with the same problem by you[2].
Can we make sure the host is without encryption? Comment out vnc_tls in the qemu config file and restart libvirtd? Then try to trigger it again.
Another solution seems to copy the CA file.

As for scenario 3, it depends on the error he gets in engine/vdsm logs. But if he uses libvirt 8 it is probably the same thing we saw in our system tests on master.


[1] - https://lists.ovirt.org/archives/list/users@ovirt.org/thread/5XA5D7RSKI4VYH4J74CQNESGDNA5FYOJ/#MACDEEWMWOTPGHIJ24WTQI5KAL4TMYS7
[2] - https://bugzilla.redhat.com/show_bug.cgi?id=1751065

Comment 13 Liran Rotenberg 2022-01-23 09:44:39 UTC
And here is another bug[1]. All of them relates to the encryption, same logs from remote-viewer.

[1] - https://bugzilla.redhat.com/show_bug.cgi?id=1685929

Comment 14 mxie@redhat.com 2022-01-25 03:12:12 UTC
(In reply to Liran Rotenberg from comment #12)
> Based on the message it's the exact same we once had in the user-list[1].
> This happened when encryption is on. I also saw a previous bug with the same
> problem by you[2].
> Can we make sure the host is without encryption? Comment out vnc_tls in the
> qemu config file and restart libvirtd? Then try to trigger it again.

Thanks, guest console which has vnc+vga graphic can open on rhv4.4 successfully with this workaround

Comment 15 Liran Rotenberg 2022-01-25 11:06:50 UTC
Based on the above it's a known issue and there are workarounds to disable the VNC encryption or manual setting the CA cert in the right path for remote-viewer.
It won't be fixed from platform based on BZ 1751073.

As for the libvirt8 (scenario 3), we didn't switch to libvirt 8 yet.
Therefore, I'm closing this bug as won't fix.

Comment 16 Chris Law 2022-03-14 15:56:11 UTC
I'm currently experiencing this bug, it was closed as "won't fix" because "we didn't switch to libvirt 8 yet", however oVirt Node 4.4.10 base install does use libvirt 8. It looks like a fix was put in place here: https://github.com/oVirt/ovirt-engine/commit/a1e7e39348550b575f1f01b701105f9e1066b09f

However, I'm not sure this has been released into a production version of the engine?

Comment 17 Liran Rotenberg 2022-03-14 17:19:00 UTC
(In reply to Chris Law from comment #16)
> I'm currently experiencing this bug, it was closed as "won't fix" because
> "we didn't switch to libvirt 8 yet", however oVirt Node 4.4.10 base install
> does use libvirt 8. It looks like a fix was put in place here:
> https://github.com/oVirt/ovirt-engine/commit/
> a1e7e39348550b575f1f01b701105f9e1066b09f
> 
> However, I'm not sure this has been released into a production version of
> the engine?

If you use upstream with node you get libvirt 8, the commit you mentioned is only in master branch (upcoming 4.5).
In order to have this code in your engine you need to install master(Alpha of 4.5 coming this week).

Other way around is to downgrade libvirt to < 8.

Comment 18 Chris Law 2022-03-15 12:23:33 UTC
Just for info. I'm not using upstream with node! I'm using oVirt Node ISO (4.4.10) 3rd March from the following URL: https://resources.ovirt.org/pub/ovirt-4.4/iso/ovirt-node-ng-installer/ovirt-node-ng-installer-latest.iso which I believe links to: https://resources.ovirt.org/pub/ovirt-4.4/iso/ovirt-node-ng-installer/4.4.10-2022030308/el8/ovirt-node-ng-installer-4.4.10-2022030308.el8.iso

Just performed a test install of this iso, fresh install of it seems to be using libvirt 8. If I go back to a previous version 4.4.10 from Februry libvirt is 7.10

So suggest this is an actual bug in the recent release of oVirt Node on 3rd March?

Comment 19 zhoujunqin 2022-04-08 10:46:28 UTC
(In reply to Liran Rotenberg from comment #15)
> As for the libvirt8 (scenario 3), we didn't switch to libvirt 8 yet.
> Therefore, I'm closing this bug as won't fix.

I filed a separate Bug 2073375 to track this issue, the final version of RHEL with full RHV support will be RHEL 8.6.
So I think there should be a fix, thanks.


Note You need to log in before you can comment on or make changes to this bug.