Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1533805

Summary: virt-viewer can't connect to guest by vnc+tls connection
Product: Red Hat Enterprise Linux 7 Reporter: Fangge Jin <fjin>
Component: virt-viewerAssignee: Virt Viewer Maint <virt-viewer-maint>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.5CC: berrange, dblechte, dyuan, fjin, fziglio, tzheng, victortoso, xiaodwan, xuzhang, zpeng
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-12-19 16:15:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
virt-viewer --gtk-vnc-debug none

Description Fangge Jin 2018-01-12 09:40:15 UTC
Description of problem:
virt-viewer can't connect to guest by vnc+tls connection

Version-Release number of selected component:
virt-viewer-5.0-10.el7.x86_64
libvirt-3.9.0-7.el7.x86_64
qemu-kvm-rhev-2.10.0-15.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Set up vnc tls environment(refer to linked Polarion Test Case)
# ll /etc/pki/libvirt
total 8
-rw-r--r--. 1 root root 741 Jan 10 18:53 clientcert.pem
drwxr-xr-x. 2 root root  48 Nov 10 17:53 private
-rw-r--r--. 1 root root 741 Jan 10 18:46 servercert.pem

# ll /etc/pki/libvirt/private/
total 8
-rw-r--r--. 1 root root 891 Jan 10 18:53 clientkey.pem
-rw-r--r--. 1 root root 887 Jan 10 18:46 serverkey.pem

# ll /etc/pki/libvirt-vnc/
total 0
lrwxrwxrwx. 1 root root 22 Jan 12 17:21 ca-cert.pem -> /etc/pki/CA/cacert.pem
lrwxrwxrwx. 1 root root 29 Jan 12 17:20 ca-key.pem -> /etc/pki/CA/private/cakey.pem
lrwxrwxrwx. 1 root root 31 Jan 12 17:26 server-cert.pem -> /etc/pki/libvirt/servercert.pem
lrwxrwxrwx. 1 root root 38 Jan 12 17:27 server-key.pem -> /etc/pki/libvirt/private/serverkey.pem


# openssl verify -verbose -CAfile /etc/pki/CA/cacert.pem /etc/pki/libvirt-vnc/server-cert.pem 
/etc/pki/libvirt-vnc/server-cert.pem: OK

# openssl verify -verbose -CAfile /etc/pki/CA/cacert.pem /etc/pki/libvirt/servercert.pem 
/etc/pki/libvirt/servercert.pem: OK

2. Start a guest with vnc graphic:
    ...
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    ...

3. Check qemu command line:
   # ps aux|grep vnc
   ... -vnc 0.0.0.0:0,tls,x509=/etc/pki/libvirt-vnc ...

4.Connect to guest by virt-viewer:
  # virt-viewer -c qemu+tls://fjin-4-179/system rhel7.4-simple --debug

Actual results:
In step4, virt-viewer quits immediately

Expected results:
Virt-viewer can connect to guest by vnc+tls connection


Additional info:
1. Guest can be connected by vncviewer successfully.

2. The debug info of virt-viewer:
(virt-viewer:23579): virt-viewer-DEBUG: connecting ...
(virt-viewer:23579): virt-viewer-DEBUG: Opening connection to libvirt with URI qemu+tls://fjin-4-179/system
(virt-viewer:23579): virt-viewer-DEBUG: initial connect
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: virt_viewer_app_set_uuid_string: UUID changed to f0ae725b-eb1c-4d3b-9e99-1468b5fdc5f8
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: Guest rhel7.4-simple is running, determining display
(virt-viewer:23579): virt-viewer-DEBUG: Set connect info: (null),(null),-1,-1,(null),(null),(null),0
(virt-viewer:23579): virt-viewer-DEBUG: Guest rhel7.4-simple has a vnc display
(virt-viewer:23579): virt-viewer-DEBUG: Guest graphics address is 0.0.0.0:5900
(virt-viewer:23579): virt-viewer-DEBUG: Guest graphics listen '0.0.0.0' is NULL or a wildcard, replacing with 'fjin-4-179'
(virt-viewer:23579): virt-viewer-DEBUG: Set connect info: fjin-4-179,fjin-4-179,5900,-1,tls,(null),(null),0
(virt-viewer:23579): virt-viewer-DEBUG: Error operation forbidden: read only access prevents virDomainOpenGraphicsFD
(virt-viewer:23579): virt-viewer-DEBUG: After open connection callback fd=-1
(virt-viewer:23579): virt-viewer-DEBUG: Opening direct TCP connection to display at fjin-4-179:5900:-1
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: reconnect_poll: 0
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: Insert display 0 0x560d7e95d680
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: Allocated 1024x740
(virt-viewer:23579): virt-viewer-DEBUG: Child allocate 1024x640

(virt-viewer:23579): Gtk-WARNING **: Allocating size to VncDisplay 0x560d7ec4e230 without calling gtk_widget_get_preferred_width/height(). How does the code know the size to allocate?
(virt-viewer:23579): virt-viewer-DEBUG: Not removing main window 0 0x560d7e9421a0
(virt-viewer:23579): virt-viewer-DEBUG: Disconnected
(virt-viewer:23579): virt-viewer-DEBUG: close vnc=0x560d7ec4e230
(virt-viewer:23579): virt-viewer-DEBUG: notebook show status 0x560d7eb86260
(virt-viewer:23579): virt-viewer-DEBUG: Guest rhel7.4-simple display has disconnected, shutting down

Comment 2 Erik Skultety 2018-02-01 12:05:01 UTC
I'm kind of failing to see how this is a libvirt issue, does a simple TLS connection to libvirtd work without any issues for you? Also, try running virt-viewer with --gtk-vnc-debug and if the problem are indeed certificates, then rerun with only GNUTLS_DEBUG_LEVEL=2 set (to filter out the rest of the logs for readability) and post the output.

Comment 3 Fangge Jin 2018-02-01 15:16:09 UTC
(In reply to Erik Skultety from comment #2)
> I'm kind of failing to see how this is a libvirt issue, does a simple TLS
> connection to libvirtd work without any issues for you? Also, try running
> virt-viewer with --gtk-vnc-debug and if the problem are indeed certificates,
> then rerun with only GNUTLS_DEBUG_LEVEL=2 set (to filter out the rest of the
> logs for readability) and post the output.

1) A simple TLS conection to libvirtd works
   # virt-viewer -c qemu+tls://fjin-4-179/system

2) Check the attachment for the output of "virt-viewer with --gtk-vnc-debug"

Comment 4 Fangge Jin 2018-02-01 15:16:52 UTC
Created attachment 1389599 [details]
virt-viewer --gtk-vnc-debug

Comment 5 Fangge Jin 2018-02-01 15:19:54 UTC
(In reply to Erik Skultety from comment #2)
> I'm kind of failing to see how this is a libvirt issue, does a simple TLS
> connection to libvirtd work without any issues for you? Also, try running
> virt-viewer with --gtk-vnc-debug and if the problem are indeed certificates,
> then rerun with only GNUTLS_DEBUG_LEVEL=2 set (to filter out the rest of the
> logs for readability) and post the output.

Maybe it is virt-viewer issue

Comment 6 Erik Skultety 2018-02-01 15:56:34 UTC
(In reply to Fangge Jin from comment #4)
> Created attachment 1389599 [details]
> virt-viewer --gtk-vnc-debug

gnutls[2]: crt_is_known: did not find cert, using issuer DN + serial, using DN only
gnutls[2]: crt_is_known: did not find any cert
gnutls[2]: crt_is_known: did not find cert, using issuer DN + serial, using DN only
gnutls[2]: crt_is_known: did not find any cert
(virt-viewer:21143): gtk-vnc-DEBUG: vncconnection.c Error: The certificate is not trusted

...There you go, you have problems with your certificates, I'd suggest using certtool to validate your client certificate, if it fails you know where the problem is...I tried with virt-manager and it failed identically when I followed the steps in your test case. There might be a bug in gnutls, but very likely the problem is with your certificates.

Comment 7 Fangge Jin 2018-02-02 03:30:16 UTC
(In reply to Erik Skultety from comment #6)
> (In reply to Fangge Jin from comment #4)
> > Created attachment 1389599 [details]
> > virt-viewer --gtk-vnc-debug
> 
> gnutls[2]: crt_is_known: did not find cert, using issuer DN + serial, using
> DN only
> gnutls[2]: crt_is_known: did not find any cert
> gnutls[2]: crt_is_known: did not find cert, using issuer DN + serial, using
> DN only
> gnutls[2]: crt_is_known: did not find any cert
> (virt-viewer:21143): gtk-vnc-DEBUG: vncconnection.c Error: The certificate
> is not trusted
> 
> ...There you go, you have problems with your certificates, I'd suggest using
> certtool to validate your client certificate, if it fails you know where the
> problem is...I tried with virt-manager and it failed identically when I
> followed the steps in your test case. There might be a bug in gnutls, but
> very likely the problem is with your certificates.

I don't know what's wrong about my certificates, I have all the certs needed and validated against the CA cert:

# ll /etc/pki/libvirt
total 8
-rw-r--r--. 1 root root 741 Jan 10 18:53 clientcert.pem
drwxr-xr-x. 2 root root  48 Nov 10 17:53 private
-rw-r--r--. 1 root root 741 Jan 10 18:46 servercert.pem


# ll /etc/pki/libvirt/private/
total 8
-rw-r--r--. 1 root root 891 Jan 10 18:53 clientkey.pem
-rw-r--r--. 1 root root 887 Jan 10 18:46 serverkey.pem

# ll /etc/pki/libvirt-vnc/
total 0
lrwxrwxrwx. 1 root root 22 Jan 12 17:21 ca-cert.pem -> /etc/pki/CA/cacert.pem
lrwxrwxrwx. 1 root root 29 Jan 12 17:20 ca-key.pem -> /etc/pki/CA/private/cakey.pem
lrwxrwxrwx. 1 root root 31 Jan 12 17:36 client-cert.pem -> /etc/pki/libvirt/clientcert.pem
lrwxrwxrwx. 1 root root 38 Jan 12 17:37 client-key.pem -> /etc/pki/libvirt/private/clientkey.pem
lrwxrwxrwx. 1 root root 31 Jan 12 17:26 server-cert.pem -> /etc/pki/libvirt/servercert.pem
lrwxrwxrwx. 1 root root 38 Jan 12 17:27 server-key.pem -> /etc/pki/libvirt/private/serverkey.pem


# ll /etc/pki/CA/
total 4
-rw-r--r--. 1 root root 863 Jan 12 17:04 cacert.pem
drwxr-xr-x. 2 root root   6 Dec  6 16:35 certs
drwxr-xr-x. 2 root root   6 Dec  6 16:35 crl
drwxr-xr-x. 2 root root   6 Dec  6 16:35 newcerts
drwx------. 2 root root  23 Jan 12 17:16 private


# ll /etc/pki/CA/private/
total 4
-rw-r--r--. 1 root root 963 Jan 12 17:05 cakey.pem


# openssl verify -verbose -CAfile /etc/pki/CA/cacert.pem /etc/pki/libvirt/servercert.pem
/etc/pki/libvirt/servercert.pem: OK

# openssl verify -verbose -CAfile /etc/pki/CA/cacert.pem /etc/pki/libvirt/clientcert.pem 
/etc/pki/libvirt/clientcert.pem: OK

Comment 8 Victor Toso 2018-12-19 15:58:15 UTC
Not sure how gtk-vnc handles CA, spice has extra command line options but... 

> (virt-viewer:21143): gtk-vnc-DEBUG: vncconnection.c Do TLS handshake
> (virt-viewer:21143): gtk-vnc-DEBUG: vncconnection.c No CA certificate provided; trying the system trust store instead

... seems to tell me that it is checking system wide CA, not sure if the paths above are included in the check.

Could you please confirm you still have this issue?

Comment 9 Daniel Berrangé 2018-12-19 16:05:52 UTC
This bug doesn't report the original gtk-vnc version, but given the date of the report, I expect is is probably a dupe of this GTK-VNC flaw: https://bugzilla.redhat.com/show_bug.cgi?id=1539148

Comment 10 Victor Toso 2018-12-19 16:15:52 UTC
Ah, many thanks Daniel.
I'm closing as DUP for now, please reopen if this is not fixed with gtk-vnc-0.7.0-3.el7 or above.

*** This bug has been marked as a duplicate of bug 1539148 ***