Bug 2196182

Summary: chardev tcp connection with tls disconnected after vm start
Product: Red Hat Enterprise Linux 9 Reporter: zhentang <zhetang>
Component: qemu-kvmAssignee: Marc-Andre Lureau <marcandre.lureau>
qemu-kvm sub component: General QA Contact: liunana <nanliu>
Status: ASSIGNED --- Docs Contact:
Severity: medium    
Priority: unspecified CC: berrange, chayang, jinzhao, juzhang, pkrempa, virt-maint, zhguo
Version: unspecifiedKeywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description zhentang 2023-05-08 09:15:06 UTC
Description of problem:
chardev tcp connection with tls disconnected after vm start

Version-Release number of selected component (if applicable):
libvirt-9.4.0-1.fc39.x86_64
qemu-kvm-8.0.0-3.fc39.x86_64

How reproducible:
100%

Steps to Reproduce:

scenario : start a guest with a chardev as a tcp client----------------------------
1.prepare tls env and start a server

# ll /etc/pki/libvirt-chardev/
total 36
-rw-r--r--. 1 root root 1448 May  4 06:31 ca-cert.pem
-rw-r--r--. 1 root root 8177 May  4 06:31 ca-key.pem
-rw-r--r--. 1 root root 1582 May  4 06:31 client-cert.pem
-rw-r--r--. 1 root root 8167 May  4 06:31 client-key.pem
-rw-r--r--. 1 root root 1582 May  4 06:31 server-cert.pem
-rw-r--r--. 1 root root 8177 May  4 06:31 server-key.pem

# gnutls-serv --echo --x509cafile /etc/pki/libvirt-chardev/ca-cert.pem --x509keyfile /etc/pki/libvirt-chardev/server-key.pem --x509certfile /etc/pki/libvirt-chardev/server-cert.pem
Processed 1 CA certificate(s).
Echo Server listening on IPv4 0.0.0.0 port 5556...done
Echo Server listening on IPv6 :: port 5556...done


2. modify qemu.conf
...
chardev_tls = 1
chardev_tls_x509_cert_dir = "/etc/pki/libvirt-chardev"
...

3. start a guest with a chardev as tcp client

# virsh dumpxml rhel9-bugverify --xpath //console
<console type="tcp">
  <source mode="connect" host="127.0.0.1" service="5556" tls="yes"/>
  <protocol type="raw"/>
  <target type="serial" port="0"/>
  <alias name="serial0"/>
</console>

# virsh start rhel9-bugverify 
Domain 'rhel9-bugverify' started


(on server side)
* Accepted connection from IPv4 127.0.0.1 port 34794 on Mon May  8 08:11:44 202
- Peer's certificate was NOT verified.
- Description: (TLS1.3-X.509)-(ECDHE-X25519)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
.......................


.......................

- Ephemeral EC Diffie-Hellman parameters
 - Using curve: X25519
 - Curve size: 256 bits
- Version: TLS1.3
- Server Signature: RSA-PSS-RSAE-SHA256
- Client Signature: RSA-PSS-RSAE-SHA256
- Cipher: AES-256-GCM
- MAC: AEAD
- Options: extended master secret, safe renegotiation,
- Channel bindings
 - 'tls-unique': not available
 - 'tls-server-end-point': 7fd09330c70a23c0cefa1a08153a2d775537662218391ce960dccf11cc6a8174
 - 'tls-exporter': 85b17844f9643d367f63effa480b916d0f95905e4d8ae9427749684245135ddf
Error while receiving data
Error: The TLS connection was non-properly terminated.








Actual results:
get error on server side and no boot info 

Error while receiving data
Error: The TLS connection was non-properly terminated.


Expected results:
client guest boot info displayed on server side.

Additional info:

Comment 1 Peter Krempa 2023-05-09 10:51:09 UTC
Moving to qemu. After startup of the VM the TLS communication is handled by qemu.

Comment 2 Marc-Andre Lureau 2023-05-10 07:30:13 UTC
Most likely, the certificate check is failing. 

Sadly, qemu doesn't report the error. I sent a patch for QEMU:
https://patchew.org/QEMU/20230510072531.3937189-1-marcandre.lureau@redhat.com/

@berrange, other idea?

Comment 3 John Ferlan 2023-06-22 11:16:08 UTC
Moving back to assigned in order to go through the proper RHEL downstream process in order to get the bug included downstream

Comment 5 Daniel Berrangé 2023-07-26 17:11:15 UTC
(In reply to Marc-Andre Lureau from comment #2)
> Most likely, the certificate check is failing. 
> 
> Sadly, qemu doesn't report the error. I sent a patch for QEMU:
> https://patchew.org/QEMU/20230510072531.3937189-1-marcandre.lureau@redhat.
> com/
> 
> @berrange, other idea?

These kind of things are almost always configuration mistakes, but diagnosing is a complete pain.

Using the qemu-trace-stap tool to watch 'qcrypto*' tracepoints is probably a good next step