Bug 1752837
| Summary: | libvirtd fails to start with TLS socket activation | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Advanced Virtualization | Reporter: | Daniel Berrangé <berrange> |
| Component: | libvirt | Assignee: | Daniel Berrangé <berrange> |
| Status: | CLOSED ERRATA | QA Contact: | Yanqiu Zhang <yanqzhan> |
| Severity: | high | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 8.1 | CC: | chhu, dzheng, jdenemar, knoel, lmen, mburman, xuzhang, yafu, yanqzhan, yicui |
| Target Milestone: | rc | Keywords: | Regression |
| Target Release: | 8.1 | Flags: | knoel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-5.6.0-6.el8 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-11-06 07:19:21 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
1. Not reproduces for tcp:
# systemctl stop libvirtd
Warning: Stopping libvirtd.service, but it can still be activated by:
libvirtd.socket
# systemctl start libvirtd-tcp.socket
# systemctl status libvirtd-tcp.socket
● libvirtd-tcp.socket - Libvirt non-TLS IP socket
Loaded: loaded (/usr/lib/systemd/system/libvirtd-tcp.socket; disabled; vendor preset: disabled)
Active: active (listening) since Wed 2019-09-18 07:30:49 EDT; 12s ago
Listen: [::]:16509 (Stream)
CGroup: /system.slice/libvirtd-tcp.socket
Sep 18 07:30:49 lenovo-*** systemd[1]: Listening on Libvirt non-TLS IP socket.
# virsh list
Id Name State
--------------------
# netstat -nltp|grep 16509
tcp6 0 0 :::16509 :::* LISTEN 1/systemd
# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2019-09-18 07:31:31 EDT; 1min 27s ago
2.Reproduces for tls, the libvirtd-tls.socket returns to inactive status when try to start the libvirtd.service:
# systemctl stop libvirtd
Warning: Stopping libvirtd.service, but it can still be activated by:
libvirtd.socket
# systemctl start libvirtd-tls.socket
# systemctl status libvirtd-tls.socket
● libvirtd-tls.socket - Libvirt TLS IP socket
Loaded: loaded (/usr/lib/systemd/system/libvirtd-tls.socket; disabled; vendor preset: disabled)
Active: active (listening) since Wed 2019-09-18 08:01:28 EDT; 4s ago
Listen: [::]:16514 (Stream)
CGroup: /system.slice/libvirtd-tls.socket
Sep 18 08:01:28 lenovo-*** systemd[1]: Listening on Libvirt TLS IP socket.
# netstat -nltp|grep 16514
tcp6 0 0 :::16514 :::* LISTEN 1/systemd
# virsh list
error: failed to connect to the hypervisor
error: Cannot recv data: Connection reset by peer
# netstat -nltp|grep 16514
# systemctl status libvirtd-tls.socket
● libvirtd-tls.socket - Libvirt TLS IP socket
Loaded: loaded (/usr/lib/systemd/system/libvirtd-tls.socket; disabled; vendor preset: disabled)
Active: failed (Result: service-start-limit-hit) since Wed 2019-09-18 08:02:53 EDT; 4s ago
Listen: [::]:16514 (Stream)
Sep 18 08:01:28 lenovo-*** systemd[1]: Listening on Libvirt TLS IP socket.
Sep 18 08:02:53 lenovo-*** systemd[1]: libvirtd-tls.socket: Failed with result 'service-start-limit-hit'.
# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2019-09-18 08:02:53 EDT; 46s ago
Docs: man:libvirtd(8)
https://libvirt.org
Process: 5383 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=6)
Main PID: 5383 (code=exited, status=6)
Tasks: 2 (limit: 32768)
Memory: 85.2M
CGroup: /system.slice/libvirtd.service
├─24364 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshe>
└─24365 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshe>
Sep 18 08:02:53 lenovo-*** systemd[1]: libvirtd.service: Service RestartSec=100ms expired, scheduling restart.
Sep 18 08:02:53 lenovo-*** systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 5.
Sep 18 08:02:53 lenovo-*** systemd[1]: Stopped Virtualization daemon.
Sep 18 08:02:53 lenovo-*** systemd[1]: libvirtd.service: Start request repeated too quickly.
Sep 18 08:02:53 lenovo-*** systemd[1]: libvirtd.service: Failed with result 'exit-code'.
Sep 18 08:02:53 lenovo-*** systemd[1]: Failed to start Virtualization daemon.
Here, if manually do "# systemctl start libvirtd" will make libvirtd.service back to work:
# virsh list
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
...
# virsh list
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
# systemctl start libvirtd
# virsh list
Id Name State
--------------------
Pkgs version for comment1: libvirt-daemon-5.6.0-5.module+el8.1.0+4229+2e4e348c.x86_64 qemu-kvm-4.1.0-10.module+el8.1.0+4234+33aa4f57.x86_64 Verify with:
libvirt-daemon-5.6.0-6.module+el8.1.0+4244+9aa4e6bb.x86_64
qemu-kvm-4.1.0-10.module+el8.1.0+4234+33aa4f57.x86_64
#cat /etc/libvirt/libvirtd.conf
auth_tls = "none"
And setup certificates.
# systemctl stop libvirtd
Warning: Stopping libvirtd.service, but it can still be activated by:
libvirtd-tls.socket
libvirtd.socket
libvirtd-tcp.socket
# systemctl start libvirtd-tls.socket
# virsh list
Id Name State
--------------------
# netstat -nltp|grep 16514
tcp6 0 0 :::16514 :::* LISTEN 1/systemd
# systemctl status libvirtd-tls.socket
● libvirtd-tls.socket - Libvirt TLS IP socket
Loaded: loaded (/usr/lib/systemd/system/libvirtd-tls.socket; disabled; vendor preset: disabled)
Active: active (listening) since Fri 2019-09-20 02:52:02 EDT; 14min ago
Listen: [::]:16514 (Stream)
CGroup: /system.slice/libvirtd-tls.socket
Sep 20 02:52:02 lenovo-*** systemd[1]: Listening on Libvirt TLS IP socket.
# virsh -c qemu+tls://lenovo-***/system
Welcome to virsh, the virtualization interactive terminal.
Type: 'help' for help with commands
'quit' to quit
virsh # quit
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:3723 *** Bug 1776323 has been marked as a duplicate of this bug. *** |
Description of problem: With new socket activation support, enable TLS and or TCP $ systemctl stop libvirtd.service ..setup certs... $ systemctl start libvirtd-tls.socket Then try to run virsh $ virsh list error: failed to connect to the hypervisor error: Cannot recv data: Connection reset by peer # systemctl status libvirtd ● libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2019-09-17 10:31:22 BST; 6s ago Docs: man:libvirtd(8) https://libvirt.org Process: 6034 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=6) Main PID: 6034 (code=exited, status=6) Sep 17 10:31:21 localhost.localdomain systemd[1]: Failed to start Virtualization daemon. Sep 17 10:31:22 localhost.localdomain systemd[1]: libvirtd.service: Service RestartSec=100ms expired, scheduling restart. Sep 17 10:31:22 localhost.localdomain systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 5. Need two fixes from upstream commit 522b3d2b24d0f7ac78dad442c990d4e34db0eaf2 Author: Michael Chapman <mike.org> Date: Tue Sep 17 17:03:57 2019 +1000 remote: fix registration of TLS socket Reviewed-by: Daniel P. Berrangé <berrange> Signed-off-by: Michael Chapman <mike.org> Version-Release number of selected component (if applicable): libvirt-daemon-5.6.0-3.module+el8.1.0+4110+a6d45c3d.x86_64 How reproducible: Always