Bug 1818554 - Libvirt service fails to start
Summary: Libvirt service fails to start
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 4.4.0
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ovirt-4.4.0
: 4.4.0
Assignee: Yuval Turgeman
QA Contact: cshao
URL:
Whiteboard:
: 1797287 1817352 (view as bug list)
Depends On:
Blocks: 1817352
TreeView+ depends on / blocked
 
Reported: 2020-03-29 09:59 UTC by Gal Zaidman
Modified: 2020-08-04 13:28 UTC (History)
18 users (show)

Fixed In Version: vdsm-4.40.13
Doc Type: Removed functionality
Doc Text:
The current version of RHV removes libvirt packages that provided non-socket activation. Now it contains only libvirt versions that provide socket activation. Socket activation provides better resource handling: There is no dedicated active daemon; libvirt is activated for certain tasks and then exits.
Clone Of:
Environment:
Last Closed: 2020-08-04 13:27:53 UTC
oVirt Team: Integration
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2020:3246 0 None None None 2020-08-04 13:28:24 UTC
oVirt gerrit 108070 0 master ABANDONED systemctl: Add 'is_enabled' function 2020-11-16 11:27:18 UTC
oVirt gerrit 108071 0 master ABANDONED tool: libvirt: Use systemctl's 'is-enabled' to check units 2020-11-16 11:27:18 UTC
oVirt gerrit 108072 0 master MERGED tool: libvirt: Drop support for libvirt without socket activation 2020-11-16 11:27:19 UTC
oVirt gerrit 108098 0 master MERGED systemd: Do not enable vdsmd after installation 2020-11-16 11:27:18 UTC
oVirt gerrit 108152 0 master MERGED tool: libvirt: Don't shutdown libvirtd on timeout 2020-11-16 11:27:40 UTC
oVirt gerrit 108153 0 master MERGED tool: libvirt: Remove deprecated libvirt settings 2020-11-16 11:27:20 UTC
oVirt gerrit 108154 0 master MERGED tool: libvirt: Dynamic socket requirements 2020-11-16 11:27:19 UTC

Description Gal Zaidman 2020-03-29 09:59:48 UTC
Description of problem:

When installing RHVH libvirtd fails to start

[root@hosted-engine ~]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/libvirtd.service.d
           └─unlimited-core.conf
   Active: failed (Result: exit-code) since Sun 2020-03-29 09:30:42 UTC; 34s ago
     Docs: man:libvirtd(8)
           https://libvirt.org
  Process: 9675 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=6)
 Main PID: 9675 (code=exited, status=6)

Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Main process exited, code=exited, status=6/NOTCONFIGURED
Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Failed with result 'exit-code'.
Mar 29 09:30:42 hosted-engine systemd[1]: Failed to start Virtualization daemon.
Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Service RestartSec=100ms expired, scheduling restart.
Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 5.
Mar 29 09:30:42 hosted-engine systemd[1]: Stopped Virtualization daemon.
Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Start request repeated too quickly.
Mar 29 09:30:42 hosted-engine systemd[1]: libvirtd.service: Failed with result 'exit-code'.
Mar 29 09:30:42 hosted-engine systemd[1]: Failed to start Virtualization daemon.

----------------------
looking into libvirt config at: /etc/sysconfig/libvirtd we can see the --listen arg is added by VDSM

# If systemd socket activation is disabled, then the following
# can be used to listen on TCP/TLS sockets
#LIBVIRTD_ARGS="--listen"
....
## beginning of configuration section by vdsm-4.40.0
DAEMON_COREFILE_LIMIT=unlimited
LIBVIRTD_ARGS=--listen
## end of configuration section by vdsm-4.40.0

this flag should not be there, look at the docs at: https://libvirt.org/manpages/libvirtd.html
"""
-l, --listen

Listen for TCP/IP connections. This should not be set if using systemd socket activation. Instead activate the libvirtd-tls.socket or libvirtd-tcp.socket unit files.
""" 

After I run vdsm-tool configure --force the line is removed and libvirt starts normally:
## beginning of configuration section by vdsm-4.40.0
DAEMON_COREFILE_LIMIT=unlimited
## end of configuration section by vdsm-4.40.0


Version-Release number of selected component (if applicable):
RHVH-4.4-20200325.0-RHVH-x86_64-dvd1.iso

Steps to Reproduce:
1. Install RHVH-4.4-20200325.0-RHVH-x86_64-dvd1.iso
2. run systemctl start libcvirtd

Comment 1 Yuval Turgeman 2020-03-29 10:23:52 UTC
libvirtd is not configured in RHVH, it will be configured when adding it as a host.  If you want to do it manually, you can run vdsm-tool configure --force.  I don't think it's a bug

Comment 2 Gal Zaidman 2020-03-29 11:09:36 UTC
(In reply to Yuval Turgeman from comment #1)
> libvirtd is not configured in RHVH, it will be configured when adding it as
> a host.  If you want to do it manually, you can run vdsm-tool configure
> --force.  I don't think it's a bug

Installing the hosted engine with cockpit require libvirtd to be running (or at least able to start).
When libvirtd cant start we get this:
https://bugzilla.redhat.com/show_bug.cgi?id=1817352

Comment 3 Michal Skrivanek 2020-03-30 10:01:25 UTC
(In reply to Gal Zaidman from comment #0)
> looking into libvirt config at: /etc/sysconfig/libvirtd we can see the --listen arg is added by VDSM

then the auto detection code must have failed.

Marcin, can't we drop it altogether? We don't need to support any older libvirt anyway, so why not just drop the whole section?

Comment 4 Yuval Turgeman 2020-03-30 10:51:29 UTC
Yes, it failed, Marcin and I discussed this, and you're probably right we should drop it, but there's another issue - the post install scripts run inside a chroot, so vdsm-tool which calls `systemctl show` will not work as well, unless we set the hidden environment variable SYSTEMD_IGNORE_CHROOT, which is probably bad.
If we could replace the calls from show to "is-active/is-enabled" in vdsm-tool it should work

Comment 5 Sandro Bonazzola 2020-04-07 07:46:19 UTC
*** Bug 1817352 has been marked as a duplicate of this bug. ***

Comment 6 Marcin Sobczyk 2020-04-07 08:20:33 UTC
(In reply to Michal Skrivanek from comment #3)
> (In reply to Gal Zaidman from comment #0)
> > looking into libvirt config at: /etc/sysconfig/libvirtd we can see the --listen arg is added by VDSM
> 
> then the auto detection code must have failed.
> 
> Marcin, can't we drop it altogether? We don't need to support any older
> libvirt anyway, so why not just drop the whole section?

Yes, the patches I posted completely drop support for libvirt that doesn't have socket activation.

Comment 9 cshao 2020-04-09 02:20:21 UTC
Test version:
redhat-virtualization-host-4.4.0-20200408.1.el8_2
vdsm-4.40.13-1.el8ev.x86_64

Test steps:
1. Install RHVH-4.4-20200408.0-RHVH-x86_64-dvd1.iso via anaconda GUI.
2. run systemctl start libcvirtd
3. Register to engine(4.4.0-0.26.master.el8ev).

Test result:
1. Libvirt service can start succeed.
2. Register to engine can succeed.

# systemctl start libvirtd
[root@dhcp-8-122 ~]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/libvirtd.service.d
           └─unlimited-core.conf
   Active: active (running) since Thu 2020-04-09 02:17:55 UTC; 55s ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 1475 (libvirtd)
    Tasks: 19 (limit: 32768)
   Memory: 46.0M
   CGroup: /system.slice/libvirtd.service
           ├─1475 /usr/sbin/libvirtd --timeout 120
           ├─1976 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper
           └─1979 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/libvirt_leaseshelper

Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: started, version 2.79 cachesize 150
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: compile time options: IPv6 GNU-getopt DBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth DNSSEC loop-detect inotify
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq-dhcp[1976]: DHCP, IP range 192.168.124.2 -- 192.168.124.254, lease time 1h
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq-dhcp[1976]: DHCP, sockets bound exclusively to interface virbr0
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: reading /etc/resolv.conf
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: using nameserver 10.72.17.5#53
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: using nameserver 10.68.5.26#53
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: read /etc/hosts - 2 addresses
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq[1976]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Apr 09 02:17:56 dhcp-8-122.nay.redhat.com dnsmasq-dhcp[1976]: read /var/lib/libvirt/dnsmasq/default.hostsfile

Comment 10 Marcin Sobczyk 2020-06-25 08:24:42 UTC
*** Bug 1797287 has been marked as a duplicate of this bug. ***

Comment 15 Rolfe Dlugy-Hegwer 2020-08-02 16:59:06 UTC
The Doc Text states this is removed functionality. Therefore, I'm changing "Doc Type" (Release Note Type) parameter value to "Removed functionality"

Comment 17 errata-xmlrpc 2020-08-04 13:27:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHV RHEL Host (ovirt-host) 4.4), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:3246


Note You need to log in before you can comment on or make changes to this bug.