Bug 1964855
Summary: | 'systemctl stop libvirt-guests' hangs after libvirtd timeout | ||||||
---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Yanqiu Zhang <yanqzhan> | ||||
Component: | libvirt | Assignee: | Martin Kletzander <mkletzan> | ||||
libvirt sub component: | General | QA Contact: | Yanqiu Zhang <yanqzhan> | ||||
Status: | CLOSED ERRATA | Docs Contact: | |||||
Severity: | unspecified | ||||||
Priority: | unspecified | CC: | dtardon, dyuan, dzheng, fjin, jdenemar, lizhu, lmen, mkletzan, mprivozn, msekleta, pkrempa, systemd-maint-list, virt-maint, xuzhang, yafu, yanqzhan | ||||
Version: | 9.0 | Keywords: | Triaged | ||||
Target Milestone: | beta | ||||||
Target Release: | --- | ||||||
Hardware: | x86_64 | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | libvirt-8.9.0-1.el9 | Doc Type: | If docs needed, set a value | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | |||||||
: | 1964861 (view as bug list) | Environment: | |||||
Last Closed: | 2023-05-09 07:26:10 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | 8.9.0 | ||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1964861 | ||||||
Attachments: |
|
Description
Yanqiu Zhang
2021-05-26 08:20:47 UTC
Reproduces on latest build: libvirt-client-7.3.0-1.el9.x86_64 Can also reproduce by : start libvirt-guests -> stop libvirtd -> stop libvirt-guests. Honestly, this smells like a systemd bug to me. There's still libvirtd.socket which is supposed to spawn libvirtd upon connection. And we can see virsh is stuck in connecting to libvirtd. But it's never spawned to process the connection request. Let me switch over to systemd team to get their expertise. What does "systemctl status libvirtd.socket" say when this happens? Does "nc -zU /run/libvirt/libvirt-sock" start libvirtd.service? (In reply to David Tardon from comment #3) > What does "systemctl status libvirtd.socket" say when this happens? Does "nc > -zU /run/libvirt/libvirt-sock" start libvirtd.service? # systemctl status libvirtd.socket -l ● libvirtd.socket - Libvirt local socket Loaded: loaded (/usr/lib/systemd/system/libvirtd.socket; enabled; vendor preset: disabled) Active: active (running) since Wed 2021-05-19 00:57:18 EDT; 1 months 5 days ago Triggers: ● libvirtd.service Listen: /run/libvirt/libvirt-sock (Stream) Tasks: 0 (limit: 38031) Memory: 0B CPU: 0 CGroup: /system.slice/libvirtd.socket May 19 00:57:18 dell-* systemd[1]: Listening on Libvirt local socket. # nc -zU /run/libvirt/libvirt-sock # echo $? 0 # systemctl status libvirtd ● libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: inactive (dead) since Thu 2021-06-24 08:04:40 EDT; 2min 55s ago TriggeredBy: ● libvirtd-admin.socket ● libvirtd.socket ● libvirtd-ro.socket Docs: man:libvirtd(8) https://libvirt.org Process: 457048 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=exited, status=0/SUCCESS) Main PID: 457048 (code=exited, status=0/SUCCESS) Tasks: 2 (limit: 32768) Memory: 33.2M CPU: 547ms CGroup: /system.slice/libvirtd.service ├─3947419 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/li> └─3947420 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/li> Jun 24 08:04:23 dell-* libvirtd[457067]: libcap-ng used by "/usr/sbin/libvirtd" failed due to not havin> Jun 24 08:04:23 dell-* libvirtd[457068]: libcap-ng used by "/usr/sbin/libvirtd" failed due to not havin> Jun 24 08:04:23 dell-* dnsmasq[3947419]: read /etc/hosts - 2 addresses Jun 24 08:04:23 dell-* dnsmasq[3947419]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses Jun 24 08:04:23 dell-* dnsmasq-dhcp[3947419]: read /var/lib/libvirt/dnsmasq/default.hostsfile Jun 24 08:04:40 dell-* systemd[1]: Stopping Virtualization daemon... Jun 24 08:04:40 dell-* systemd[1]: libvirtd.service: Succeeded. Jun 24 08:04:40 dell-* systemd[1]: libvirtd.service: Unit process 3947419 (dnsmasq) remains running aft> Jun 24 08:04:40 dell-* systemd[1]: libvirtd.service: Unit process 3947420 (dnsmasq) remains running aft> Jun 24 08:04:40 dell-* systemd[1]: Stopped Virtualization daemon. And 'libvirt-guests stop' still hangs. This might be related to a stuck libvirt-guests script which is left there even if you killed the "systemctl stop libvirt-guests". It is running "virsh connect" which succeeded at connecting to the socket which was not picked up by a started daemon due to misconfiguration fixed in 1868537. After I killed processes connected to the socket(s) everything started to work again. Could you check that there is no virsh, libvirt-guests or anything else running that could be connected to the socket? Anyway this could be fixed if systemd closed the connection on the socket if it failed to start the triggered service, maybe once it stops trying. According to bz1868537#c20, if never executed 'stop libvirt-guests' before, 'start virtqemud' or 'virsh list' doesn't hang. Also, killing the 'virsh connect' process left by libvirt-guests also makes everything back to work. So it's better for libvirt-guests to work well by itself I think. virtqemud was inactive so no virsh or running vms, libvirt-guests yes its 'stop' called the 'virsh connect', I don't know where to find other things that could be connected to the socket. The problem is After= dependency between libvirt-guests.service (G) and virt-qemud.service (Q). Stop job on G will cause start of Q. However, start job for Q is not runnable because of the dependency. Generally, whenever we have B.service after A.service and we have stop job for B and start job for A systemd is always going to try to run stop job first and then start job. This is done in order to achieve more natural restart behaviour, this might seem unrelated but actually in systemd restart is just stop job which is executed and then turned into start job. Solution is to drop After= dependency on the virt-qemud.service as it is not needed. Ideally, you should After= on socket or on service but not both at the same time. Fixed upstream with v8.7.0-137-g59d30adacd1d: commit 59d30adacd1d5a4de8c25200ebfc666bd180fe1e Author: Martin Kletzander <mkletzan> Date: Tue Aug 30 08:29:53 2022 +0200 libvirt-guests: Fix dependency ordering in service file Tested on rhel9.2 with: libvirt-8.8.0-1.el9.x86_64 qemu-kvm-7.1.0-3.el9.x86_64 Steps: 1. man virtqemud If using libvirt-guests service then the ordering for that service needs to be adapted so that it is ordered after the service unit instead of the socket unit. Since dependencies and ordering cannot be changed with drop-in overrides, the whole libvirt-guests unit file needs to be changed. In order to preserve such change copy the installed /usr/lib/systemd/system/libvirt-guests.ser‐ vice to /etc/systemd/system/libvirt-guests.service and make the change there, specifically make sure the After= ordering mentions virtqemud.service and not virtqemud.socket: [Unit] After=virtqemud.service 2. # cat /usr/lib/systemd/system/libvirt-guests.service [Unit] Description=Suspend/Resume Running libvirt Guests Requires=virt-guest-shutdown.target After=network.target After=time-sync.target After=libvirtd.socket After=virtqemud.socket After=virtlxcd.socket After=virtvboxd.socket After=virtvzd.socket After=virtxend.socket After=virt-guest-shutdown.target Documentation=man:libvirt-guests(8) Documentation=https://libvirt.org [Service] EnvironmentFile=-/etc/sysconfig/libvirt-guests # Hack just call traditional service until we factor # out the code ExecStart=/usr/libexec/libvirt-guests.sh start ExecStop=/usr/libexec/libvirt-guests.sh stop Type=oneshot RemainAfterExit=yes StandardOutput=journal+console TimeoutStopSec=0 [Install] WantedBy=multi-user.target 3. Start&destroy libvirt-guests when virtqemud.service stopped but virtqemud.socket active, with default service file: # systemctl start virtqemud # ps aux|grep virtqemud root 82007 2.3 0.0 1581368 33416 ? Ssl 22:28 0:00 /usr/sbin/virtqemud --timeout 120 (Wait for it timeout) # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive # systemctl start libvirt-guests # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive # systemctl stop libvirt-guests (hangs here) # ps aux|grep virsh root 82062 0.5 0.0 325904 16036 ? Sl 22:32 0:00 virsh connect # kill -9 82062 # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active active # systemctl is-active libvirt-guests.service inactive 4.Start&destroy libvirt-guests when both virtqemud.service and virtqemud.socket inactive, with default service file: # systemctl start libvirt-guests # systemctl is-active virtqemud.socket inactive # systemctl is-active virtqemud.service inactive # systemctl is-active libvirt-guests.service active # systemctl stop libvirt-guests # 5.Modify service file as guided, # cp /usr/lib/systemd/system/libvirt-guests.service /etc/systemd/system/libvirt-guests.service # vim /etc/systemd/system/libvirt-guests.service (add After=virtqemud.service under [unit], and remove 'After=virtqemud.socket') # cat /etc/systemd/system/libvirt-guests.service|grep virtqemud After=virtqemud.service # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive # systemctl stop libvirt-guests (still hangs) 6.Modify /usr/lib/systemd/system/libvirt-guests.service directly: # cat /usr/lib/systemd/system/libvirt-guests.service|grep virtqemud After=virtqemud.service # systemctl daemon-reload # systemctl start libvirt-guests (hangs) (manually kill virsh connect) and # systemctl start virtqemud # systemctl start libvirt-guests # systemctl stop virtqemud # systemctl stop libvirt-guests (hangs) Hi Martin and Peter, In latest test, 'stop libvirt-guests' still hangs when virtqemud inactive and virtqemud.socket active. Could you check above tests&results please? Anything I missed or did wrongly? Thanks! Did you do systemctl daemon-reload after changing the service file? Not sure this needs to be done without libvirt-guests, whether it registers the change for the already running service, but that could be the only last missing piece, hopefully. Retested: # cp /usr/lib/systemd/system/libvirt-guests.service /etc/systemd/system/libvirt-guests.service # vim /etc/systemd/system/libvirt-guests.service # cat /etc/systemd/system/libvirt-guests.service|grep virtqemud After=virtqemud.service # systemctl daemon-reload # systemctl start libvirt-guests # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive # systemctl is-active libvirt-guests active # systemctl stop libvirt-guests hangs # ps aux|grep virsh root 90132 0.2 0.0 325904 16000 ? Sl 08:13 0:00 virsh connect # systemctl status libvirt-guests ● libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/etc/systemd/system/libvirt-guests.service; disabled; vendor preset: disabled) Active: deactivating (stop) since Mon 2022-10-17 08:13:30 EDT; 4min 59s ago ... Hi Martin, The test result is still the same: hangs. Please help confirm. Thank you! I'm not sure, is it possible to skip test_connect if virtqemud.service inactive when stop libvirt-guests? Oh, I see, I missed what you are trying to do there. The paragraph in the man page only relates to the masking of the sockets and switching to traditional service. I guess that could be more specific in the man page, it's also missing the fact that with traditional service users should remove the --timeout option from the service files. Basically you should only do the change if switching to traditional non-socket activated services. (In reply to Martin Kletzander from comment #13) But by default rhel9 is using socket activated services. Do you mean libvirt-guests can not be used in current mode? # cat /usr/lib/systemd/system/virtqemud.service|grep timeout #Environment=VIRTQEMUD_ARGS="--timeout 120" <---commented # systemctl daemon-reload 1.In current mode(socket activation): (1)if virtqemud.socket active and virtqemud.service inactive: # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive # systemctl start libvirt-guests # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active inactive <---start libvirt-guests didn't invoke virtqemud.service # systemctl stop libvirt-guests.service (still hangs) (2)if both virqemud.socket and virtqemud.service active: # systemctl is-active virtqemud.socket; systemctl is-active virtqemud.service active active # systemctl start libvirt-guests # ps aux|grep virtqemud root 91343 0.4 0.0 1581368 29396 ? Ssl 21:54 0:00 /usr/sbin/virtqemud # systemctl stop libvirt-guests # (succeed) 2. try to disable socket activation: # for drv in qemu interface network nodedev nwfilter secret storage proxy; do systemctl stop virt${drv}d{,-ro,-admin}.socket; systemctl mask virt${drv}d{,-ro,-admin}.socket;done # systemctl start virtqemud Failed to start virtqemud.service: Unit virtqemud-admin.socket is masked. --->So virtqemud does not support non-socket activation since it uses 'Requires=virtqemud.socket' instead of 'wants'. Could you tell a complete set of steps for current socket-activation mode please? And need this bug be set back to 'assigned'? Thank you! For socket activation mode there is nothing you need to do (with the patch included),that should just work. For traditional mode (no socket activation) there are (were) steps to be taken, but we decided that supporting non-socket activation for anything else than the modular libvirt daemon is not necessary, so we removed the docs on how to revert it (since it could have never worked since the introduction of modular daemons anyway). I think this BZ is done as it is, since the only issue is not related to libvirt-guests and is not supported now. (In reply to Martin Kletzander from comment #15) So the conclusion is: 1. For socket activation mode there is nothing need to do, since they have already used 'After=*.socket' instead of *.service. 2. For non-socket activation mode, need manually replace *.socket with *.service, since libvirt-guests.service unit file does not handle every mode now. 3.The usage of ‘modular daemons(like virtqemud) only working with socket activation mode‘ and 'monolithic daemon can work with both socket/non-socket activation modes' should be user well-known, nothing to concern by this fix. 4.The issue of libvirt-guests hanging is not fixed, user self need just ensure their virtqemud/libvirtd service will not timeout(remove the --timeout option) or manually start to make them alive when trying to use libvirt-guests.service. Are the above correct? Thanks! 1-3 are correct libvirt-guests should work correctly and not get stuck, even with the non-socket activated monolithic daemon What is the issue with the current state of libvirt-guests then? 1. monolithic daemon with non-socket, works well. # systemctl status libvirtd.socket ○ libvirtd.socket Loaded: masked (Reason: Unit libvirtd.socket is masked.) # systemctl status libvirt-guests ○ libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/etc/systemd/system/libvirt-guests.service; disabled; vendor preset: disabl> Active: inactive (dead) since Thu 2022-10-20 04:05:18 EDT; 2min 25s ago # cat /etc/systemd/system/libvirt-guests.service|grep libvirtd After=libvirtd.service # systemctl is-active libvirt-guests;systemctl is-active libvirtd inactive inactive # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd active inactive # systemctl stop libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd inactive inactive 2.monolithic daemon with socket activation: # systemctl status libvirt-guests ○ libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; disabled; vendor preset: di> # cat /usr/lib/systemd/system/libvirt-guests.service|grep libvirtd After=libvirtd.socket (1)start and stop libvirt-guests when libvirtd inactive, stop will still hang. # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket inactive inactive active # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active inactive active # systemctl stop libvirt-guests (hangs) # ps aux|grep virsh root 145478 0.2 0.0 325904 15860 ? Sl 04:16 0:00 virsh connect (2)stop libvirt-guests when libvirtd.service running, works well. # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket inactive active active # systemctl start libvirt-guests # systemctl stop libvirt-guests #(succeed) (3)stop libvirt-guests when libvirtd.service timeout, wil still hang. # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active active active # ps aux|grep libvirtd root 145499 0.5 0.0 1892384 52600 ? Ssl 04:17 0:00 /usr/sbin/libvirtd --timeout 120 (wait for it timeout) # ps aux|grep libvirtd (no libvirtd running) # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active inactive active # systemctl stop libvirt-guests (still hangs) # ps aux|grep virsh root 145673 0.1 0.0 325904 16052 ? Sl 04:22 0:00 virsh connect 3. modular daemon with socket activation: # systemctl status libvirt-guests ○ libvirt-guests.service - Suspend/Resume Running libvirt Guests Loaded: loaded (/usr/lib/systemd/system/libvirt-guests.service; disabled; vendor preset: disabled) # cat /usr/lib/systemd/system/libvirt-guests.service|grep virtqemud After=virtqemud.socket (1)start and stop libvirt-guests when virtqemud inactive, stop will still hang. # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive inactive active # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active inactive active # systemctl stop libvirt-guests (still hangs) # ps aux|grep virsh root 146343 0.0 0.0 325904 16092 ? Sl 04:37 0:00 virsh connect # pkill virsh # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive active active (2)stop libvirt-guests when virtqemud.service running, works well # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active active active # systemctl stop libvirt-guests # (succeed) (3)stop libvirt-guests when virtqemud.service timeout, wil still hang. # systemctl start libvirt-guests # ps aux|grep virtqemud root 146355 0.1 0.0 1581368 31140 ? Ssl 04:42 0:00 /usr/sbin/virtqemud --timeout 120 (wait for it timeout) # ps aux|grep virtqemud (no virtqemud service running) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active inactive active # systemctl stop libvirt-guests (still hangs) # ps aux|grep virsh root 146343 0.0 0.0 325904 16092 ? Sl 04:37 0:00 virsh connect From scenarios 2(1)(3), 3(1)(3), the issue same with comment0 still exists. The issue is: Stopping libvirt-guests will still hang when libvirtd/virtqemud service inactive, even though the libvirt-guests.service unit file only uses ‘After=libvirtd.socket/virtqemud.socket’ instead of ‘After=libvirtd.service/virtqemud.service’. The libvirtd/virtqemud service can only be started(automatically) after ‘stop libvirt-guests’ exits by ‘#pkill virsh’. Hi, I find that if remove 'Before=libvirt-guests.service' in virtqemud.service unit file, the 'hang' issue will disappear: Still under modular daemon with socket activation situation: # vim /usr/lib/systemd/system/virtqemud.service # systemctl daemon-reload # cat /usr/lib/systemd/system/virtqemud.service|grep libvirt-guests #Before=libvirt-guests.service <==comment it(equals to remove) Start libvirt-guests even when virtqemud not active: # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active inactive active # systemctl stop libvirt-guests (succeed) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive active active Is this missed in patch? But I'm not sure side effect for any other function scenarios. That was definitely missed, there is no need for virtqemud.service (or the other services) to have Before=libvirt-guests. I am not sure how I tested it that it worked, maybe I had a modification on the other service file. Thank you for also figuring out the cause. I sent a patch upstream, will update this BZ once something changes. Feel free to move this to assigned. Thanks Martin's confirmation. Patches applied: https://gitlab.com/libvirt/libvirt/-/commit/de842f37a1d7626c6e2d334f7966d17a75c5a81f https://gitlab.com/libvirt/libvirt/-/commit/86e27b3506cf2aacc86c4286c21623fbaf57e468 Moving to POST Tested with libvirt-v8.9.0-rc2-1-g72d4709ab9: 1.default modular daemon with socket activation: (1) start&destroy libvirt-guests while virtqemud.service inactive: # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive inactive active # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active inactive active # systemctl stop libvirt-guests (succeed) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive active active (2)destroy libvirt-guests when virtqemud.service timeout # systemctl start libvirt-guests # ps aux|grep virtqemud root 205266 0.0 0.0 1367228 21720 ? Ssl 04:08 0:00 /usr/sbin/virtqemud --timeout 120 (wait for it timeout) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active inactive active # systemctl stop libvirt-guests (succeed) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive active active (3)stop libvirt-guests when virtqemud.service running with vm, then start it when virtqemud timeout: # virsh start avocado-vt-vm1 Domain 'avocado-vt-vm1' started # virsh console avocado-vt-vm1 …localhost login: root Password: [root@localhost ~]# # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active active active # systemctl stop libvirt-guests (succeed) # virsh list --all --managed-save Id Name State ------------------------------ - avocado-vt-vm1 saved # ps aux|grep virtqemud root 206508 0.1 0.1 1530700 36164 ? Ssl 04:21 0:00 /usr/sbin/virtqemud --timeout 120 (wait for it timeout) # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket inactive inactive active # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active virtqemud; systemctl is-active virtqemud.socket active active active # virsh list --all --managed-save Id Name State -------------------------------- 1 avocado-vt-vm1 running # virsh console avocado-vt-vm1 …[root@localhost ~]# (vm restored) ---------------- 2.monolithic daemon with socket activation: (1)start&stop libvirt-guests when libvirtd.service inactive # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active inactive active # systemctl stop libvirt-guests (succeed) # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket inactive active active (2)stop libvirt-guests when libvirtd.service timeout # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active active active # ps aux|grep libvirtd root 210615 7.0 0.1 1559484 50236 ? Ssl 05:34 0:00 /usr/sbin/libvirtd --timeout 120 (wait for it timeout) # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active inactive active # systemctl stop libvirt-guests (succeed) # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket inactive active active (3)stop libvirt-guests when libvirtd.service running with vm, then start it when libvirtd timeout: # virsh start avocado-vt-vm1 Domain 'avocado-vt-vm1' started # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active active active # systemctl stop libvirt-guests (succeed) # virsh list --all --managed-save Id Name State ------------------------------ - avocado-vt-vm1 saved # ps aux|grep libvirtd root 210792 0.3 0.1 1641452 52416 ? Ssl 05:36 0:00 /usr/sbin/libvirtd --timeout 120 (wait for it timeout) # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket inactive inactive active # systemctl start libvirt-guests # systemctl is-active libvirt-guests;systemctl is-active libvirtd;systemctl is-active libvirtd.socket active active active # virsh list --all Id Name State -------------------------------- 1 avocado-vt-vm1 running PASS for these two scenarios. 4. Supplementary docs checking: (1)Check 'Before=libvirt-guests.service' removement: # grep libvirt-guests /usr/lib/systemd/system/virt*.service (nothing output) # grep libvirt-guests /usr/lib/systemd/system/libvirt*.service /usr/lib/systemd/system/libvirt-guests.service:Documentation=man:libvirt-guests(8) /usr/lib/systemd/system/libvirt-guests.service:EnvironmentFile=-/etc/sysconfig/libvirt-guests /usr/lib/systemd/system/libvirt-guests.service:ExecStart=/usr/libexec/libvirt-guests.sh start /usr/lib/systemd/system/libvirt-guests.service:ExecStop=/usr/libexec/libvirt-guests.sh stop (no Before=libvirt-guests.service anymore) # grep libvirt-guests /usr/lib/systemd/system/*.service /usr/lib/systemd/system/libvirt-guests.service:Documentation=man:libvirt-guests(8) /usr/lib/systemd/system/libvirt-guests.service:EnvironmentFile=-/etc/sysconfig/libvirt-guests /usr/lib/systemd/system/libvirt-guests.service:ExecStart=/usr/libexec/libvirt-guests.sh start /usr/lib/systemd/system/libvirt-guests.service:ExecStop=/usr/libexec/libvirt-guests.sh stop /usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service:Before=xendomains.service libvirtd.service libvirt-guests.service <---still exists /usr/lib/systemd/system/xenstored.service:Before=libvirtd.service libvirt-guests.service <---still exists (two corner texts left) (2)split daemon service manpage:(per commit f53988d657c6aafd7aa4f2070df01c43f1a936ab:“docs: Do not support non-socket activated modular daemons with systemd”) # man virtqemud ... DAEMON STARTUP MODES The virtqemud daemon is capable of starting in two modes. Socket activation mode On hosts with systemd it is started in socket activation mode and it will rely on systemd to create and listen on the UNIX sockets and pass them as pre-opened file descriptors. In this mode most of the socket related config options in /etc/lib‐ virt/virtqemud.conf will no longer have any effect. Traditional service mode On hosts without systemd, it will create and listen on UNIX sockets itself. ... (no "revert to the traditional" texts anymore) # grep 'revert to the traditional' libvirt/docs/manpages/ -Rin libvirt/docs/manpages/libvirtd.rst:74:OS that uses systemd. To revert to the traditional mode, all the socket (only libvirtd has the words.) Almost PASS. Hi martin, There're still two 'Before=libvirt-guests' left in 2 files per comment30 scenario4(1). But they don't affect our current test scope. Could you please modify them later? Thanks! (In reply to yanqzhan from comment #32) > Hi martin, > > There're still two 'Before=libvirt-guests' left in 2 files per comment30 > scenario4(1). But they don't affect our current test scope. Could you please > modify them later? Thanks! I'm not sure what package is the owner of those files, but to my knowledge that is not libvirt. Can you check it with rpm -qf /usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service /usr/lib/systemd/system/xenstored.service please? I can't seem to even install those. (In reply to Martin Kletzander from comment #33) Yes. # rpm -qf /usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service /usr/lib/systemd/system/xenstored.service xen-runtime-4.16.2-2.fc37.x86_64 xen-runtime-4.16.2-2.fc37.x86_64 Maybe they were installed on my host as a dependency when compiling libvirt source. Need they be modified or maybe not? (In reply to yanqzhan from comment #34) > (In reply to Martin Kletzander from comment #33) > Yes. > # rpm -qf /usr/lib/systemd/system/xen-qemu-dom0-disk-backend.service > /usr/lib/systemd/system/xenstored.service > xen-runtime-4.16.2-2.fc37.x86_64 > xen-runtime-4.16.2-2.fc37.x86_64 > > Maybe they were installed on my host as a dependency when compiling libvirt > source. > Need they be modified or maybe not? So I took a look at that and those services do not have any socket-activated options and we cannot depend on them, so I think it is correct the way it is. Tested with: libvirt-8.9.0-2.el9.x86_64 qemu-kvm-7.1.0-5.el9.x86_64 PASS with same steps and results as comment28, comment29, comment30. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (libvirt bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:2171 |