Updated to systemd 253.5 today and libvirtd/virtqemud seem to be dying and causing virsh commands to hang. I generally spin up new VMs with virt-install and use virsh domifaddr to get the IP after they boot. They boot fine, run my given task to completion and then do cleanup where they query virsh again. At this point they hang and virtqemud log says: Jun 07 11:02:00 cloud-dev01 systemd[1]: Starting virtqemud.service - Virtualization qemu daemon... Jun 07 11:02:00 cloud-dev01 systemd[1]: Started virtqemud.service - Virtualization qemu daemon. Jun 07 11:58:49 cloud-dev01 virtqemud[3817]: libvirt version: 9.0.0, package: 3.fc38 (Fedora Project, 2023-05-07-17:31:32, ) Jun 07 11:58:49 cloud-dev01 virtqemud[3817]: hostname: cloud-dev01 Jun 07 11:58:49 cloud-dev01 virtqemud[3817]: End of file while reading data: Input/output error Jun 07 11:59:52 cloud-dev01 virtqemud[3817]: End of file while reading data: Input/output error Restarting the libvirtd/virtqemud goes back to the beginning of the process. I have also tried removing the socket activation from libvirtd. Reproducible: Always Steps to Reproduce: 1. Install new VM using vert-install. 2. Boot VM and use for a while (20-30 minutes). 3. Use any virsh command and have it hang. Actual Results: virsh command hangs and virtqemud gives I/O error. Expected Results: virsh command should work.
I downgraded my systemd packages to 253.4 and we're back to not hanging, so that's good at least.
I've added links to systemd and libvirt's issue trackers where someone else has reported this problem.
I can confirm what Shane describes ... exactly the same thing here : systemd 253.5 is the culprit - after downgrading the systemd packages everything works as expected again. @Shane : Can you add the component "systemd" to your bug report ? I am not able to do this. Another related bug report -> https://bugzilla.redhat.com/show_bug.cgi?id=2213398
I cannot add a second component it seems; sorry, I'm not super familiar with BugZilla. It's possible that this could just be a systemd bug instead, there seems to be some confusion on the matter.
I seem to have the same issue with: systemd-253.5-1.fc38.x86_64 libvirt-daemon-driver-qemu-9.2.0-1.fc38.x86_64
It looks like a PR has been merged upstream. I don't know the etiquette on this, whether I should leave this open until it gets to Fedora or just close now.
*** This bug has been marked as a duplicate of bug 2213660 ***