Bug 2232361

Tested on libvirt-9.6.0-1.fc39.x86_64 qemu-system-ppc-core-8.0.0-4.fc39.x86_64. No problem
# virsh dumpxml rhel-ppc64 --xpath //interface
<interface type="network">
  <mac address="52:54:00:a1:a5:96"/>
  <source network="default" portid="2aa31b6f-16d5-4fa2-bc04-d644afee71c9" bridge="virbr0"/>
  <target dev="vnet0"/>
  <model type="virtio"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</interface>

# virsh detach-interface rhel-ppc64 network --mac 52:54:00:a1:a5:96
Interface detached successfully


# virsh dumpxml rhel-ppc64 --xpath //interface

If device is not detached after a hot-unplug request it's usually caused by the guest OS not allowing the unplug. This can be either because the device is in use, or the OS is not responding. Since active cooperation with the guest OS is needed there is no way to force removal.

If you're able to reproduce the problem please attach debug logs of libvirtd, but based on my experience it'll be caused by the guest not allowing the removal. (Libvirt will sucessfully request the removal in qemu, which will also forward it to the OS but no acknowledgement will be seen).

Sorry for the late reply, here is the debug logs of libvirtd

So libvirt sends 'device_del' ...

2023-08-22 07:58:47.832+0000: 11056: info : qemuMonitorSend:868 : QEMU_MONITOR_SEND_MSG: mon=0x7fff4c1852f0 msg={"execute":"device_del","arguments":{"id":"net0"},"id":"libvirt-33"}
 fd=-1

... and qemu acknowledged it.

2023-08-22 07:58:47.834+0000: 11304: debug : qemuMonitorJSONIOProcessLine:225 : Line [{"return": {}, "id": "libvirt-33"}]

The DEVICE_DELETED event was never delivered, so the guest OS didn't acknowledge the unplug most likely. If you think there's a technical problem, you can move this bug to 'qemu' to investigate further, otherwise I'll close it.

Hi, I also try with x86_64, it seems that the DEVICE_DELETED event was delivered normally, so I think maybe moving the bug to 'qemu' to get more information is better, many thanks.

Reproduced this issue,and failed to hotunplug nic device on fedora38 guest.
Log:
(qemu) device_del nic0
Error: PCI device unplug already in progress for device nic0
Guest log:
[ 2181.084755] RTAS: event: 1, Type: Hotplug Event (229), Severity: 1
[root@dhcp16-215-248 ~]# lspci
00:00.0 VGA compatible controller: Device 1234:1111 (rev 02)
00:01.0 SCSI storage controller: Red Hat, Inc. Virtio SCSI
00:02.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:03.0 USB controller: Red Hat, Inc. QEMU XHCI Host Controller (rev 01)

Guest kernel:6.2.9-300.fc38.ppc64le
Host qemu-kvm:qemu-kvm-6.2.0-39.module+el8.9.0+19787+17a83bb7.ppc64le

Oops, updating here on Bugzilla, since this one isn't migrated to Jira yet.

Xujun,

1. Is the Fedora guest running an rtas_errd?  IIRC we need that for hotplug/unplug to be processed properly.

2. Whether or not that's the problem, the problem appears to be that the guest is not acknowledging the unplug request (comment 5).  That makes this a guest kernel bug - and therefore a Fedora bug - rather than a host or RHEL bug.  As you've noticed I barely have capacity to look at any RHEL ppc bugs, Fedora bugs are definitely not going to be investigated.

(In reply to David Gibson from comment #8)
> Oops, updating here on Bugzilla, since this one isn't migrated to Jira yet.
> 
> Xujun,
> 
> 1. Is the Fedora guest running an rtas_errd?  IIRC we need that for
> hotplug/unplug to be processed properly.
Not running rtas_errd in guest because not installed.
When I installed ppc64-diag-rtas,it's no problem to hotunplug device.
> 
> 2. Whether or not that's the problem, the problem appears to be that the
> guest is not acknowledging the unplug request (comment 5).  That makes this
> a guest kernel bug - and therefore a Fedora bug - rather than a host or RHEL
> bug.  As you've noticed I barely have capacity to look at any RHEL ppc bugs,
> Fedora bugs are definitely not going to be investigated.

looking at https://pagure.io/fedora-comps/blob/main/f/comps-f38.xml.in the package ppc64-diag-rtas is missing here, therefore it is not installed by default.

It's not a bug in ppc64-diag, but in fedora-comps. I also reported this issue to fedora-comps

Many thanks. Could you put the link about the mirror in fedora-comps so that I could also track the issue, just record it as a known issue in the automation/manual test.

the link about the mirror in fedora-comps:  https://pagure.io/fedora-comps/issue/889