Description of problem: Cannot use nodedev-reset/reattach/detach command with split daemons Version-Release number of selected component (if applicable): libvirt-daemon-6.0.0-7.module+el8.2.0+5869+c23fe68b.x86_64 How reproducible: 100% Steps to Reproduce: 1. stop libvirtd and start split daemons: #systemctl stop libvirtd.service #systemctl stop libvirtd{,-ro,-admin,-tcp,-tls}.socket #for drv in qemu network nodedev nwfilter secret storage proxy; do systemctl stop virt${drv}d{,-ro,-admin}.socket; done #systemctl start virtnodedevd #systemctl start virtqemud 2. test nodedev-reset/reattach/detach command with nodedevd virsh -c nodedev:///system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # nodedev-reset pci_0000_ff_0f_4 error: Failed to reset device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceReset virsh # nodedev-reattach pci_0000_ff_0f_4 error: Failed to re-attach device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceReAttach virsh # nodedev-detach pci_0000_ff_0f_4 error: Failed to detach device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceDettach 3. test nodedev-reset/reattach/detach command with qemud # virsh -c qemu:///system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # nodedev-reset pci_0000_ff_0f_4 error: Failed to reset device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceReset virsh # nodedev-reattach pci_0000_ff_0f_4 error: Failed to re-attach device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceReAttach virsh # nodedev-detach pci_0000_ff_0f_4 error: Failed to detach device pci_0000_ff_0f_4 error: this function is not supported by the connection driver: virNodeDeviceDettach Actual results: Cannot use nodedev-reset/reattach/detach command with virtnodedevd or virtqemud Expected results: Support execute nodedev-reset/reattach/detach command with virtnodedevd or virtqemud Additional info:
This is fixed with these patches: https://www.redhat.com/archives/libvir-list/2020-March/msg00330.html
commit 744208e7db96c023f0c813708f5ed012f1ae17f8 Refs: v6.1.0-165-g744208e7db Author: Daniel P. Berrangé <berrange> AuthorDate: Mon Mar 9 11:59:35 2020 +0000 Commit: Daniel P. Berrangé <berrange> CommitDate: Tue Mar 17 13:58:20 2020 +0000 rpc: fix dispatch for node device APIs for virt drivers Despite their names, the following APIs: virNodeDeviceDettach virNodeDeviceDetachFlags virNodeDeviceReAttach virNodeDeviceReset are all handled by the virt drivers, not the node device driver. A bug in the RPC generator meant that these APIs were sent to the nodedev driver for handling. This caused breakage with the split daemons, since nothing was available to process them. Reviewed-by: Michal Privoznik <mprivozn> Signed-off-by: Daniel P. Berrangé <berrange> commit 69eee587da9648b5fe98ee7d4c7495d824d03203 Refs: v6.1.0-166-g69eee587da Author: Daniel P. Berrangé <berrange> AuthorDate: Tue Mar 10 16:40:47 2020 +0000 Commit: Daniel P. Berrangé <berrange> CommitDate: Tue Mar 17 13:58:20 2020 +0000 rpc: avoid name lookup when dispatching node device APIs The node device APIs are a little unusual because we don't use a "remote_nonnull_node_device" object on the wire, instead we just have a "remote_string" for the device name. This meant dispatcher code generation needed special cases. In doing so we mistakenly used the virNodeDeviceLookupByName() API which gets dispatched into the driver, instead of get_nonnull_node_device() which directly populates a virNodeDevicePtr object. This wasn't a problem with monolithic libvirtd, as the virNodeDeviceLookupByName() API call was trivially satisfied by the registered driver, albeit with an extra (undesirable) authentication check. With the split daemons, the call to virNodeDeviceLookupByName() fails in virtqemud, because the node device driver obviously doesn't exist in that daemon. Reviewed-by: Michal Privoznik <mprivozn> Signed-off-by: Daniel P. Berrangé <berrange> commit b8a3ed957e643077bbf3b4d8de5ca694903d7f93 Refs: v6.1.0-167-gb8a3ed957e Author: Daniel P. Berrangé <berrange> AuthorDate: Tue Mar 10 17:03:54 2020 +0000 Commit: Daniel P. Berrangé <berrange> CommitDate: Tue Mar 17 13:58:20 2020 +0000 qemu: lookup node device against nodedev driver before getting XML Some of the node device APIs are a little odd because they accept a virNodeDevicePtr object but are still implemented by the virt drivers. The first thing the virt drivers need to do is get the XML config associated with the node device, and that means talking to the node device driver. This worked previously because with monolithic libvirtd, both the virt driver and node device driver were in the same daemon and thus a single virConnectPtr can talk to both drivers. With the split daemon world though, the virNodeDevicePtr passed into the APIs is associated with the QEMU driver virConnectPtr, which has no ability to invoke APIs against the node device driver. We must thus get a duplicate virNodeDevicePtr object which is associated with a virConnectPtr for the node device driver. Reviewed-by: Michal Privoznik <mprivozn> Signed-off-by: Daniel P. Berrangé <berrange>
Test on libvirt-libs-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 with the steps in comment 0, the result is the same as in comment 0: # virsh -c nodedev:///system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # nodedev-reattach pci_0000_03_00_1 error: Failed to re-attach device pci_0000_03_00_1 error: this function is not supported by the connection driver: virNodeDeviceReAttach virsh # nodedev-detach pci_0000_03_00_1 error: Failed to detach device pci_0000_03_00_1 error: this function is not supported by the connection driver: virNodeDeviceDettach virsh # nodedev-reset pci_0000_03_00_1 error: Failed to reset device pci_0000_03_00_1 error: this function is not supported by the connection driver: virNodeDeviceReset virsh # quit
(In reply to yalzhang from comment #7) > Test on libvirt-libs-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 with the > steps in comment 0, the result is the same as in comment 0: > > # virsh -c nodedev:///system > Welcome to virsh, the virtualization interactive terminal. > > Type: 'help' for help with commands > 'quit' to quit > > virsh # nodedev-reattach pci_0000_03_00_1 > error: Failed to re-attach device pci_0000_03_00_1 > error: this function is not supported by the connection driver: > virNodeDeviceReAttach > > virsh # nodedev-detach pci_0000_03_00_1 > error: Failed to detach device pci_0000_03_00_1 > error: this function is not supported by the connection driver: > virNodeDeviceDettach > > virsh # nodedev-reset pci_0000_03_00_1 > error: Failed to reset device pci_0000_03_00_1 > error: this function is not supported by the connection driver: > virNodeDeviceReset Somewhat counter-intuitively, the nodedev-{reattach,.detach,reset} commands are actually provided by the QEMU driver, not the nodedev driver. So you need to use qemu:///system for this test
Thank you, Daniel. I should read the patches carefully before the testing. Test with QEMU driver, it works fine. 1. stop libvirtd and start split daemons: # systemctl stop libvirtd.service # systemctl stop libvirtd{,-ro,-admin,-tcp,-tls}.socket # for drv in qemu network nodedev nwfilter secret storage proxy; do systemctl stop virt${drv}d{,-ro,-admin}.socket; done # systemctl start virtqemud 2. test the nodedev-reset/detach/reattach command with QEMU drvier: # virsh -c qemu:///system Welcome to virsh, the virtualization interactive terminal. Type: 'help' for help with commands 'quit' to quit virsh # nodedev-reset pci_0000_05_00_0 Device pci_0000_05_00_0 reset virsh # nodedev-detach pci_0000_05_00_0 Device pci_0000_05_00_0 detached virsh # nodedev-reattach pci_0000_05_00_0 Device pci_0000_05_00_0 re-attached virsh #
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137