Description of problem: VDSM does not properly handle disappearing devices. How reproducible: Race, "possibly". Steps to Reproduce: 1. hostdevListByCaps 2. before 1. finishes, detach a NIC from the host 3. (race) you may see or not see the failure depending on whether we already processed the device Actual results: 2017-03-01 10:55:20,038 ERROR (jsonrpc/0) [jsonrpc.JsonRpcServer] Internal server error (__init__:552) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 547, in _handle_request res = method(**params) File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 202, in _dynamicMethod result = fn(*methodArgs) File "/usr/share/vdsm/API.py", line 1415, in hostdevListByCaps devices = hostdev.list_by_caps(caps) File "/usr/lib/python2.7/site-packages/vdsm/hostdev.py", line 478, in list_by_caps libvirt_devices = _get_devices_from_libvirt(flags) File "/usr/lib/python2.7/site-packages/vdsm/hostdev.py", line 447, in _get_devices_from_libvirt __device_tree_hash(libvirt_devices) == _last_alldevices_hash): File "/usr/lib/python2.7/site-packages/vdsm/hostdev.py", line 129, in __device_tree_hash current_hash.update(device.XMLDesc(0)) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 5426, in XMLDesc if ret is None: raise libvirtError ('virNodeDeviceGetXMLDesc() failed') libvirtError: Node device not found: no node device with matching name 'net_vnet45_fe_1a_4a_23_14_af' Expected results: hostdevListByCaps output without the devices that are gone Additional info: Network devices are the ones most likely do be disappearing (e.g. detach of net's parent causes net to disappear).
Verification build: ovirt-engine-4.1.2-0.1.el7 vdsm-4.19.11-1.el7ev.x86_64 qemu-kvm-rhev-2.6.0-28.el7_3.9.x86_64 libvirt-client-2.0.0-10.el7_3.5.x86_64 sanlock-3.4.0-1.el7.x86_64 Verification scenario: 1. hostdevListByCaps 2. before 1. finishes, detach a NIC from the host using virsh nodedev-detach 3. Verify NIC detached is not listed. Open cdsm.log and verify no ERRORs related to this action.