Description of problem: Issue with vdsm-hook-vmfex-dev-4.16.33-1 - "InvalidatedWeakRef" It's not possible to use the latest vdsm-hook-vmfex-dev-4.16.33-1(3.5.8) for migration/hotplug/hotunplug vdsm.log migration error --> Thread-965::DEBUG::2016-02-07 11:40:47,698::utils::776::root::(execCmd) FAILED: <err> = 'vmfex_dev hook: [unexpected error]: Traceback (most recent call last):\n File "/usr/libexec/vdsm/hooks/before_device_migrat e_destination/50_vmfex", line 179, in <module>\n main()\n File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 166, in main\n handleDirectPool(libvirtconnection.get())\n File "/u sr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 110, in handleDirectPool\n elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):\n File "/usr/libexec/vdsm/hooks/before_device_migr ate_destination/50_vmfex", line 84, in qbhInUse\n domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))\n File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper\n ret = f(*args, **kwargs)\n File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper\n raise InvalidatedWeakRef()\nInvalidatedWeakRef\n\n\n'; <rc> = 2 Thread-965::INFO::2016-02-07 11:40:47,700::hooks::100::root::(_runHooksDir) vmfex_dev hook: [unexpected error]: Traceback (most recent call last): File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 179, in <module> main() File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 166, in main handleDirectPool(libvirtconnection.get()) File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 110, in handleDirectPool elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn): File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 84, in qbhInUse domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0)) File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper raise InvalidatedWeakRef() InvalidatedWeakRef Thread-965::ERROR::2016-02-07 11:40:47,701::BindingXMLRPC::1152::vds::(wrapper) unexpected error Traceback (most recent call last): File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1136, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 495, in vmMigrationCreate return vm.migrationCreate(params) File "/usr/share/vdsm/API.py", line 555, in migrationCreate if not v.waitForMigrationDestinationPrepare(): File "/usr/share/vdsm/virt/vm.py", line 5812, in waitForMigrationDestinationPrepare dev._deviceXML, self.conf, dev.custom) File "/usr/share/vdsm/hooks.py", line 182, in before_device_migrate_destination vmconf=vmconf, params=customProperties) File "/usr/share/vdsm/hooks.py", line 110, in _runHooksDir raise HookError() HookError vdsm.log hotplug error --> jsonrpc.Executor-worker-5::DEBUG::2016-02-07 11:44:01,728::utils::776::root::(execCmd) FAILED: <err> = 'vmfex_dev hook: [unexpected error]: Traceback (most recent call last):\n File "/usr/libexec/vdsm/hooks/befor e_nic_hotplug/50_vmfex", line 179, in <module>\n main()\n File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 166, in main\n handleDirectPool(libvirtconnection.get())\n File "/usr/libexec/vdsm /hooks/before_nic_hotplug/50_vmfex", line 110, in handleDirectPool\n elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):\n File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 84, in qbhInU se\n domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))\n File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper\n ret = f(*args, **kwargs)\n File "/usr/lib/pytho n2.6/site-packages/vdsm/utils.py", line 1412, in wrapper\n raise InvalidatedWeakRef()\nInvalidatedWeakRef\n\n\n'; <rc> = 2 jsonrpc.Executor-worker-5::INFO::2016-02-07 11:44:01,729::hooks::100::root::(_runHooksDir) vmfex_dev hook: [unexpected error]: Traceback (most recent call last): File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 179, in <module> main() File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 166, in main handleDirectPool(libvirtconnection.get()) File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 110, in handleDirectPool elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn): File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 84, in qbhInUse domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0)) File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper raise InvalidatedWeakRef() InvalidatedWeakRef jsonrpc.Executor-worker-5::ERROR::2016-02-07 11:44:01,729::__init__::506::jsonrpc.JsonRpcServer::(_serveRequest) Internal server error Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/yajsonrpc/__init__.py", line 501, in _serveRequest res = method(**params) File "/usr/share/vdsm/rpc/Bridge.py", line 271, in _dynamicMethod result = fn(*methodArgs) File "/usr/share/vdsm/API.py", line 419, in hotplugNic return curVm.hotplugNic(params) File "/usr/share/vdsm/virt/vm.py", line 3492, in hotplugNic params=nic.custom) File "/usr/share/vdsm/hooks.py", line 266, in before_nic_hotplug params=params) File "/usr/share/vdsm/hooks.py", line 110, in _runHooksDir raise HookError() HookError Version-Release number of selected component (if applicable): vdsm-4.16.33-1.el6ev.x86_64 3.5.7-0.1.el6ev Steps to Reproduce: 1. Tr to use the latest vdsm-hook-vmfex-dev-4.16.33-1 hook for migration/hotplug Actual results: Failed with error "Unexpected exception" Expected results: Should work as expected
Created attachment 1121866 [details] vdsm logs
Verified on - 3.6.4-0.1.el6 and vdsm-4.17.23.1-0.el7ev.noarch
3.6.5 released
bygone