Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1305338 - Issue with vdsm-hook-vmfex-dev-4.16.33-1 - "InvalidatedWeakRef"
Issue with vdsm-hook-vmfex-dev-4.16.33-1 - "InvalidatedWeakRef"
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm (Show other bugs)
3.5.7
x86_64 Linux
unspecified Severity urgent
: ovirt-4.0.0-alpha
: ---
Assigned To: Dan Kenigsberg
Michael Burman
: Regression
Depends On: 1308839
Blocks: 1305586
  Show dependency treegraph
 
Reported: 2016-02-07 04:53 EST by Michael Burman
Modified: 2016-05-01 05:20 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1305586 (view as bug list)
Environment:
Last Closed: 2016-04-25 10:26:41 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
vdsm logs (394.46 KB, application/x-gzip)
2016-02-07 04:54 EST, Michael Burman
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 53160 master MERGED vmfex hook: hold a ref for a virDomain object 2016-02-08 04:51 EST
oVirt gerrit 53200 ovirt-3.6 MERGED vmfex hook: hold a ref for a virDomain object 2016-02-08 08:23 EST
oVirt gerrit 53202 ovirt-3.6.3 MERGED vmfex hook: hold a ref for a virDomain object 2016-02-08 08:23 EST
oVirt gerrit 53203 ovirt-3.5 MERGED vmfex hook: hold a ref for a virDomain object 2016-02-08 08:32 EST
Red Hat Product Errata RHEA-2016:1671 normal SHIPPED_LIVE VDSM 4.0 GA bug fix and enhancement update 2016-09-02 17:32:03 EDT

  None (edit)
Description Michael Burman 2016-02-07 04:53:49 EST
Description of problem:
Issue with vdsm-hook-vmfex-dev-4.16.33-1 - "InvalidatedWeakRef"

It's not possible to use the latest vdsm-hook-vmfex-dev-4.16.33-1(3.5.8) for migration/hotplug/hotunplug 

vdsm.log migration error -->

Thread-965::DEBUG::2016-02-07 11:40:47,698::utils::776::root::(execCmd) FAILED: <err> = 'vmfex_dev hook: [unexpected error]: Traceback (most recent call last):\n  File "/usr/libexec/vdsm/hooks/before_device_migrat
e_destination/50_vmfex", line 179, in <module>\n    main()\n  File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 166, in main\n    handleDirectPool(libvirtconnection.get())\n  File "/u
sr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 110, in handleDirectPool\n    elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):\n  File "/usr/libexec/vdsm/hooks/before_device_migr
ate_destination/50_vmfex", line 84, in qbhInUse\n    domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))\n  File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper\n    ret
 = f(*args, **kwargs)\n  File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper\n    raise InvalidatedWeakRef()\nInvalidatedWeakRef\n\n\n'; <rc> = 2
Thread-965::INFO::2016-02-07 11:40:47,700::hooks::100::root::(_runHooksDir) vmfex_dev hook: [unexpected error]: Traceback (most recent call last):
  File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 179, in <module>
    main()
  File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 166, in main
    handleDirectPool(libvirtconnection.get())
  File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 110, in handleDirectPool
    elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):
  File "/usr/libexec/vdsm/hooks/before_device_migrate_destination/50_vmfex", line 84, in qbhInUse
    domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))
  File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper
    raise InvalidatedWeakRef()
InvalidatedWeakRef



Thread-965::ERROR::2016-02-07 11:40:47,701::BindingXMLRPC::1152::vds::(wrapper) unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1136, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 495, in vmMigrationCreate
    return vm.migrationCreate(params)
  File "/usr/share/vdsm/API.py", line 555, in migrationCreate
    if not v.waitForMigrationDestinationPrepare():
  File "/usr/share/vdsm/virt/vm.py", line 5812, in waitForMigrationDestinationPrepare
    dev._deviceXML, self.conf, dev.custom)
  File "/usr/share/vdsm/hooks.py", line 182, in before_device_migrate_destination
    vmconf=vmconf, params=customProperties)
  File "/usr/share/vdsm/hooks.py", line 110, in _runHooksDir
    raise HookError()
HookError

vdsm.log hotplug error -->

jsonrpc.Executor-worker-5::DEBUG::2016-02-07 11:44:01,728::utils::776::root::(execCmd) FAILED: <err> = 'vmfex_dev hook: [unexpected error]: Traceback (most recent call last):\n  File "/usr/libexec/vdsm/hooks/befor
e_nic_hotplug/50_vmfex", line 179, in <module>\n    main()\n  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 166, in main\n    handleDirectPool(libvirtconnection.get())\n  File "/usr/libexec/vdsm
/hooks/before_nic_hotplug/50_vmfex", line 110, in handleDirectPool\n    elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):\n  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 84, in qbhInU
se\n    domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))\n  File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper\n    ret = f(*args, **kwargs)\n  File "/usr/lib/pytho
n2.6/site-packages/vdsm/utils.py", line 1412, in wrapper\n    raise InvalidatedWeakRef()\nInvalidatedWeakRef\n\n\n'; <rc> = 2
jsonrpc.Executor-worker-5::INFO::2016-02-07 11:44:01,729::hooks::100::root::(_runHooksDir) vmfex_dev hook: [unexpected error]: Traceback (most recent call last):
  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 179, in <module>
    main()
  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 166, in main
    handleDirectPool(libvirtconnection.get())
  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 110, in handleDirectPool
    elif not qbhInUse(conn) and not isDirectPoolUpToDate(conn):
  File "/usr/libexec/vdsm/hooks/before_nic_hotplug/50_vmfex", line 84, in qbhInUse
    domxml = minidom.parseString(conn.lookupByID(vm).XMLDesc(0))
  File "/usr/lib/python2.6/site-packages/vdsm/libvirtconnection.py", line 119, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/vdsm/utils.py", line 1412, in wrapper
    raise InvalidatedWeakRef()
InvalidatedWeakRef



jsonrpc.Executor-worker-5::ERROR::2016-02-07 11:44:01,729::__init__::506::jsonrpc.JsonRpcServer::(_serveRequest) Internal server error
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/yajsonrpc/__init__.py", line 501, in _serveRequest
    res = method(**params)
  File "/usr/share/vdsm/rpc/Bridge.py", line 271, in _dynamicMethod
    result = fn(*methodArgs)
  File "/usr/share/vdsm/API.py", line 419, in hotplugNic
    return curVm.hotplugNic(params)
  File "/usr/share/vdsm/virt/vm.py", line 3492, in hotplugNic
    params=nic.custom)
  File "/usr/share/vdsm/hooks.py", line 266, in before_nic_hotplug
    params=params)
  File "/usr/share/vdsm/hooks.py", line 110, in _runHooksDir
    raise HookError()
HookError

Version-Release number of selected component (if applicable):
vdsm-4.16.33-1.el6ev.x86_64
3.5.7-0.1.el6ev


Steps to Reproduce:
1. Tr to use the latest vdsm-hook-vmfex-dev-4.16.33-1 hook for migration/hotplug


Actual results:
Failed with error "Unexpected exception"

Expected results:
Should work as expected
Comment 1 Michael Burman 2016-02-07 04:54 EST
Created attachment 1121866 [details]
vdsm logs
Comment 3 Michael Burman 2016-03-17 07:04:55 EDT
Verified on - 3.6.4-0.1.el6 and vdsm-4.17.23.1-0.el7ev.noarch
Comment 4 Eyal Edri 2016-04-25 10:26:41 EDT
3.6.5 released
Comment 5 Dan Kenigsberg 2016-05-01 04:58:36 EDT
bygone

Note You need to log in before you can comment on or make changes to this bug.