Bug 1478054 - VM fails to run with all hooks
VM fails to run with all hooks
Status: CLOSED CURRENTRELEASE
Product: vdsm
Classification: oVirt
Component: General (Show other bugs)
4.20.0
Unspecified Unspecified
high Severity urgent (vote)
: ovirt-4.2.0
: 4.20.0
Assigned To: Yaniv Bronhaim
Raz Tamir
: Automation, Regression
Depends On:
Blocks: 1479484
  Show dependency treegraph
 
Reported: 2017-08-03 09:49 EDT by Mor
Modified: 2017-12-20 06:40 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-12-20 06:40:43 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.2+
rule-engine: blocker+


Attachments (Terms of Use)
logs (745.88 KB, application/octet-stream)
2017-08-06 02:56 EDT, Mor
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 80341 master ABANDONED hooking: change vdsm hooks imports 2017-08-19 15:53 EDT
oVirt gerrit 80444 master MERGED Revert "hooking: move to lib" 2017-08-10 07:53 EDT

  None (edit)
Description Mor 2017-08-03 09:49:32 EDT
Created attachment 1308744 [details]
logs

Description of problem:
VM with OVN network attached fails to run.

vdsm log report the following error:
2017-08-03 16:30:09,359+0300 ERROR (vm/0cb1b7a9) [virt.vm] (vmId='0cb1b7a9-c7b2-401e-9c0f-3f46257e5154') The vm start process failed (vm:808)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 742, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2411, in _run
    dom.createWithFlags(flags)
  File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 125, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 586, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1069, in createWithFlags
    if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
libvirtError: Cannot get interface MTU on 'ovn_net_4': No such device

ovirt-provider-ovn-driver hooks are installed on the VM host.

Version-Release number of selected component (if applicable):
openvswitch-ovn-common-2.7.2-3.1fc27.el7.x86_64
openvswitch-ovn-host-2.7.2-3.1fc27.el7.x86_64
ovirt-provider-ovn-driver-1.1-2.20170710074946.git5abef7f.el7.centos.noarch
openvswitch-ovn-central-2.7.2-3.1fc27.el7.x86_64
ovirt-provider-ovn-1.1-2.20170710074946.git5abef7f.el7.centos.noarch


How reproducible:
100%

Steps to Reproduce:
1. Use engine-setup and answer-file to install oVirt engine and OVN.
2. Install ovirt-provider-ovn-driver on host.
3. Configure using vdsm-tool: vdsm-tool ovn-config {provider_ip} {host_ip}
4. Create OVN network and attach the network to a new vNIC on VM.
5. Start the VM.

Actual results:
VM fails to start and vdsm reports an error.

Expected results:
Should start normally.

Additional info:
Comment 1 Mor 2017-08-06 02:56 EDT
Created attachment 1309585 [details]
logs
Comment 2 Mor 2017-08-08 03:07:02 EDT
Please note that this bug was opened originally for OVN. We also need to fix the path in the OVN hook.
Comment 3 Red Hat Bugzilla Rules Engine 2017-08-09 02:28:46 EDT
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Comment 4 Dan Kenigsberg 2017-08-09 02:32:21 EDT
(In reply to Mor from comment #2)
> Please note that this bug was opened originally for OVN. We also need to fix
> the path in the OVN hook.

No, we need to fix Vdsm so that `import hooking` inside a hook would still work. We shouldn't force all 3rd party hooks out there to suddenly change without warning.
Comment 5 Yaniv Bronhaim 2017-08-09 06:45:01 EDT
I verified that revert for https://gerrit.ovirt.org/#/c/80444/ fixes the hooks run issue. this revert should get in asap before moving on. seems like we have issues with specific hooks that not related to the hooking mechanism. 

Once the move to lib will be developed fulfully we will re-post "hooking: move to lib".

Please check if with the revert the hook works properly or not, and if not - check if the issue is in this hook specifically or any hook.
I verified that other hooks are indeed being read.
Comment 6 Nir Soffer 2017-08-09 15:28:21 EDT
Maybe the issue is caused by stale /usr/share/vdsm/hooking.pyc?
Comment 7 Yaniv Bronhaim 2017-08-10 05:11:55 EDT
I'm not sure how to fix the move to lib/vdsm right now, it requires checking and verification. for sure not related to hooking.pyc there because I don't see any try for loading hooks at all. maybe it worked during testing while composing this patch because of stale pyc. sounds more reasonable to me.
 I prefer to revert it until we'll see that the move is properly done. seems like there is another issue related to specific hook and I prefer to expose that instead of creating new issue
Comment 8 Meni Yakove 2017-08-10 07:50:31 EDT
After Yaniv patch, we can see that the hooks are called but we have a few that still not working and we opened bugs for them.

https://bugzilla.redhat.com/show_bug.cgi?id=1479808
https://bugzilla.redhat.com/show_bug.cgi?id=1479796
Comment 9 Yaniv Bronhaim 2017-08-20 04:11:28 EDT
Moving to MODIFIES - as I see that this bug was only about the regression and not about the specific hook itself. for that we have separate bugs
Comment 10 Mor 2017-08-21 03:03:36 EDT
Verified on: 4.2.0-0.0.master.20170820180837.git59243e9.el7.centos
Comment 11 Sandro Bonazzola 2017-12-20 06:40:43 EST
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Note You need to log in before you can comment on or make changes to this bug.