Bug 1483309 - startVM fails with OVN network
Summary: startVM fails with OVN network
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: vdsm
Classification: oVirt
Component: Core
Version: ---
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.2.0
: ---
Assignee: Francesco Romani
QA Contact: Mor
URL:
Whiteboard:
Depends On:
Blocks: 988285
TreeView+ depends on / blocked
 
Reported: 2017-08-20 10:57 UTC by Mor
Modified: 2019-04-28 13:41 UTC (History)
8 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-20 10:42:00 UTC
oVirt Team: Virt
Embargoed:
rule-engine: ovirt-4.2+
rule-engine: blocker+


Attachments (Terms of Use)
vdsm supervdsm and engine logs (2.13 MB, application/zip)
2017-08-20 10:57 UTC, Mor
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 81195 0 master MERGED virt: tests: enhance test for OVS NIC with engine XML 2017-09-18 12:08:59 UTC
oVirt gerrit 81286 0 master POST virt: metadata: per-device custom properties 2017-08-31 15:31:58 UTC
oVirt gerrit 81382 0 master POST core: enable ovn with engine xml 2017-09-03 13:46:14 UTC
oVirt gerrit 81389 0 master POST virt: consume custom properties of nics with engine xml 2017-09-04 07:04:40 UTC

Description Mor 2017-08-20 10:57:48 UTC
Created attachment 1315927 [details]
vdsm supervdsm and engine logs

Description of problem:
VM with OVN network attached fails to start.

vdsm.log shows the following error:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 117, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 301, in destroy
    res = self.vm.destroy(gracefulAttempts)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 129, in vm
    raise exception.NoSuchVM(vmId=self._UUID)
NoSuchVM: Virtual machine does not exist: {'vmId': '0cb1b7a9-c7b2-401e-9c0f-3f46257e5154'}

Version-Release number of selected component (if applicable):
vdsm-api-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-hook-vmfex-dev-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-http-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-yajsonrpc-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-client-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-jsonrpc-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-python-4.20.2-90.git6511af5.el7.centos.noarch
vdsm-4.20.2-90.git6511af5.el7.centos.x86_64
openvswitch-ovn-common-2.7.2-3.1fc27.el7.x86_64
ovirt-provider-ovn-driver-1.1-2.20170818071729.git634477f.el7.centos.noarch
openvswitch-ovn-host-2.7.2-3.1fc27.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. Use engine-setup and answer-file to install oVirt engine and OVN.
2. Install ovirt-provider-ovn-driver on host.
3. Create OVN network and attach the network to a new vNIC on VM.
4. Start the VM.

Actual results:
VM fails to start, engine and vdsm report an error.

Expected results:
Should start normally.

Comment 1 Francesco Romani 2017-08-31 09:57:46 UTC
plain OVS works with
vdsm-hook-vmfex-dev-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-client-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-python-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-jsonrpc-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-api-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-4.20.2-122.git87f120b.el7.centos.x86_64
vdsm-yajsonrpc-4.20.2-122.git87f120b.el7.centos.noarch
vdsm-http-4.20.2-122.git87f120b.el7.centos.noarch


Vdsm produces:

        <interface type="bridge">
            <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci" />
            <mac address="$MAC" />
            <model type="virtio" />
            <source bridge="vdsmbr_6gnlz2dB" />
            <virtualport type="openvswitch" />
            <filterref filter="vdsm-no-mac-spoofing" />
        </interface>

virsh dumpxml:

    <interface type='bridge'>
      <mac address='$MAC'/>
      <source bridge='vdsmbr_6gnlz2dB'/>
      <virtualport type='openvswitch'>
        <parameters interfaceid='e74663f0-b181-4a53-a5a8-117ee26f0d24'/>
      </virtualport>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <filterref filter='vdsm-no-mac-spoofing'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

ovs-vsctl show:
ac17af3c-1a10-4740-a920-fbc3c20c9205
    Bridge "vdsmbr_6gnlz2dB"
        Port "vnet0"
            Interface "vnet0"
        Port "vdsmbr_6gnlz2dB"
            Interface "vdsmbr_6gnlz2dB"
                type: internal
        Port ovirtmgmt
            Interface ovirtmgmt
                type: internal
        Port "eth0"
            Interface "eth0"
    ovs_version: "2.7.0"

so looks like the issue is related to ovirt-provider-ovn-driver?
On the host on which I tried, we have:

# rpm -qa | grep ovn
(nothing)
# rpm -qa | grep openvswitch
openvswitch-2.7.0-1.el7.x86_64

Dan/Edward: I'm not sure how to proceed further. Maybe just installing the missing packages and try again? any configuration needed on the host?

Comment 2 Dan Kenigsberg 2017-08-31 10:44:48 UTC
Let us leave switchType=ovs aside, for a while. I do not think it is anyway related.

I don't see in the attached vdsm.log that ovirt-provider-ovn-driver's hook was actually executed. Do you?

Comment 3 Francesco Romani 2017-08-31 11:34:14 UTC
(In reply to Dan Kenigsberg from comment #2)
> Let us leave switchType=ovs aside, for a while. I do not think it is anyway
> related.
> 
> I don't see in the attached vdsm.log that ovirt-provider-ovn-driver's hook
> was actually executed. Do you?

Nope, I don't see the hook being called.
I reviewed the code to doublecheck we still call the hooks in the engine xml flow, and we do. That part is not supposed to change.

Comment 4 Francesco Romani 2017-09-01 07:52:42 UTC
(In reply to Francesco Romani from comment #3)
> (In reply to Dan Kenigsberg from comment #2)
> > Let us leave switchType=ovs aside, for a while. I do not think it is anyway
> > related.
> > 
> > I don't see in the attached vdsm.log that ovirt-provider-ovn-driver's hook
> > was actually executed. Do you?
> 
> Nope, I don't see the hook being called.
> I reviewed the code to doublecheck we still call the hooks in the engine xml
> flow, and we do. That part is not supposed to change.

I was half-wrong.
The code to call the hook is indeed there, but that code is triggered only if the device has some custom properties. And currently we have none, because Vdsm doesn't know where to find them

Patch https://gerrit.ovirt.org/#/c/81286/1 will make Vdsm consume the custom properties, but to fix this bug, Engine needs to send them.

Comment 5 Mor 2017-09-07 06:12:57 UTC
The patches are not merged yet.

Comment 6 Arik 2017-09-07 06:28:11 UTC
(In reply to Mor from comment #5)
> The patches are not merged yet.

I wonder if a test actually failed or is it only because of the tracking of the patches here in bugzilla... the patches are in and should be available for you in the nightly build already, this bz has not been updated with the right status of patches for some reason.

Comment 7 Francesco Romani 2017-09-11 07:57:05 UTC
regression in pre-alpha. Customers should never seen this, hence no need for doc_text.

Comment 8 Mor 2017-09-11 10:41:06 UTC
Verified:
4.2.0-0.0.master.20170906185835.gitcee3e58.el7.centos
ovirt-provider-ovn-1.1-2.20170905145914.gitaf4fe87.el7.centos.noarch
ovirt-provider-ovn-driver-1.1-2.20170905145914.gitaf4fe87.el7.centos.noarch

Comment 9 Sandro Bonazzola 2017-12-20 10:42:00 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.