Bug 1551971

Adding some logging to /usr/libexec/vdsm/hooks/before_device_create/ovirt_provider_ovn_hook shows that it is executed and does it job. It seems that vdsm ignores its output and uses the Engine-generated device xml instead. Thus I believe that this is a recent virt regression.

final iface <interface type="bridge"><address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/><mac address="00:00:00:00:0
0:20"/><model type="virtio"/><source bridge="br-int"/><filterref filter="vdsm-no-mac-spoofing"/><boot order="2"/><alias name="ua-77b53de8-9
7d5-4c90-96ca-6ab12b34f96f"/><virtualport type="openvswitch"><parameters interfaceid="11572e42-47fb-4372-9b3d-7a86aa21081c"/></virtualport>
</interface>

The problem does not occur on hotplug.

This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

vmfex effected as well, wrong interface type is passed in the xml

Patch https://gerrit.ovirt.org/#/c/88547/ is supposed to fix this. RPMs: http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-on-demand-el7-x86_64/821/

(In reply to Francesco Romani from comment #4)
> Patch https://gerrit.ovirt.org/#/c/88547/ is supposed to fix this. RPMs:
> http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-on-demand-el7-
> x86_64/821/

Hi Francesco,
I have verified the ovn + vmfex flow with this patch(will give you +1)

Hi

VM with SR-IOV vNIC can't start as well, does it the same issue/bug? 

2018-03-15 10:27:05,788+0200 ERROR (vm/f9bd0e85) [virt.vm] (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') The vm start process failed (vm:940)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 869, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2829, in _run
    dom = self._connection.defineXML(domxml)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 130, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3676, in defineXML
    if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)
libvirtError: XML error: non unique alias detected: ua-04c2decd-4e33-4023-84de-a2205c777af7
2018-03-15 10:27:05,789+0200 INFO  (vm/f9bd0e85) [virt.vm] (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') Changed state to Down: XML error: non unique alias detected: ua-04c2decd-4e33-4023-84de-a2205c777af7 (code=1) (vm:1677)

(In reply to Michael Burman from comment #6)
> Hi
> 
> VM with SR-IOV vNIC can't start as well, does it the same issue/bug? 
> 
> 2018-03-15 10:27:05,788+0200 ERROR (vm/f9bd0e85) [virt.vm]
> (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') The vm start process failed
> (vm:940)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 869, in
> _startUnderlyingVm
>     self._run()
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2829, in _run
>     dom = self._connection.defineXML(domxml)
>   File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py",
> line 130, in wrapper
>     ret = f(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92,
> in wrapper
>     return func(inst, *args, **kwargs)
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3676, in
> defineXML
>     if ret is None:raise libvirtError('virDomainDefineXML() failed',
> conn=self)
> libvirtError: XML error: non unique alias detected:
> ua-04c2decd-4e33-4023-84de-a2205c777af7
> 2018-03-15 10:27:05,789+0200 INFO  (vm/f9bd0e85) [virt.vm]
> (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') Changed state to Down: XML
> error: non unique alias detected: ua-04c2decd-4e33-4023-84de-a2205c777af7
> (code=1) (vm:1677)

This is likely caused by some recent changes in Engine about management of aliases. Let's ping Arik for more insights.

(In reply to Francesco Romani from comment #7)
> (In reply to Michael Burman from comment #6)
> 
> This is likely caused by some recent changes in Engine about management of
> aliases. Let's ping Arik for more insights.

Michael, can you please attach the engine and vdsm logs of this failure?

(In reply to Arik from comment #8)
> (In reply to Francesco Romani from comment #7)
> > (In reply to Michael Burman from comment #6)
> > 
> > This is likely caused by some recent changes in Engine about management of
> > aliases. Let's ping Arik for more insights.
> 
> Michael, can you please attach the engine and vdsm logs of this failure?

Sure thing, the versions are the latest no need to mention - 
4.2.2.2-0.1.el7
vdsm-4.20.20-1.el7ev.x86_64

Created attachment 1408384 [details]
sr-iov vm failed to run

(In reply to Michael Burman from comment #10)
> Created attachment 1408384 [details]
> sr-iov vm failed to run

Thanks.
The XML we generate seems valid.
The fix for the recent issue with user-aliases we reported to libvirt seems specific to unplugging and then plugging a device with the same user-alias, but its investigation lead to few other changes in that area in libvirt. Can we test this flow against libvirt version that includes all those recent changes?

(In reply to Arik from comment #11)
> (In reply to Michael Burman from comment #10)
> > Created attachment 1408384 [details]
> > sr-iov vm failed to run
> 
> Thanks.
> The XML we generate seems valid.
> The fix for the recent issue with user-aliases we reported to libvirt seems
> specific to unplugging and then plugging a device with the same user-alias,
> but its investigation lead to few other changes in that area in libvirt. Can
> we test this flow against libvirt version that includes all those recent
> changes?

Do you say that this bug is blocked on libvirt as well(i saw Francesco uploaded patched here) ? then we need to change summary and wait for libvirt fix..I'm wondering if we need new bug for this specific issue? 
This bug and the new SR_IOV issue are not involving unplug/plug

Any how, even if the new libvirt fix this issue, it deserves a bug to track the issue. Reporting new bug for the SR-IOV flow.

Arik, the issue reproduced with new libvrit libvirt-3.9.0-14.el7.x86_64

2018-03-15 12:27:16,562+0200 ERROR (vm/f9bd0e85) [virt.vm] (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') The vm start process failed (vm:940)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 869, in _startUnderlyingVm
    self._run()
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2829, in _run
    dom = self._connection.defineXML(domxml)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 130, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3676, in defineXML
    if ret is None:raise libvirtError('virDomainDefineXML() failed', conn=self)
libvirtError: XML error: non unique alias detected: ua-04c2decd-4e33-4023-84de-a2205c777af7
2018-03-15 12:27:16,566+0200 INFO  (vm/f9bd0e85) [virt.vm] (vmId='f9bd0e85-54e5-46ec-9ae7-5a61861120a9') Changed state to Down: XML error: non unique alias detected: ua-04c2decd-4e33-4023-84de-a2205c777af7 (code=1
) (vm:1677)

I have reported new bug BZ 1556828

Verified on - 4.2.2.4-0.1.el7 and vdsm-4.20.22-1.el7ev.x86_64
with
libvirt-client-3.9.0-14.el7.x86_64
libvirt-daemon-3.9.0-14.el7.x86_64

OVN and vfmex flows are fixed now

no doc_text required

This bugzilla is included in oVirt 4.2.2 release, published on March 28th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.2 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.