Bug 1518688

Summary: Unable to run a VM with a display network that is non-vm
Product: [oVirt] vdsm Reporter: Evgheni Dereveanchin <ederevea>
Component: GeneralAssignee: Edward Haas <edwardh>
Status: CLOSED CURRENTRELEASE QA Contact: Michael Burman <mburman>
Severity: medium Docs Contact:
Priority: high    
Version: 4.20.8CC: bugs, danken, ederevea, myakove, rich, tjelinek
Target Milestone: ovirt-4.2.0Keywords: Regression
Target Release: ---Flags: rule-engine: ovirt-4.2+
rule-engine: blocker+
mburman: testing_plan_complete+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.20.8-79.gitd2a37c7 (4.20.9) Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-20 10:45:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Evgheni Dereveanchin 2017-11-29 13:15:51 UTC
Description of problem:
After upgrading to ovirt-4.2-pre VMs fail to start on hosts where the management network is not a VM network (hence no ovirtmgmt bridge is present)

Version-Release number of selected component (if applicable):
vdsm-4.20.8-1.el7.centos.x86_64
libvirt-daemon-3.2.0-14.el7_4.3.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a new DC in oVirt and go to Networks
2. edit ovirtmgmt network and untick "VM network" checkbox
3. add another network on a VLAN to act as a VM network
4. add a 4.1 host to this DC, create a VM
5. start the VM - it runs fine
6. upgrade the host to 4.2
7. try to start the VM now

Actual results:
 VM start fails with error:
  Changed state to Down: internal error: no IP address found for interface 'ovirtmgmt' (code=1) (vm:1619)

Expected results:
 VM starts successfully

Additional info:
 As ovirtmgmt is not a VM network, a network bridge is not necessary and the IP is assigned to the interface/bond directly. The failure is likely coming from the SPICE port binding which happens to the wrong interface for some reason.

Comment 2 Evgheni Dereveanchin 2017-11-29 14:06:13 UTC
A workaround for this is to run the VM in headless mode with no SPICE/VNC port listening. This is not an option for each use case.

Comment 3 Tomas Jelinek 2017-11-29 14:32:11 UTC
(In reply to Evgheni Dereveanchin from comment #0)
> Description of problem:
> After upgrading to ovirt-4.2-pre VMs fail to start on hosts where the
> management network is not a VM network (hence no ovirtmgmt bridge is present)
> 
> Version-Release number of selected component (if applicable):
> vdsm-4.20.8-1.el7.centos.x86_64
> libvirt-daemon-3.2.0-14.el7_4.3.x86_64
> 
> How reproducible:
> always
> 
> Steps to Reproduce:
> 1. create a new DC in oVirt and go to Networks
> 2. edit ovirtmgmt network and untick "VM network" checkbox

so, the ovirtmgmt is still the display network?

> 3. add another network on a VLAN to act as a VM network
> 4. add a 4.1 host to this DC, create a VM
> 5. start the VM - it runs fine
> 6. upgrade the host to 4.2
> 7. try to start the VM now
> 
> Actual results:
>  VM start fails with error:
>   Changed state to Down: internal error: no IP address found for interface
> 'ovirtmgmt' (code=1) (vm:1619)
> 
> Expected results:
>  VM starts successfully
> 
> Additional info:
>  As ovirtmgmt is not a VM network, a network bridge is not necessary and the
> IP is assigned to the interface/bond directly. The failure is likely coming
> from the SPICE port binding which happens to the wrong interface for some
> reason.

it should be the correct interface as soon as the ovirtmgmt is the indeed the display network.

Comment 4 Evgheni Dereveanchin 2017-11-29 14:36:56 UTC
Correct, ovirtmgmt has all the other roles (management, display, migration) but it's not a VM network so no bridge is present. This same configuration worked fine in 4.1 and earlier versions.

Comment 5 Tomas Jelinek 2017-11-29 14:52:23 UTC
Dan, do you know if something have changed around this in 4.2?

Comment 6 Dan Kenigsberg 2017-11-29 15:32:02 UTC
It might be related to how we create the libvirt network for the display network (that happens to be ovirtmgmt, too).

Could you attach supervdsm.log, the content of /var/lib/vdsm/ and the output of
  virsh -r network-dumpxml vdsm-ovirtmgmt
?

Comment 9 Evgheni Dereveanchin 2017-11-29 16:33:59 UTC
There is no vdsm-ovirtmgmt network according to virsh output:

#  virsh -r net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 ;vdsmdummy;          active     no            no

# virsh -r net-dumpxml vdsm-ovirtmgmt
error: failed to get network 'vdsm-ovirtmgmt'
error: Network not found: no network with matching name 'vdsm-ovirtmgmt'

# rpm -q vdsm
vdsm-4.20.8-1.el7.centos.x86_64


NOTE:
On a different host with matching network configs that wasn't upgraded to 4.2 this shows correct results:

# virsh -r net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 ;vdsmdummy;          active     no            no
 vdsm-ovirtmgmt       active     yes           yes
 vdsm-workers         active     yes           yes


# virsh -r net-dumpxml vdsm-ovirtmgmt
<network>
  <name>vdsm-ovirtmgmt</name>
  <uuid>ef6e7179-4a34-46aa-8d27-79c4d70a35ad</uuid>
  <forward dev='bond0' mode='passthrough'>
    <interface dev='bond0'/>
  </forward>
</network>

# rpm -q vdsm
vdsm-4.19.31-1.el7.centos.x86_64

Comment 10 Edward Haas 2017-11-30 07:48:48 UTC
This is a regression in 4.2: Specifying a display network on a non-VM network is broken.

Comment 11 Red Hat Bugzilla Rules Engine 2017-11-30 07:53:26 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 12 Michael Burman 2017-12-03 10:20:35 UTC
Verified on - vdsm-4.20.8-79.gitd2a37c7.el7.centos.x86_64

Comment 13 Evgheni Dereveanchin 2017-12-04 16:22:31 UTC
I can also confirm that VMs start fine after upgrading to vdsm-4.20.9-1.el7.centos from ovirt-4.2-pre

Comment 14 Michael Burman 2017-12-04 17:08:27 UTC
(In reply to Evgheni Dereveanchin from comment #13)
> I can also confirm that VMs start fine after upgrading to
> vdsm-4.20.9-1.el7.centos from ovirt-4.2-pre

Thank you very much Evgheni for confirming this)

Comment 15 Sandro Bonazzola 2017-12-20 10:45:58 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Comment 16 Rich Davis 2018-07-12 13:03:57 UTC
Hi - This little blighter has re appeared in VDSM Version:
vdsm-4.20.32-1.el7ev

Red Hat Virtualization Manager Web Administration
Software Version:4.2.4.5-0.1.el7_3

Support case will be raised to track.

Cheers