Bug 1270688 - [New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync
Summary: [New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Network
Version: 3.6.0
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ovirt-3.6.2
: 3.6.2.5
Assignee: Dan Kenigsberg
QA Contact: Meni Yakove
URL:
Whiteboard: network
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-12 07:27 UTC by Michael Burman
Modified: 2016-01-20 12:12 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-20 12:12:40 UTC
oVirt Team: Network
Embargoed:
ylavi: ovirt-3.6.z?
ylavi: planning_ack+
rule-engine: devel_ack+
mburman: testing_ack?


Attachments (Terms of Use)
getVdsCaps output (19.47 KB, text/plain)
2015-11-04 13:44 UTC, Eliraz Levi
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 48064 0 master MERGED engine: make gateway as sync property Never
oVirt gerrit 48365 0 master ABANDONED net: ifup ignores invalid gateway settings Never
oVirt gerrit 51151 0 ovirt-engine-3.6 MERGED engine: make gateway as sync property 2015-12-30 11:59:27 UTC
oVirt gerrit 51152 0 refs/tags/ovirt-engine-3.6.2 ABANDONED engine: make gateway as sync property 2015-12-29 19:37:20 UTC
oVirt gerrit 51153 0 ovirt-engine-3.6.2 MERGED engine: make gateway as sync property 2016-01-03 09:02:04 UTC

Description Michael Burman 2015-10-12 07:27:07 UTC
Description of problem:
[New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync

As part of BZ 1242235 gateway configuration changes on host should be reported as Out-of-sync.

Version-Release number of selected component (if applicable):
3.6.0-0.18.el6

How reproducible:
100

Steps to Reproduce:
1. Change the gateway of a network via the host
2. Refresh caps

Actual results:
The 'gateway' is not marked as 'out-of-sync'

Expected results:
The 'gateway' should be marked as 'out-of-sync'

Additional info:
See also - https://bugzilla.redhat.com/show_bug.cgi?id=1242235

Comment 1 Yaniv Lavi 2015-10-29 12:40:00 UTC
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.

Comment 2 Eliraz Levi 2015-11-04 13:42:27 UTC
fixed in engine side.
vdsm is not reporting gateway in vdsGetCaps.
can't be resolved until vdsm will report the configured gateway.

relevant vdsm data:

vdsGetCaps -please see the attached getVdsCaps file.

================================================================

[root@dhcp-0-237 ~]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.1.0/24 dev eth1  scope link 
7.7.7.0/24 dev NET1  proto kernel  scope link  src 7.7.7.7 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev eth2  scope link  metric 1005 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1007 
169.254.0.0/16 dev NET0  scope link  metric 1010 
169.254.0.0/16 dev NET1  scope link  metric 1021

Comment 3 Eliraz Levi 2015-11-04 13:44:00 UTC
Created attachment 1089683 [details]
getVdsCaps output

Comment 4 Petr Horáček 2015-11-09 16:41:42 UTC
A simple reproducer:

    In [1]: ! ip link add dummy_1 type dummy

    In [2]: from vdsm import vdscli

    In [3]: c = vdscli.connect()

    In [4]: c.setupNetworks({'ovirtmgmt': {'nic': 'eth0', 'bootproto': 'dhcp', 'defaultRoute': True}}, {}, {'connectivityCheck': False})
    Out[4]: {'status': {'code': 0, 'message': 'Done'}}

    In [5]: c.getVdsCapabilities()['info']['networks']
    Out[5]:
    {'ovirtmgmt': {'addr': '10.34.61.130',
    'gateway': '10.34.63.254',
    'netmask': '255.255.252.0',
    ...

    In [6]: c.setupNetworks({'test-network': {'nic': 'dummy_1', 'ipaddr': '7.7.7.7', 'netmask': '255.255.255.0', 'gateway': '1.1.1.1'}}, {}, {'connectivityCheck': False})
    Out[6]: {'status': {'code': 0, 'message': 'Done'}}

    In [7]: c.getVdsCapabilities()['info']['networks']
    Out[7]:
    {'ovirtmgmt': {'addr': '10.34.61.130',
    'gateway': '10.34.63.254',
    'netmask': '255.255.252.0',
    ...
    'test-network': {'addr': '7.7.7.7',
    'gateway': '',
    'netmask': '255.255.255.0',
    ...

Comment 5 Petr Horáček 2015-11-09 17:02:32 UTC
Note, that when ip address and gateway is on the same subnet, it works.

Comment 6 Petr Horáček 2015-11-10 12:42:52 UTC
No need for ovirtmgmt network, it's enough to create a network with gateway and ipaddr on different subnets. In this case, `ifup test-network` prints 'RTNETLINK answers: Network is unreachable' but returns 0, so we don't fail setupNetworks command.

Comment 7 Yaniv Lavi 2015-11-11 10:25:12 UTC
Can you check the case that is appearing in comment 5 ?

Comment 8 Michael Burman 2015-11-11 16:01:25 UTC
Hi

gateway is not marked as out-of-sync in both cases.

Comment 9 Petr Horáček 2015-11-16 13:21:39 UTC
Hi Eliraz,

could you please try to verify your Engine patch with gateway and ip address on the same network? I think, that the gateway-on-a-different-subnet difficulty should be not a part of this bug report, but a different one.

Thanks

Comment 10 Eliraz Levi 2015-12-03 15:23:00 UTC
Vdsm is probably calculating the gateway in a mix way.
Say a user is running:
# ip route change <subnet_address>/<netmask> via <new_gateway_in_subnet>

when running # ip route 
the default gateway will be changed.

example:
[root@dhcp-0-237 network-scripts]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.0.0/16 dev NET2  proto kernel  scope link  src 1.1.1.1 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1088 
169.254.0.0/16 dev NET2  scope link  metric 1105 

[root@dhcp-0-237 network-scripts]# ip route change 1.1.0.0/16 via 1.1.1.23
[root@dhcp-0-237 network-scripts]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.0.0/16 via 1.1.1.23 dev NET2 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1088 
169.254.0.0/16 dev NET2  scope link  metric 1105 



Some static configurations are saved in route-<network_name> and rule-<network_name> (files are saved in /etc/sysconfig/network-scripts/)
probably causing a confusion for vdsm.

Engine code is verified.
Please fix vdsm side.

Comment 11 Sandro Bonazzola 2015-12-23 13:40:43 UTC
oVirt 3.6.2 RC1 has been released for testing, moving to ON_QA

Comment 12 Red Hat Bugzilla Rules Engine 2015-12-29 19:30:02 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 13 Dan Kenigsberg 2015-12-30 14:37:13 UTC
To verify the Engine-side fix, create a network with static IP. Edit its /etc/sysconfig/network-scripts/{rule,route}-BLA files to a different gateway, and ifdown/ifup the interface.

Comment 14 Dan Kenigsberg 2015-12-30 15:02:44 UTC
Note: Vdsm using each network's gateway in the default routing table, as well as in the network's specific routing table. Vdsm does not notice if the default table entry is corrupted.

Comment 15 Dan Kenigsberg 2016-01-20 12:12:40 UTC
We have verified that after setting an out-of-sync gateway in /etc/sysconfig/network-scripts/route-network, the out-of-sync gateway is reported to Engine, and Engine reports the network as out of sync.


Note You need to log in before you can comment on or make changes to this bug.