Bug 1270688 - [New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync
[New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync
Status: CLOSED NEXTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Network (Show other bugs)
3.6.0
x86_64 Linux
low Severity medium (vote)
: ovirt-3.6.2
: 3.6.2.5
Assigned To: Dan Kenigsberg
Meni Yakove
network
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-12 03:27 EDT by Michael Burman
Modified: 2016-01-20 07:12 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-20 07:12:40 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ylavi: ovirt‑3.6.z?
ylavi: planning_ack+
rule-engine: devel_ack+
mburman: testing_ack?


Attachments (Terms of Use)
getVdsCaps output (19.47 KB, text/plain)
2015-11-04 08:44 EST, Eliraz Levi
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 48064 master MERGED engine: make gateway as sync property Never
oVirt gerrit 48365 master ABANDONED net: ifup ignores invalid gateway settings Never
oVirt gerrit 51151 ovirt-engine-3.6 MERGED engine: make gateway as sync property 2015-12-30 06:59 EST
oVirt gerrit 51152 refs/tags/ovirt-engine-3.6.2 ABANDONED engine: make gateway as sync property 2015-12-29 14:37 EST
oVirt gerrit 51153 ovirt-engine-3.6.2 MERGED engine: make gateway as sync property 2016-01-03 04:02 EST

  None (edit)
Description Michael Burman 2015-10-12 03:27:07 EDT
Description of problem:
[New HostSetupNetworksCommand] - Marking gateway configuration as Out-of-sync

As part of BZ 1242235 gateway configuration changes on host should be reported as Out-of-sync.

Version-Release number of selected component (if applicable):
3.6.0-0.18.el6

How reproducible:
100

Steps to Reproduce:
1. Change the gateway of a network via the host
2. Refresh caps

Actual results:
The 'gateway' is not marked as 'out-of-sync'

Expected results:
The 'gateway' should be marked as 'out-of-sync'

Additional info:
See also - https://bugzilla.redhat.com/show_bug.cgi?id=1242235
Comment 1 Yaniv Lavi (Dary) 2015-10-29 08:40:00 EDT
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.
Comment 2 Eliraz Levi 2015-11-04 08:42:27 EST
fixed in engine side.
vdsm is not reporting gateway in vdsGetCaps.
can't be resolved until vdsm will report the configured gateway.

relevant vdsm data:

vdsGetCaps -please see the attached getVdsCaps file.

================================================================

[root@dhcp-0-237 ~]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.1.0/24 dev eth1  scope link 
7.7.7.0/24 dev NET1  proto kernel  scope link  src 7.7.7.7 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev eth2  scope link  metric 1005 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1007 
169.254.0.0/16 dev NET0  scope link  metric 1010 
169.254.0.0/16 dev NET1  scope link  metric 1021
Comment 3 Eliraz Levi 2015-11-04 08:44 EST
Created attachment 1089683 [details]
getVdsCaps output
Comment 4 Petr Horáček 2015-11-09 11:41:42 EST
A simple reproducer:

    In [1]: ! ip link add dummy_1 type dummy

    In [2]: from vdsm import vdscli

    In [3]: c = vdscli.connect()

    In [4]: c.setupNetworks({'ovirtmgmt': {'nic': 'eth0', 'bootproto': 'dhcp', 'defaultRoute': True}}, {}, {'connectivityCheck': False})
    Out[4]: {'status': {'code': 0, 'message': 'Done'}}

    In [5]: c.getVdsCapabilities()['info']['networks']
    Out[5]:
    {'ovirtmgmt': {'addr': '10.34.61.130',
    'gateway': '10.34.63.254',
    'netmask': '255.255.252.0',
    ...

    In [6]: c.setupNetworks({'test-network': {'nic': 'dummy_1', 'ipaddr': '7.7.7.7', 'netmask': '255.255.255.0', 'gateway': '1.1.1.1'}}, {}, {'connectivityCheck': False})
    Out[6]: {'status': {'code': 0, 'message': 'Done'}}

    In [7]: c.getVdsCapabilities()['info']['networks']
    Out[7]:
    {'ovirtmgmt': {'addr': '10.34.61.130',
    'gateway': '10.34.63.254',
    'netmask': '255.255.252.0',
    ...
    'test-network': {'addr': '7.7.7.7',
    'gateway': '',
    'netmask': '255.255.255.0',
    ...
Comment 5 Petr Horáček 2015-11-09 12:02:32 EST
Note, that when ip address and gateway is on the same subnet, it works.
Comment 6 Petr Horáček 2015-11-10 07:42:52 EST
No need for ovirtmgmt network, it's enough to create a network with gateway and ipaddr on different subnets. In this case, `ifup test-network` prints 'RTNETLINK answers: Network is unreachable' but returns 0, so we don't fail setupNetworks command.
Comment 7 Yaniv Lavi (Dary) 2015-11-11 05:25:12 EST
Can you check the case that is appearing in comment 5 ?
Comment 8 Michael Burman 2015-11-11 11:01:25 EST
Hi

gateway is not marked as out-of-sync in both cases.
Comment 9 Petr Horáček 2015-11-16 08:21:39 EST
Hi Eliraz,

could you please try to verify your Engine patch with gateway and ip address on the same network? I think, that the gateway-on-a-different-subnet difficulty should be not a part of this bug report, but a different one.

Thanks
Comment 10 Eliraz Levi 2015-12-03 10:23:00 EST
Vdsm is probably calculating the gateway in a mix way.
Say a user is running:
# ip route change <subnet_address>/<netmask> via <new_gateway_in_subnet>

when running # ip route 
the default gateway will be changed.

example:
[root@dhcp-0-237 network-scripts]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.0.0/16 dev NET2  proto kernel  scope link  src 1.1.1.1 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1088 
169.254.0.0/16 dev NET2  scope link  metric 1105 

[root@dhcp-0-237 network-scripts]# ip route change 1.1.0.0/16 via 1.1.1.23
[root@dhcp-0-237 network-scripts]# ip route
default via 10.35.1.254 dev ovirtmgmt 
1.1.0.0/16 via 1.1.1.23 dev NET2 
10.35.0.0/23 dev ovirtmgmt  proto kernel  scope link  src 10.35.0.237 
169.254.0.0/16 dev ovirtmgmt  scope link  metric 1088 
169.254.0.0/16 dev NET2  scope link  metric 1105 



Some static configurations are saved in route-<network_name> and rule-<network_name> (files are saved in /etc/sysconfig/network-scripts/)
probably causing a confusion for vdsm.

Engine code is verified.
Please fix vdsm side.
Comment 11 Sandro Bonazzola 2015-12-23 08:40:43 EST
oVirt 3.6.2 RC1 has been released for testing, moving to ON_QA
Comment 12 Red Hat Bugzilla Rules Engine 2015-12-29 14:30:02 EST
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 13 Dan Kenigsberg 2015-12-30 09:37:13 EST
To verify the Engine-side fix, create a network with static IP. Edit its /etc/sysconfig/network-scripts/{rule,route}-BLA files to a different gateway, and ifdown/ifup the interface.
Comment 14 Dan Kenigsberg 2015-12-30 10:02:44 EST
Note: Vdsm using each network's gateway in the default routing table, as well as in the network's specific routing table. Vdsm does not notice if the default table entry is corrupted.
Comment 15 Dan Kenigsberg 2016-01-20 07:12:40 EST
We have verified that after setting an out-of-sync gateway in /etc/sysconfig/network-scripts/route-network, the out-of-sync gateway is reported to Engine, and Engine reports the network as out of sync.

Note You need to log in before you can comment on or make changes to this bug.