Bug 1602047 - vdsm-tool upgrade-networks fails with KeyError: 'defaultRoute'
Summary: vdsm-tool upgrade-networks fails with KeyError: 'defaultRoute'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.3.0
: 4.3.0
Assignee: Edward Haas
QA Contact: Jan Zmeskal
URL:
Whiteboard:
Depends On:
Blocks: 1607860
TreeView+ depends on / blocked
 
Reported: 2018-07-17 16:59 UTC by Miguel Martin
Modified: 2021-09-09 15:07 UTC (History)
10 users (show)

Fixed In Version: v4.30.3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1607860 (view as bug list)
Environment:
Last Closed: 2019-05-08 12:36:02 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43555 0 None None None 2021-09-09 15:07:08 UTC
Red Hat Knowledge Base (Solution) 3534831 0 None None None 2018-07-18 15:19:19 UTC
Red Hat Product Errata RHBA-2019:1077 0 None None None 2019-05-08 12:36:24 UTC
oVirt gerrit 93122 0 master MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2020-09-02 13:35:14 UTC
oVirt gerrit 93125 0 ovirt-4.2 MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2020-09-02 13:35:14 UTC

Description Miguel Martin 2018-07-17 16:59:29 UTC
Description of problem:

After upgrading the RHEL based hypervisor the vdsm-network service fails to start with:
~~~
Jul 17 10:52:40 systemd: Starting Virtual Desktop Server Manager network restoration...                                                                                                                     
Jul 17 10:52:41 vdsm-tool: Traceback (most recent call last):                                                                                                                                               
Jul 17 10:52:41 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main                                                                                                                                     
Jul 17 10:52:41 vdsm-tool: return tool_command[cmd]["command"](*args)                                                                                                                                       
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 55, in upgrade_networks                                                                                       
Jul 17 10:52:41 vdsm-tool: netupgrade.upgrade()                                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 67, in upgrade                                                                                          
Jul 17 10:52:41 vdsm-tool: _upgrade_unified_configuration(rconfig)                                                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 108, in _upgrade_unified_configuration                                                                  
Jul 17 10:52:41 vdsm-tool: canonicalize_networks(config.networks)                                                                                                                                           
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 43, in canonicalize_networks                                                                          
Jul 17 10:52:41 vdsm-tool: _canonicalize_ip_default_route(nets)                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 205, in _canonicalize_ip_default_route                                                                
Jul 17 10:52:41 vdsm-tool: existing_net_with_default_route = _net_with_default_route_from_config()                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 217, in _net_with_default_route_from_config                                                           
Jul 17 10:52:41 vdsm-tool: if attrs['defaultRoute']:                                                                                                                                                        
Jul 17 10:52:41 vdsm-tool: KeyError: 'defaultRoute'                                                                                                                                                         
Jul 17 10:52:41 systemd: vdsm-network.service: control process exited, code=exited status=1                                                                                                                 
Jul 17 10:52:41 systemd: Failed to start Virtual Desktop Server Manager network restoration
~~~

Version-Release number of selected component (if applicable):

vdsm-4.20.32-1.el7ev.x86_64

How reproducible:
Always

Steps to Reproduce:
- Upgrade vdsm to 4.20.32
- At least one network without 'defaultRoute' key defined in '/var/lib/vdsm/persistence/netconf/nets' directory.
- Restart vdsmd service

Actual results:

The vdsmd service fails to start because vdsm-network dependency failed to start

Expected results:

The vdsmd and vdsm-network services restart correctly

Additional info:

The corresponding code is:
~~~
215 def _net_with_default_route_from_config():                                                                                                                                                              
216     for net, attrs in six.iteritems(RunningConfig().networks):                                                                                                                                          
217         if attrs['defaultRoute']:                                                                                                                                                                       
218     ┆   ┆   return net, attrs                                                                                                                                                                           
219     return None     
~~~

I think the following patch should fix the issue:
~~~
diff --git a/lib/vdsm/network/canonicalize.py b/lib/vdsm/network/canonicalize.py
index 674f98e53..91ebce77e 100644
--- a/lib/vdsm/network/canonicalize.py
+++ b/lib/vdsm/network/canonicalize.py
@@ -214,7 +214,7 @@ def _canonicalize_ip_default_route(nets):
 
 def _net_with_default_route_from_config():
     for net, attrs in six.iteritems(RunningConfig().networks):
-        if attrs['defaultRoute']:
+        if attrs.get('defaultRoute'):
             return net, attrs
     return None
~~~

Comment 2 RHV bug bot 2018-07-24 12:27:20 UTC
WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops

Comment 4 Lukas Svaty 2018-09-05 05:44:17 UTC
Hi Edward, I believe this one should be on 4.3.0 as we had a clone for 4.2.5 - bug#1607860. And 4.2.6 is already out (with this patch since 4.2.5)

Comment 7 Jan Zmeskal 2019-01-18 16:01:41 UTC
Verified on vdsm package version 4.30.6-1 using this network:
{
    "ipv6autoconf": true, 
    "nameservers": [], 
    "nic": "eth1", 
    "mtu": 1500, 
    "switch": "legacy", 
    "dhcpv6": false, 
    "stp": false, 
    "bridged": true, 
    "bootproto": "dhcp"
}
and running "systemctl restart vdsmd".

Comment 9 errata-xmlrpc 2019-05-08 12:36:02 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1077


Note You need to log in before you can comment on or make changes to this bug.