Bug 1607860 - [downstream clone - 4.2.5] vdsm-tool upgrade-networks fails with KeyError: 'defaultRoute'
Summary: [downstream clone - 4.2.5] vdsm-tool upgrade-networks fails with KeyError: 'd...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: unspecified
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.2.5
: ---
Assignee: Edward Haas
QA Contact: Jiri Belka
URL:
Whiteboard:
Depends On: 1602047
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-24 12:40 UTC by RHV bug bot
Modified: 2021-09-09 15:11 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1602047
Environment:
Last Closed: 2018-07-31 17:50:12 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-43561 0 None None None 2021-09-09 15:11:00 UTC
Red Hat Knowledge Base (Solution) 3534831 0 None None None 2018-07-24 12:40:44 UTC
Red Hat Product Errata RHEA-2018:2319 0 None None None 2018-07-31 17:50:26 UTC
oVirt gerrit 93122 0 master MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2018-07-24 12:40:44 UTC
oVirt gerrit 93125 0 ovirt-4.2 MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2018-07-24 12:40:44 UTC

Description RHV bug bot 2018-07-24 12:40:23 UTC
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1602047 +++
======================================================================

Description of problem:

After upgrading the RHEL based hypervisor the vdsm-network service fails to start with:
~~~
Jul 17 10:52:40 systemd: Starting Virtual Desktop Server Manager network restoration...                                                                                                                     
Jul 17 10:52:41 vdsm-tool: Traceback (most recent call last):                                                                                                                                               
Jul 17 10:52:41 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main                                                                                                                                     
Jul 17 10:52:41 vdsm-tool: return tool_command[cmd]["command"](*args)                                                                                                                                       
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 55, in upgrade_networks                                                                                       
Jul 17 10:52:41 vdsm-tool: netupgrade.upgrade()                                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 67, in upgrade                                                                                          
Jul 17 10:52:41 vdsm-tool: _upgrade_unified_configuration(rconfig)                                                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 108, in _upgrade_unified_configuration                                                                  
Jul 17 10:52:41 vdsm-tool: canonicalize_networks(config.networks)                                                                                                                                           
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 43, in canonicalize_networks                                                                          
Jul 17 10:52:41 vdsm-tool: _canonicalize_ip_default_route(nets)                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 205, in _canonicalize_ip_default_route                                                                
Jul 17 10:52:41 vdsm-tool: existing_net_with_default_route = _net_with_default_route_from_config()                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 217, in _net_with_default_route_from_config                                                           
Jul 17 10:52:41 vdsm-tool: if attrs['defaultRoute']:                                                                                                                                                        
Jul 17 10:52:41 vdsm-tool: KeyError: 'defaultRoute'                                                                                                                                                         
Jul 17 10:52:41 systemd: vdsm-network.service: control process exited, code=exited status=1                                                                                                                 
Jul 17 10:52:41 systemd: Failed to start Virtual Desktop Server Manager network restoration
~~~

Version-Release number of selected component (if applicable):

vdsm-4.20.32-1.el7ev.x86_64

How reproducible:
Always

Steps to Reproduce:
- Upgrade vdsm to 4.20.32
- At least one network without 'defaultRoute' key defined in '/var/lib/vdsm/persistence/netconf/nets' directory.
- Restart vdsmd service

Actual results:

The vdsmd service fails to start because vdsm-network dependency failed to start

Expected results:

The vdsmd and vdsm-network services restart correctly

Additional info:

The corresponding code is:
~~~
215 def _net_with_default_route_from_config():                                                                                                                                                              
216     for net, attrs in six.iteritems(RunningConfig().networks):                                                                                                                                          
217         if attrs['defaultRoute']:                                                                                                                                                                       
218     ┆   ┆   return net, attrs                                                                                                                                                                           
219     return None     
~~~

I think the following patch should fix the issue:
~~~
diff --git a/lib/vdsm/network/canonicalize.py b/lib/vdsm/network/canonicalize.py
index 674f98e53..91ebce77e 100644
--- a/lib/vdsm/network/canonicalize.py
+++ b/lib/vdsm/network/canonicalize.py
@@ -214,7 +214,7 @@ def _canonicalize_ip_default_route(nets):
 
 def _net_with_default_route_from_config():
     for net, attrs in six.iteritems(RunningConfig().networks):
-        if attrs['defaultRoute']:
+        if attrs.get('defaultRoute'):
             return net, attrs
     return None
~~~

(Originally by Miguel Martin Villamuelas)

Comment 3 RHV bug bot 2018-07-24 12:40:33 UTC
WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops

(Originally by rhv-bugzilla-bot)

Comment 6 Jiri Belka 2018-07-27 09:59:28 UTC
ok, vdsm-4.20.35-1.el7ev.x86_64


# grep default /var/lib/vdsm/persistence/netconf/nets/isolated; echo $?
1

# rpm -q vdsm ; grep default /var/lib/vdsm/persistence/netconf/nets/isolated; echo $?
vdsm-4.20.35-1.el7ev.x86_64
    "defaultRoute": false, 
0

Comment 8 errata-xmlrpc 2018-07-31 17:50:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2319


Note You need to log in before you can comment on or make changes to this bug.