Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1607860 - [downstream clone - 4.2.5] vdsm-tool upgrade-networks fails with KeyError: 'defaultRoute'
[downstream clone - 4.2.5] vdsm-tool upgrade-networks fails with KeyError: 'd...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm (Show other bugs)
unspecified
Unspecified Unspecified
high Severity high
: ovirt-4.2.5
: ---
Assigned To: Edward Haas
Jiri Belka
: ZStream
Depends On: 1602047
Blocks:
  Show dependency treegraph
 
Reported: 2018-07-24 08:40 EDT by RHV Bugzilla Automation and Verification Bot
Modified: 2018-07-31 13:50 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1602047
Environment:
Last Closed: 2018-07-31 13:50:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3534831 None None None 2018-07-24 08:40 EDT
oVirt gerrit 93122 master MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2018-07-24 08:40 EDT
oVirt gerrit 93125 ovirt-4.2 MERGED net: Do not assume pre-upgrade config has the defaultRoute key 2018-07-24 08:40 EDT
Red Hat Product Errata RHEA-2018:2319 None None None 2018-07-31 13:50 EDT

  None (edit)
Description RHV Bugzilla Automation and Verification Bot 2018-07-24 08:40:23 EDT
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1602047 +++
======================================================================

Description of problem:

After upgrading the RHEL based hypervisor the vdsm-network service fails to start with:
~~~
Jul 17 10:52:40 systemd: Starting Virtual Desktop Server Manager network restoration...                                                                                                                     
Jul 17 10:52:41 vdsm-tool: Traceback (most recent call last):                                                                                                                                               
Jul 17 10:52:41 vdsm-tool: File "/usr/bin/vdsm-tool", line 219, in main                                                                                                                                     
Jul 17 10:52:41 vdsm-tool: return tool_command[cmd]["command"](*args)                                                                                                                                       
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/tool/network.py", line 55, in upgrade_networks                                                                                       
Jul 17 10:52:41 vdsm-tool: netupgrade.upgrade()                                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 67, in upgrade                                                                                          
Jul 17 10:52:41 vdsm-tool: _upgrade_unified_configuration(rconfig)                                                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/netupgrade.py", line 108, in _upgrade_unified_configuration                                                                  
Jul 17 10:52:41 vdsm-tool: canonicalize_networks(config.networks)                                                                                                                                           
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 43, in canonicalize_networks                                                                          
Jul 17 10:52:41 vdsm-tool: _canonicalize_ip_default_route(nets)                                                                                                                                             
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 205, in _canonicalize_ip_default_route                                                                
Jul 17 10:52:41 vdsm-tool: existing_net_with_default_route = _net_with_default_route_from_config()                                                                                                          
Jul 17 10:52:41 vdsm-tool: File "/usr/lib/python2.7/site-packages/vdsm/network/canonicalize.py", line 217, in _net_with_default_route_from_config                                                           
Jul 17 10:52:41 vdsm-tool: if attrs['defaultRoute']:                                                                                                                                                        
Jul 17 10:52:41 vdsm-tool: KeyError: 'defaultRoute'                                                                                                                                                         
Jul 17 10:52:41 systemd: vdsm-network.service: control process exited, code=exited status=1                                                                                                                 
Jul 17 10:52:41 systemd: Failed to start Virtual Desktop Server Manager network restoration
~~~

Version-Release number of selected component (if applicable):

vdsm-4.20.32-1.el7ev.x86_64

How reproducible:
Always

Steps to Reproduce:
- Upgrade vdsm to 4.20.32
- At least one network without 'defaultRoute' key defined in '/var/lib/vdsm/persistence/netconf/nets' directory.
- Restart vdsmd service

Actual results:

The vdsmd service fails to start because vdsm-network dependency failed to start

Expected results:

The vdsmd and vdsm-network services restart correctly

Additional info:

The corresponding code is:
~~~
215 def _net_with_default_route_from_config():                                                                                                                                                              
216     for net, attrs in six.iteritems(RunningConfig().networks):                                                                                                                                          
217         if attrs['defaultRoute']:                                                                                                                                                                       
218     ┆   ┆   return net, attrs                                                                                                                                                                           
219     return None     
~~~

I think the following patch should fix the issue:
~~~
diff --git a/lib/vdsm/network/canonicalize.py b/lib/vdsm/network/canonicalize.py
index 674f98e53..91ebce77e 100644
--- a/lib/vdsm/network/canonicalize.py
+++ b/lib/vdsm/network/canonicalize.py
@@ -214,7 +214,7 @@ def _canonicalize_ip_default_route(nets):
 
 def _net_with_default_route_from_config():
     for net, attrs in six.iteritems(RunningConfig().networks):
-        if attrs['defaultRoute']:
+        if attrs.get('defaultRoute'):
             return net, attrs
     return None
~~~

(Originally by Miguel Martin Villamuelas)
Comment 3 RHV Bugzilla Automation and Verification Bot 2018-07-24 08:40:33 EDT
WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops@redhat.comINFO: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]

For more info please contact: rhv-devops@redhat.com

(Originally by rhv-bugzilla-bot)
Comment 6 Jiri Belka 2018-07-27 05:59:28 EDT
ok, vdsm-4.20.35-1.el7ev.x86_64


# grep default /var/lib/vdsm/persistence/netconf/nets/isolated; echo $?
1

# rpm -q vdsm ; grep default /var/lib/vdsm/persistence/netconf/nets/isolated; echo $?
vdsm-4.20.35-1.el7ev.x86_64
    "defaultRoute": false, 
0
Comment 8 errata-xmlrpc 2018-07-31 13:50:12 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2319

Note You need to log in before you can comment on or make changes to this bug.