Bug 1100164

Summary: L2 agent fails to start with VXLAN configuration
Product: Red Hat OpenStack Reporter: Ofer Blaut <oblaut>
Component: openstack-neutronAssignee: Terry Wilson <twilson>
Status: CLOSED ERRATA QA Contact: Ofer Blaut <oblaut>
Severity: high Docs Contact:
Priority: high    
Version: 5.0 (RHEL 7)CC: a-hashiguchi, chrisw, jlibosva, jsimonel, lpeer, mlopes, nyechiel, oblaut, twilson, yeylon
Target Milestone: rcKeywords: Regression
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 2014-05-27.2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1081011
: 1111697 (view as bug list) Environment:
Last Closed: 2014-07-08 15:36:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1081011    
Bug Blocks: 1111697    

Description Ofer Blaut 2014-05-22 07:06:46 UTC
+++ This bug was initially created as a clone of Bug #1081011 +++

Description of problem:

I have used foreman to install openstack RDO with VXLAN configuration

L2 agent fails to start ( attached logs and info )

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. install openstack with foreman 
2. check l2 agent status on both neutron networker and compute node
3.

Actual results:

neutron-openvswitch-agent:              dead

Expected results:

neutron-openvswitch-agent:              active

Additional info:

--- Additional comment from Ofer Blaut on 2014-03-26 09:34:56 EDT ---

Was tested with RHEL 6.5

--- Additional comment from lpeer on 2014-03-31 07:57:54 EDT ---

can you please attach config files of your setup

--- Additional comment from Jakub Libosvar on 2014-04-01 10:08:53 EDT ---

Also running with debug=True would be helpful. There is meaningful message prior to checking to see, what the versions are.

--- Additional comment from Terry Wilson on 2014-04-08 11:14:26 EDT ---

This might also be related to the ovs_neutron_plugin.ini/plugin.ini permissions issues.

--- Additional comment from Terry Wilson on 2014-04-14 12:43:23 EDT ---

It looks like the version check for the openvswitch kernel module doesn't work for us because modinfo doesn't have a "version:" field for openvswitch (or for most of the modules I have loaded on my RHEL box). So this code is broken:

470 def get_installed_ovs_klm_version():
471     args = ["modinfo", "openvswitch"]
472     try:
473         cmd = utils.execute(args)
474         for line in cmd.split('\n'):
475             if 'version: ' in line and not 'srcversion' in line:
476                 ver = re.findall("\d+\.\d+", line)
477                 return ver[0]
478     except Exception:
479         LOG.exception(_("Unable to retrieve OVS kernel module version."))

Seeing as this is the only information we have:

[root@localhost ~]# modinfo openvswitch
filename:       /lib/modules/2.6.32-431.11.2.el6.x86_64/kernel/net/openvswitch/openvswitch.ko
license:        GPL
description:    Open vSwitch switching datapath
srcversion:     993363C44DF474BD67B03CC
depends:        vxlan
vermagic:       2.6.32-431.11.2.el6.x86_64 SMP mod_unload modversions

I see no way to fix this in the upstream code at all. We could disable the check downstream or ship the kernel module with an actual "version" field. For reference, on Ubuntu:

terry@ubuntu1310:~$ modinfo openvswitch
filename:       /lib/modules/3.11.0-12-generic/updates/dkms/openvswitch.ko
version:        1.10.2
license:        GPL
description:    Open vSwitch switching datapath
srcversion:     3367192C878D3C0CD7F68D2
depends:        
vermagic:       3.11.0-12-generic SMP mod_unload modversions 


This is again a very real example of why we need upstream gating jobs that run on RHEL.

--- Additional comment from Terry Wilson on 2014-04-17 11:07:50 EDT ---

Fix submitted upstream for review: https://review.openstack.org/#/c/88121/

--- Additional comment from Ofer Blaut on 2014-05-22 03:05:45 EDT ---

Added files as requested

Comment 1 Ofer Blaut 2014-05-22 10:40:48 UTC
workaround :

1. edit /usr/lib/python2.7/site-packages/neutron/plugins/openvswitch/common/constants.py

2. change :

# The first version of the Linux kernel with converged VXLAN code for OVS
MINIMUM_LINUX_KERNEL_OVS_VXLAN = "3.10.0"

3.  service neutron-openvswitch-agent restart

Comment 2 lpeer 2014-05-22 12:36:57 UTC
Terry,
Can we get a d/s only patch to  adjust the config value. I don't think the issue would be handled in u/s anytime soon.

we also need to take care of this issue in RDO ASAP (BZ 1081011 )

Comment 3 Ihar Hrachyshka 2014-05-23 09:09:21 UTC
*** Bug 1100555 has been marked as a duplicate of this bug. ***

Comment 5 Ofer Blaut 2014-05-28 13:56:37 UTC
Tested 

openstack-neutron-2014.1-22.el7ost.noarch
python-neutron-2014.1-22.el7ost.noarch
openstack-neutron-ml2-2014.1-22.el7ost.noarch
openstack-neutron-openvswitch-2014.1-22.el7ost.noarch



16:47:18.747943 IP 55.55.55.19.53624 > 55.55.55.89.4789: VXLAN, flags [I] (0x08), vni 20000
IP 23.62.6.186 > 21.0.0.3: ICMP echo reply, id 46086, seq 168, length 64
^C

Comment 8 errata-xmlrpc 2014-07-08 15:36:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-0848.html