Bug 922459 - Traffic will not NAT when DHCP and L3 Agents are on the same host
Summary: Traffic will not NAT when DHCP and L3 Agents are on the same host
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-quantum
Version: 2.0 (Folsom)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: snapshot2
: 3.0
Assignee: Gary Kotton
QA Contact: Ofer Blaut
URL:
Whiteboard:
Depends On: 869004 880142 889264 971672
Blocks: quantum_ovs_tracker
TreeView+ depends on / blocked
 
Reported: 2013-03-17 05:33 UTC by Ofer Blaut
Modified: 2016-04-26 17:32 UTC (History)
10 users (show)

Fixed In Version: openstack-quantum-2013.1.1-9.el6
Doc Type: Enhancement
Doc Text:
A Red Hat Enterprise Linux kernel with network namespaces support enabled is now available in Red Hat OpenStack. As a result OpenStack networking agents are now able to co-exist on the same host while running in isolated network namespaces. By default OpenStack networking as deployed by Red Hat OpenStack is now configured to use network namespaces.
Clone Of:
Environment:
Last Closed: 2013-06-11 18:56:24 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Traffic on interface qr-6e6fed23-9b (256 bytes, application/octet-stream)
2013-03-17 18:23 UTC, Gary Kotton
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:0933 0 normal SHIPPED_LIVE openstack-quantum bug fix update 2013-06-11 22:48:36 UTC

Description Ofer Blaut 2013-03-17 05:33:03 UTC
Description of problem:

When DHCP and L3 Agents are installed on the same host
traffic will not NAT.

VM gets the same MAC address for both DHCP and L3 (GW ), cuasing traffic to be sent to the DHCP MAC address which is different PORT/MAC of L3 interface  

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.configure Quantum All-In-One , or DHCP and L3 Agents are installed on the same host.
2.configure floating ip and assign it to VM
3.Try to access using floating ip 
4. check arp -n on the VM , check both DHCP ip and L3/GW ip have same MAC address
5. check quantum port-list and quantum port-show <port-id> and check which MAC is used 
  
Actual results:

L3 will not be used when DHCP and L3 Agents are installed on the same host

Expected results:

L3 should  be used when DHCP and L3 Agents are installed on the same host

Additional info:

Happens both on Linux Bridge and OVS

Comment 1 Gary Kotton 2013-03-17 05:51:07 UTC
Hi,
Please configure the following to see if the problem occurs:

Globally:
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

Per interface:
net.ipv4.conf.eth0.arp_ignore=1
net.ipv4.conf.eth0.arp_announce=2

Thanks
Gary

Comment 3 Ofer Blaut 2013-03-17 10:34:15 UTC
L3 and DHCP on same host without any change to net.ipv4.conf.all - 

LinuxBridge - works
OVS - Doesn't work 


L3 and DHCP on same host with globally change to net.ipv4.conf.all - 

LinuxBridge - Doesn't work 
OVS - Doesn't work 



When ARP is working 


[root@puma34 ~(keystone_admin_tenant1)]$ ovs-dpctl dump-flows br-int | grep 88.66
in_port(8),eth(src=fa:16:3e:1f:9c:7c,dst=fa:16:3e:ba:bb:fa),eth_type(0x0806),arp(sip=88.66.66.5,tip=88.66.66.1,op=1,sha=fa:16:3e:1f:9c:7c,tha=fa:16:3e:ba:bb:fa), packets:95, bytes:3990, used:0.254s, actions:3
in_port(3),eth(src=fa:16:3e:ba:bb:fa,dst=fa:16:3e:1f:9c:7c),eth_type(0x0806),arp(sip=88.66.66.2,tip=88.66.66.5,op=2,sha=fa:16:3e:ba:bb:fa,tha=fa:16:3e:1f:9c:7c), packets:0, bytes:0, used:never, actions:8
in_port(3),eth(src=fa:16:3e:ba:bb:fa,dst=fa:16:3e:1f:9c:7c),eth_type(0x0806),arp(sip=88.66.66.1,tip=88.66.66.5,op=2,sha=fa:16:3e:ba:bb:fa,tha=fa:16:3e:1f:9c:7c), packets:96, bytes:4032, used:0.254s, actions:8
in_port(8),eth(src=fa:16:3e:1f:9c:7c,dst=fa:16:3e:ba:bb:fa),eth_type(0x0806),arp(sip=88.66.66.5,tip=88.66.66.2,op=1,sha=fa:16:3e:1f:9c:7c,tha=00:00:00:00:00:00), packets:0, bytes:0, used:never, actions:3


When ARP is not working 


in_port(8),eth(src=fa:16:3e:1f:9c:7c,dst=ff:ff:ff:ff:ff:ff),eth_type(0x0806),arp(sip=88.66.66.5,tip=88.66.66.1,op=1,sha=fa:16:3e:1f:9c:7c,tha=00:00:00:00:00:00), packets:2, bytes:84, used:1.657s, actions:7,5,push_vlan(vid=1,pcp=0),4,0,pop_vlan,3


ovs-dpctl dump-flows system@br-eth3 | grep 88.66
in_port(5),eth(src=fa:16:3e:1f:9c:7c,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=1,pcp=0),encap(eth_type(0x0806),arp(sip=88.66.66.5,tip=88.66.66.1,op=1,sha=fa:16:3e:1f:9c:7c,tha=ff:ff:ff:ff:ff:ff)), packets:171, bytes:7866, used:0.807s, actions:pop_vlan,push_vlan(vid=202,pcp=0),2,0




[root@puma34 ~(keystone_admin_tenant1)]$ ovs-dpctl show
system@br-eth3:
	lookups: hit:6209 missed:5220 lost:0
	flows: 7
	port 0: br-eth3 (internal)
	port 2: eth3
	port 5: phy-br-eth3
system@br-int:
	lookups: hit:7351 missed:6316 lost:0
	flows: 9
	port 0: br-int (internal)
	port 3: tap552d606a-1b (internal)
	port 4: int-br-eth3 ->>>> Bridge between br-int and ETH3 
	port 5: qr-3043755b-8e (internal) >>> Router Port
	port 6: qg-5ef3d8dc-fc (internal)
	port 7: qvo072c2766-5b
	port 8: qvoffd849ed-00 ->>>> VM PORT 
system@br-ex:
	lookups: hit:0 missed:294 lost:0
	flows: 0
	port 0: br-ex (internal)
	port 2: eth3.195

Comment 5 Gary Kotton 2013-03-17 18:23:27 UTC
Created attachment 711512 [details]
Traffic on interface qr-6e6fed23-9b

Comment 9 Jay Turner 2013-03-26 00:23:28 UTC
QE ack.  Reproducer information is available and there are security aspects to not enforcing this distributed solution.

Comment 12 Bob Kukura 2013-03-28 15:46:52 UTC
The inability to run quantum-l3-agent and quantum-dhcp-agent on the same node is likely related to the lack of network namespace support in RHEL 6.4, so I'm adding dependencies on the relevant RHEL bugs.

Comment 13 Bruce Reeler 2013-04-03 05:54:11 UTC
re needinfo: 
Hi Maru. 
What is meant by "configurable" in the sentence that was added to doc-text:
"This restriction is configurable and by default one type of agent will refuse to run if the other type is already running on the host."

Does it mean "you can choose one of the two but not both"  ?

Comment 14 Maru Newby 2013-04-03 17:04:55 UTC
Hi Bruce,

Since I added the doc text, it appears that the patch that allowed the dhcp and l3 agents to be configured to not run if the other were detected has been removed.

I've updated the doc text accordingly.

Comment 15 Bruce Reeler 2013-04-04 01:54:44 UTC
(In reply to comment #14)
> Hi Bruce,
> 
> Since I added the doc text, it appears that the patch that allowed the dhcp
> and l3 agents to be configured to not run if the other were detected has
> been removed.
> 
> I've updated the doc text accordingly.

Updated Release Notes as per Maru's latest doc text update.

Comment 16 Ofer Blaut 2013-06-05 08:44:26 UTC
It works now with netns support

quantum version :

openstack-quantum-2013.1.1-9.el6ost.noarch
openstack-quantum-openvswitch-2013.1.1-9.el6ost.noarc

kernel

kernel-2.6.32-358.6.2.openstack.el6.x86_64


I used quantum reference architecture 

quantum server on host A
DHCP/L3 on host B
computes on hosts C & D
using OVS and VLANs and  floating ip works fine

Comment 18 errata-xmlrpc 2013-06-11 18:56:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0933.html


Note You need to log in before you can comment on or make changes to this bug.