Bug 1418261 - [FdBeta] OVN-DHCP is not sending DHCP responses after a MAC change in north db
Summary: [FdBeta] OVN-DHCP is not sending DHCP responses after a MAC change in north db
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: openvswitch
Version: 7.3
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Lance Richardson
QA Contact: qding
URL:
Whiteboard:
Depends On:
Blocks: 1408121
TreeView+ depends on / blocked
 
Reported: 2017-02-01 12:07 UTC by Mor
Modified: 2017-07-12 15:26 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-07-12 15:23:41 UTC
Target Upstream Version:


Attachments (Terms of Use)
ovn/ovs logs on host (3.65 KB, application/octet-stream)
2017-02-01 12:14 UTC, Mor
no flags Details

Description Mor 2017-02-01 12:07:02 UTC
Description of problem:
When we modify an existing logical switch port row in the north db, to change MAC address of a VM, changes are propagated to the OVN-controller on the OVN host, we can also see some openflow rules being created with the new MAC address, but a VM (with the new MAC address) is unable to receive IP address from DHCP. When we revert the address change, and set the original MAC address, VM is able to get IP.

Log output below shows that they are missing openflow rules for the changed MAC address: 00:1a:4a:16:20:61, when comparing it to the old MAC address: 00:1a:4a:16:20:31.

ovn-controller.log (host of VM):
2017-01-30T14:02:30.381Z|00054|binding|INFO|Claiming lport 3ef3d175-f1fa-47cb-9a5a-2bb6b1efa860 for this chassis.
2017-01-30T14:02:30.381Z|00055|binding|INFO|Claiming 00:1a:4a:16:20:31 dynamic
2017-01-30T14:03:18.484Z|00056|binding|INFO|Releasing lport 3ef3d175-f1fa-47cb-9a5a-2bb6b1efa860 from this chassis.
2017-01-30T14:03:18.484Z|00057|binding|INFO|Releasing 00:1a:4a:16:20:31 dynamic
2017-01-30T14:03:25.831Z|00058|binding|INFO|Claiming lport 3ef3d175-f1fa-47cb-9a5a-2bb6b1efa860 for this chassis.
2017-01-30T14:03:25.831Z|00059|binding|INFO|Claiming 00:1a:4a:16:20:61 dynamic


openflow rules (host of VM):

# ovs-ofctl dump-flows br-int | grep 00:1a:4a:16:20:61
 cookie=0x0, duration=1772.475s, table=29, n_packets=0, n_bytes=0, idle_age=1772, priority=50,metadata=0x1,dl_dst=00:1a:4a:16:20:61 actions=load:0x1->NXM_NX_REG15[],resubmit(,32)

# ovs-ofctl dump-flows br-int | grep 00:1a:4a:16:20:31
 cookie=0x0, duration=1775.567s, table=26, n_packets=0, n_bytes=0, idle_age=1775, priority=50,arp,metadata=0x1,arp_tpa=172.16.0.2,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],mod_dl_src:00:1a:4a:16:20:31,load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],load:0x1a4a162031->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xac100002->NXM_OF_ARP_SPA[],move:NXM_NX_REG14[]->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
 cookie=0x0, duration=1775.577s, table=27, n_packets=0, n_bytes=0, idle_age=1775, priority=100,udp,reg14=0x1,metadata=0x1,dl_src=00:1a:4a:16:20:31,nw_src=0.0.0.0,nw_dst=255.255.255.255,tp_src=68,tp_dst=67 actions=controller(userdata=00.00.00.02.00.00.00.00.00.01.de.10.00.00.00.63.ac.10.00.02.01.04.ff.ff.ff.00.03.04.ac.10.00.fe.36.04.ac.10.00.00.33.04.00.01.51.80.06.04.08.08.08.08,pause),resubmit(,28)
 cookie=0x0, duration=1775.577s, table=28, n_packets=0, n_bytes=0, idle_age=1775, priority=100,udp,reg0=0x8/0x8,reg14=0x1,metadata=0x1,dl_src=00:1a:4a:16:20:31,nw_src=0.0.0.0,nw_dst=255.255.255.255,tp_src=68,tp_dst=67 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],mod_dl_src:02:00:00:00:00:00,mod_nw_dst:172.16.0.2,mod_nw_src:172.16.0.0,mod_tp_src:67,mod_tp_dst:68,move:NXM_NX_REG14[]->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)

Version-Release number of selected component (if applicable):
openvswitch 2.6.90

How reproducible:
100%

Steps to Reproduce:

1. Create OVN switch, and make sure that there are settings for subnet, so VM's attached to the switch will able to get DHCP.
1. Create a VM with vNIC that has MAC address: 00:1a:4a:16:20:31
2. Hot-unplug the vNIC from the switch, and change the MAC address to: 00:1a:4a:16:20:61.
3. Use dhclient on VM to get IP.

Actual results:
VM doesn't receive IP.

Expected results:
VM should receive IP.

Additional info:

Comment 1 Mor 2017-02-01 12:14:52 UTC
Created attachment 1246622 [details]
ovn/ovs logs on host

Comment 3 Numan Siddique 2017-02-09 16:43:36 UTC
I am looking into the issue.

Comment 4 Numan Siddique 2017-02-09 16:51:55 UTC
Can you share the the outputs of below commands before and after changing the mac address.

ovn-nbctl show
ovn-sbctl dump-flows

Comment 5 Numan Siddique 2017-02-10 04:39:37 UTC
I got the required information from here - https://bugzilla.redhat.com/show_bug.cgi?id=1408121https://bugzilla.redhat.com/show_bug.cgi?id=1408121

Comment 6 Numan Siddique 2017-02-10 04:39:49 UTC
I got the required information from here - https://bugzilla.redhat.com/show_bug.cgi?id=1408121https://bugzilla.redhat.com/show_bug.cgi?id=1408121

Comment 7 Numan Siddique 2017-02-13 07:16:02 UTC
Submitted the patch upstream to fix this issue - https://mail.openvswitch.org/pipermail/ovs-dev/2017-February/328772.html

Comment 8 Lance Richardson 2017-02-14 17:44:28 UTC
Note that the upstream patch for this issue has not yet been accepted
and merged, we can backport to fast-datapath-rhel-7 when that has occurred.

Comment 9 Russell Bryant 2017-02-15 20:32:43 UTC
(In reply to Lance Richardson from comment #8)
> Note that the upstream patch for this issue has not yet been accepted
> and merged, we can backport to fast-datapath-rhel-7 when that has occurred.

I merged this patch to master and branch-2.7 upstream.

Comment 10 Lance Richardson 2017-02-16 15:09:33 UTC
Back-ported to fast-datapath-rhel-7, brew build:

   https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=538910

Comment 11 Lance Richardson 2017-02-16 16:17:02 UTC
Previous comments and bz flags indicated that this should have been
backported to fast-datapath-rhel-7, I had not noticed this and applied
the backport to fast-datapath-beta-rhel-7 instead.

Since this is an OVN fix, and OVN is not packaged in the
fast-datapath-rhel-7 branch (it is based on OVS 2.5), the
correct stream should be fast-datapath-beta-rhel-7.

Comment 13 Mor 2017-03-06 15:16:38 UTC
Verified on OVN provider and RHV setup.

Packages:

ovirt-provider-ovn-1.0-6.el7ev.noarch
openvswitch-ovn-common-2.6.1-10.git20161206.el7fdb.x86_64
openvswitch-2.6.1-10.git20161206.el7fdb.x86_64
openvswitch-ovn-central-2.6.1-10.git20161206.el7fdb.x86_64
python-openvswitch-2.6.1-10.git20161206.el7fdb.noarch

Thanks.

Comment 14 qding 2017-03-08 07:33:01 UTC
(In reply to Mor from comment #13)
> Verified on OVN provider and RHV setup.
Thanks for providing the info.

Comment 15 qding 2017-03-08 07:33:38 UTC
Reproduced with packages: openvswitch-2.6.1-8.git20161206.el7fdb

Verified with packages
[root@dell-per730-04 vnets]# rpm -q openvswitch
openvswitch-2.6.1-10.git20161206.el7fdp.x86_64
[root@dell-per730-04 vnets]# rpm -qa | grep ovn
openvswitch-ovn-central-2.6.1-10.git20161206.el7fdp.x86_64
openvswitch-ovn-common-2.6.1-10.git20161206.el7fdp.x86_64
openvswitch-ovn-host-2.6.1-10.git20161206.el7fdp.x86_64

Steps to verify

1. create logical port
ovn-nbctl dhcp-options-list
ovn-nbctl lsp-add ls1 5473c488-1990-40bb-ba3a-ab68ea49b7b3
ovn-nbctl lsp-set-addresses 5473c488-1990-40bb-ba3a-ab68ea49b7b3 "00:de:ad:00:12:34 dynamic"
ovn-nbctl lsp-set-dhcpv4-options 5473c488-1990-40bb-ba3a-ab68ea49b7b3 xxxxxxxxxxxxxx
ovn-nbctl lsp-get-dhcpv4-options 5473c488-1990-40bb-ba3a-ab68ea49b7b3

2. create vnic ethx and attached to br-int
[root@dell-per730-04 vnets]# cat vnet.xml 
<interface type='bridge'>
<target dev='vnet1234'/>
<mac address='00:de:ad:00:12:34'/>
<source bridge='br-int'/>
<virtualport type='openvswitch'>
<parameters interfaceid='5473c488-1990-40bb-ba3a-ab68ea49b7b3'/>
</virtualport>
<model type='virtio'/>
</interface>

virsh attach-device hv1_vm00 vnet.xml

3. start DHCP on vnic in guest
virsh console hv1_vm00
pkill dhclient;sleep 2; dhclient -v ethx

4. change to new mac of vnic
virsh detach-device hv1_vm00 vnet.xml 

[root@dell-per730-04 vnets]# cat vnet.xml 
<interface type='bridge'>
<target dev='vnet1234'/>
<mac address='00:de:ad:00:22:34'/>
<source bridge='br-int'/>
<virtualport type='openvswitch'>
<parameters interfaceid='5473c488-1990-40bb-ba3a-ab68ea49b7b3'/>
</virtualport>
<model type='virtio'/>
</interface>

ovn-nbctl lsp-set-addresses 5473c488-1990-40bb-ba3a-ab68ea49b7b3 "00:de:ad:00:22:34 dynamic"

5. start DHCP on vnic in guest
virsh console hv1_vm00
pkill dhclient;sleep 2; dhclient -v ethx


Note You need to log in before you can comment on or make changes to this bug.