Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1174215

Summary: IP addresses are not properly reported with IPv6 stateful subnets
Product: Red Hat OpenStack Reporter: Nir Magnezi <nmagnezi>
Component: openstack-neutronAssignee: Ihar Hrachyshka <ihrachys>
Status: CLOSED ERRATA QA Contact: Nir Magnezi <nmagnezi>
Severity: high Docs Contact:
Priority: medium    
Version: 6.0 (Juno)CC: chrisw, ihrachys, lpeer, mlopes, nyechiel, oblaut, sclewis, tfreger, yeylon
Target Milestone: z2Keywords: ZStream
Target Release: 6.0 (Juno)   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-neutron-2014.2.2-1.el7ost Doc Type: Known Issue
Doc Text:
Stateful DHCPv6 is currently nonfunctional in the Red Hat Enterprise Linux OpenStack Platform 6 (Juno) GA as a result of upstream bug 1377843. Further details are available at https://bugs.launchpad.net/neutron/+bug/1377843.
Story Points: ---
Clone Of:
: 1179756 (view as bug list) Environment:
Last Closed: 2015-04-07 14:57:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1168526, 1177797, 1195356, 1196533    
Bug Blocks: 1044271    

Description Nir Magnezi 2014-12-15 12:11:32 UTC
Description of problem:
=======================
I Created n radvd IPv6 subnet with:
 - ipv6_ra_mode: dhcpv6-stateful
 - ipv6_address_mode: dhcpv6-stateful

Meaning: 
a. The Neutron DHCP agent (dnsmasq) provides IP addresses and additional parameters.
b. Neutron router (radvd) sends out RAs.

Version-Release number of selected component (if applicable):
=============================================================
RHEL-OSP6-Beta: openstack-neutron-2014.2.1-2.el7ost.noarch

How reproducible:
=================
2/2

Steps to Reproduce:
===================
1. Create an IPv6 neutron network:
   # neutron net-create tenant_a_radvd_stateful --shared --provider:physical_network=ipv6_vlan_range --provider:network_type=vlan --provider:segmentation_id=64

2. Create an IPv6 subnet:
   # neutron subnet-create <IPv6_net_id> 2001:65:65:65::1/64 --name tenant_a_radvd_stateless_subnet --ipv6-ra-mode dhcpv6-stateless --ipv6-address-mode dhcpv6-stateless --dns-nameserver 2001:4860:4860::8888 --ip-version 6
3. Create a neutron router:
   # neutron router-create router1

4. Attach subnet to the router
   # neutron router-interface-add <router_id> <ipv6_subnet>

5. boot an instance with that network
   # nova boot tenant_a_instance_radvd_stateful --flavor m1.small --image <image_id> --key-name keypair --security-groups default --nic net-id=<ipv6_net_id>


Actual results:
===============
# nova show <instance_id> | grep network
| tenant_a_radvd_stateful network      | 2001:64:64:64::5                                                      |

# neutron port-show <port_id> | grep fixed_ips
| fixed_ips             | {"subnet_id": "<subnet_id>", "ip_address": "2001:64:64:64::5"} |

Dispite that fact that the port mac address matches the instance NIC, the IP obtained by the instance is different.

Expected results:
=================
In this case, the NIC IP address is: 2001:64:64:64:f816:3eff:fe2b:6197

Additional info:
================
1. Instance guest image: rhel7
2. Hypervisor: rhel7

Comment 1 Nir Magnezi 2014-12-15 12:22:33 UTC
Some more info:
===============
radvd:
======
# cat /var/lib/neutron/ra/76f98730-bb40-40e7-bc57-e89f120efd4a.radvd.conf
interface qr-23b0614a-cd
{
   AdvSendAdvert on;
   MinRtrAdvInterval 3;
   MaxRtrAdvInterval 10;
   prefix 2001:65:65:65::/64
   {
        AdvOnLink on;
        AdvAutonomous on;
   };
};
interface qr-9741429b-bf
{
   AdvSendAdvert on;
   MinRtrAdvInterval 3;
   MaxRtrAdvInterval 10;
   prefix 2001:63:63:63::/64
   {
        AdvOnLink on;
        AdvAutonomous on;
   };
};
interface qr-ace4c312-6c
{
   AdvSendAdvert on;
   MinRtrAdvInterval 3;
   MaxRtrAdvInterval 10;
};


dnsmasq:
========
nobody   12197     1  0 Dec14 ?        00:00:01 dnsmasq --no-hosts --no-resolv --strict-order --bind-interfaces --interface=tap74f92a94-90 --except-interface=lo --pid-file=/var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/host --addn-hosts=/var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/opts --leasefile-ro --dhcp-range=set:tag0,2001:64:64:64::,static,86400s --dhcp-lease-max=16777216 --conf-file=/etc/neutron/dnsmasq-neutron.conf --domain=openstacklocal


# cat /var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/host
fa:16:3e:07:2d:5d,host-2001-64-64-64--1.openstacklocal,[2001:64:64:64::1]
fa:16:3e:2b:61:97,host-2001-64-64-64--5.openstacklocal,[2001:64:64:64::5]
fa:16:3e:31:94:88,host-2001-64-64-64--6.openstacklocal,[2001:64:64:64::6]
fa:16:3e:a9:e0:e8,host-2001-64-64-64--7.openstacklocal,[2001:64:64:64::7]


# cat /var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/addn_hosts
2001:64:64:64::1	host-2001-64-64-64--1.openstacklocal host-2001-64-64-64--1
2001:64:64:64::5	host-2001-64-64-64--5.openstacklocal host-2001-64-64-64--5
2001:64:64:64::6	host-2001-64-64-64--6.openstacklocal host-2001-64-64-64--6
2001:64:64:64::7	host-2001-64-64-64--7.openstacklocal host-2001-64-64-64--7


# cat /var/lib/neutron/dhcp/308b1dd1-185c-451b-8912-2a323616acce/opts
tag:tag0,option6:dns-server,[2001:4860:4860::8888]
tag:tag0,option6:domain-search,openstacklocal

# cat /etc/neutron/dnsmasq-neutron.conf
dhcp-option-force=26,1400

Comment 3 Nir Magnezi 2014-12-15 13:04:43 UTC
I've found that the issue reproduces with provider statful subnet as well:
 - ipv6_ra_mode: not set
 - ipv6-address-mode: dhcpv6-stateful

Comment 5 Nir Magnezi 2014-12-29 20:04:54 UTC
Restested with: openstack-neutron-2014.2.1-5.el7ost.noarch

Now, The instance does not obtain an IPv6 address at all.
Will post more as I find it (any maybe file another bug for this issue).

details regarding dnsmasq:
[root@puma07 neutron]# cat /var/lib/neutron/dhcp/53800104-7e5f-4dd7-8888-7042070cc73a/host
fa:16:3e:a8:4e:04,host-2001-64-64-64--2.openstacklocal,[2001:64:64:64::2]
fa:16:3e:b4:04:70,host-2001-64-64-64--8.openstacklocal,[2001:64:64:64::8]
[root@puma07 neutron]# cat /var/lib/neutron/dhcp/53800104-7e5f-4dd7-8888-7042070cc73a/addn_hosts
2001:64:64:64::2	host-2001-64-64-64--2.openstacklocal host-2001-64-64-64--2
2001:64:64:64::8	host-2001-64-64-64--8.openstacklocal host-2001-64-64-64--8
[root@puma07 neutron]# cat /var/lib/neutron/dhcp/53800104-7e5f-4dd7-8888-7042070cc73a/opts
tag:tag0,option6:dns-server,[2001:4860:4860::8888]
[root@puma07 neutron]# cat /etc/neutron/dnsmasq-neutron.conf
dhcp-option-force=26,1400

Comment 6 Nir Magnezi 2014-12-29 22:23:22 UTC
dnsmasq does not seem to respond to the instance requests.

00:16:20.907593 ethertype IPv4, IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 308)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from fa:16:3e:97:77:72 (oui Unknown), length 280, xid 0x74d4fe05, Flags [none] (0x0000)
	  Client-Ethernet-Address fa:16:3e:97:77:72 (oui Unknown)
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message Option 53, length 1: Discover
	    Client-ID Option 61, length 7: ether fa:16:3e:97:77:72
	    MSZ Option 57, length 2: 576
	    Parameter-Request Option 55, length 7: 
	      Subnet-Mask, Default-Gateway, Domain-Name-Server, Hostname
	      Domain-Name, BR, NTP
	    Vendor-Class Option 60, length 12: "udhcp 1.20.1"
	    END Option 255, length 0


This does not seem to be related to SELinux.

Comment 7 Nir Magnezi 2014-12-30 13:03:06 UTC
Filed a separate bug for this issue: Bug 1177797

Comment 8 lpeer 2015-01-05 08:54:44 UTC
In the latest version this bug does not reproduce because of Bug 1177797.
After fixing BZ 1177797 we need to see if this bug can be reproduced.

Comment 9 Ihar Hrachyshka 2015-01-07 12:44:53 UTC
The problem is two folded:

1. for stateful subnets, we don't set M flag in RAs, so clients are not aware of the fact that DHCP server is able to provide them an address.

2. we use old dnsmasq (2.66) that does not support MAC address matching for IPv6. We should upgrade it to at least 2.67.

Comment 11 Nir Yechiel 2015-01-08 13:27:54 UTC
Hi Martin,

Can you please review this bug for inclusion in the GA Release Notes?

Thanks,
Nir

Comment 12 Martin Lopes 2015-01-19 05:26:52 UTC
Updated doc text, reviewed flags and doc type for inclusion in GA docs.

Comment 13 Ihar Hrachyshka 2015-02-20 14:28:14 UTC
Nir, please retest with A1 + the following scratch build for dnsmasq:

https://brewweb.devel.redhat.com/taskinfo?taskID=8759868

Comment 15 Ihar Hrachyshka 2015-03-09 10:40:49 UTC
Note to errata release manager: it should depend on RHEL 7.1.z errata [1]. (I was told errata tool supports such dependencies.)

[1]: https://errata.devel.redhat.com/advisory/20016

Comment 18 Ihar Hrachyshka 2015-03-19 12:05:22 UTC
The bug must be tested with dnsmasq-2.66-13, as in errata [1] that is linked from openstack one [2].

[1]: https://errata.devel.redhat.com/advisory/20016
[2]: https://errata.devel.redhat.com/advisory/20123

Comment 19 Nir Magnezi 2015-03-23 21:27:02 UTC
Verified NVR:
openstack-neutron-2014.2.2-5.el7ost.noarch
dnsmasq-2.66-13.el7_1.x86_64

Verification Steps:
===================
1. Create a Neutron Network & DHCPv6 Stateful subnet:
   # neutron net-create tenant_a_radvd_stateful
   # neutron subnet-create <net_id> 2001::1/64 --name stateful_subnet --ipv6-ra-mode dhcpv6-stateful --ipv6-address-mode dhcpv6-statefu --ip-version 6

2. Create a Router:
   # neutron router-create router1

3. Attach router interface:
   # neutron router-interface-add <router_id> <subnet_id>

4. Spawn an instance
   # nova boot instance_stateful --flavor m1.medium --image <image_id> --nic net-id=<net_id>

Result:
=======
1. Instance Obtained IPv6 Address.
2. The address matches the neutron port.

Comment 21 errata-xmlrpc 2015-04-07 14:57:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0785.html