Bug 1465161
Summary: | Seeing ipv6 duplicate address, causing network issues | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | rlopez |
Component: | openstack-neutron | Assignee: | Daniel Alvarez Sanchez <dalvarez> |
Status: | CLOSED ERRATA | QA Contact: | Toni Freger <tfreger> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | 10.0 (Newton) | CC: | amuller, chrisw, dalvarez, nyechiel, rlopez, srevivo, tfreger |
Target Milestone: | z6 | Keywords: | TestOnly, Triaged, ZStream |
Target Release: | 10.0 (Newton) | ||
Hardware: | Unspecified | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | openstack-neutron-9.3.1-9.el7ost | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2017-11-15 13:53:31 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
rlopez
2017-06-26 20:37:07 UTC
Not sure if I'm running into something like: https://bugs.launchpad.net/neutron/+bug/1459856 Also found this (older): https://bugs.launchpad.net/nova/+bug/1011134/comments/2 FYI: I've gone into every osp instance regarding this stack and disabled ipv6 within sysctl.conf as follows and rebooted. net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 I STILL see the dups :-/ The duplicate and the interface it is complaining about: Jun 27 14:33:56 overcloud-controller-0 kernel: IPv6: qg-d5aa7c20-41: IPv6 duplicate address 2620:52:0:1372:f816:3eff:fe97:560 detected! Look at the netns of the router running that qg-d5aa7c20-41 59: qg-d5aa7c20-41: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1496 qdisc noqueue state UNKNOWN qlen 1000 link/ether fa:16:3e:97:05:60 brd ff:ff:ff:ff:ff:ff inet 10.19.114.187/23 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.198/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.199/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.200/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.207/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.211/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.213/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet 10.19.114.188/32 scope global qg-d5aa7c20-41 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe97:560/64 scope link nodad valid_lft forever preferred_lft forever 60: qr-e0eb81e9-5d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 qdisc noqueue state UNKNOWN qlen 1000 link/ether fa:16:3e:bc:41:15 brd ff:ff:ff:ff:ff:ff inet 172.22.10.1/24 scope global qr-e0eb81e9-5d valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:febc:4115/64 scope link nodad valid_lft forever preferred_lft forever The only one that has a inet6 is an instance labeled test-devs which on the system itself has the ipv6 disabled but for some reason the inet6 exists on the qg but not within the instance itself there are no inet6 [cloud-user@test-devs ~]$ sudo -i [root@test-devs ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 qdisc pfifo_fast state UP qlen 1000 link/ether fa:16:3e:5c:e5:5e brd ff:ff:ff:ff:ff:ff inet 172.22.10.10/24 brd 172.22.10.255 scope global dynamic eth0 valid_lft 86288sec preferred_lft 86288sec [root@test-devs ~]# cat /etc/sysctl.conf # sysctl settings are defined through files in # /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/. # # Vendors settings live in /usr/lib/sysctl.d/. # To override a whole file, create a new file with the same in # /etc/sysctl.d/ and put new settings there. To override # only specific settings, add a file with a lexically later # name in /etc/sysctl.d/ and put new settings there. # # For more information, see sysctl.conf(5) and sysctl.d(5). net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 [root@test-devs ~]# cat /proc/sys/net/ipv6/conf/default/disable_ipv6 1 When I reboot this system, or shut it off, the duplicate address go quiet. As soon as its back up, the dup messages come back up. Disabling ipv6 via the netns removes the duplicate: ip netns exec qrouter-43fc42e1-f6ee-40bf-9657-1f463ab5f901 sysctl -w net.ipv6.conf.qr-e0eb81e9-5d.disable_ipv6=1 I went ahead and did for all ip netns exec qrouter-43fc42e1-f6ee-40bf-9657-1f463ab5f901 sysctl -w net.ipv6.conf.all.disable_ipv6=1 However, the above seems like a total hack. Why does an inet6 get linked if its not being used to begin with? More info that might be useful: https://bugs.launchpad.net/mos/+bug/1596846 Can I get a response please? Assigned to Daniel for triage. Could this be a duplicate of [0]? If so, it would be fixed in openstack-neutron-9.3.1-9.el7ost We could confirm by capturing traffic in the controllers. However, if it's easy to replicate, I would try that version since it looks very similar and disabling ipv6 forwarding on the backup instance could possibly solve it. Or at least: ip netns exec qrouter-43fc42e1-f6ee-40bf-9657-1f463ab5f901 sysctl -w net.ipv6.conf.all.forwarding=0 [0] https://bugzilla.redhat.com/show_bug.cgi?id=1426735 Hi Daniel, Thanks for your reply. Question: Can I easily implement this by just upgrading my controllers with that specific RPM package and restarting neutron? Would I always require having to disable forwarding for every router that is created? Any idea why ipv4 and ipv6 are not happy with each other when both enabled? Especially since I'm not even using the ipv6... (In reply to rlopez from comment #9) > Hi Daniel, > > Thanks for your reply. > > Question: > > Can I easily implement this by just upgrading my controllers with that > specific RPM package and restarting neutron? Yes, that should be fine. > > Would I always require having to disable forwarding for every router that is > created? The RPM package includes the patch that does it automatically every time a failover occurs. > > Any idea why ipv4 and ipv6 are not happy with each other when both enabled? > Especially since I'm not even using the ipv6... In this case, as the interface has ipv6 forwarding enabled, it will automatically get subscribed to several multicast groups. Therefore, when multicast traffic is received, it will respond to the ToR switch and this will learn the MAC address from the backup node on its port, disrupting traffic to the master. Please, note that I'll be away for the next two weeks and I won't be able to look into this case until then. Sorry for the inconvenience. According to our records, this should be resolved by openstack-neutron-9.4.1-1.el7ost. This build is available now. Thanks Daniel, we can consider it verified on latest osp10 with openstack-neutron-9.4.1-2.el7ost.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3234 |