Bug 1622660 - Ping sometimes fails to controller nodes.
Summary: Ping sometimes fails to controller nodes.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z3
: 13.0 (Queens)
Assignee: Bob Fournier
QA Contact: mlammon
URL:
Whiteboard:
Depends On:
Blocks: 1635423
TreeView+ depends on / blocked
 
Reported: 2018-08-27 17:10 UTC by Darin Sorrentino
Modified: 2019-04-08 14:14 UTC (History)
6 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.7-2.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1635423 (view as bug list)
Environment:
Last Closed: 2018-11-13 22:28:47 UTC
Target Upstream Version:
Embargoed:
lmarsh: needinfo-
lmarsh: needinfo-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1793598 0 None None None 2018-09-20 20:58:45 UTC
OpenStack gerrit 604229 0 None MERGED Ping default gateways before controllers 2020-10-30 03:45:20 UTC
OpenStack gerrit 607367 0 None MERGED Ping default gateways before controllers 2020-10-30 03:45:20 UTC
Red Hat Product Errata RHBA-2018:3587 0 None None None 2018-11-13 22:29:30 UTC

Description Darin Sorrentino 2018-08-27 17:10:52 UTC
Description of problem:

While deploying an Overcloud that is utilizing IPv6 addressing, we sometimes see where the pinging to the controllers fails as part of the deployment validation which then results in a failed deploy.  Through troubleshooting, we've determined that pings to the gateways do not fail and when pinging the gateway, it appears to "fix" the ability to ping to the Controller Nodes.

Within our team we've tried to figure out why this would be the case and one of our team members pointed out that IPV6 does not have ARP, it has neighbor solicitation and router solicitation, and sometimes neighbor solicitation isn't on or doesn't work, in which case you -have- to establish connectivity to the router before any other host communication works.  He said this only applies to non eui64 (mac address embedded in ipv6 address) setups.

We're not sure if that's what is happening here, however, we do know if we continually run an Ansible Playbook that connects tot he overcloud nodes and has them ping their default gateways during deployment, the deployment succeeds 100% of the time.

In looking at the all_nodes.sh validation script:

/usr/share/openstack-tripleo-heat-templates/validation-scripts/all-nodes.sh

We noticed that it is performing 2 tests, ping_controller_ips and ping_default_gateways, in that order.  We believe flipping these around will correct this issue.

This BZ is to have those 2 tests swapped so the ping of the gateways happens before pinging the controllers.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 6 bjacot 2018-10-31 16:40:49 UTC
Verified on OSP13 recommended change is included.

[stack@undercloud-0 ~]$ cat core_puddle_version 
2018-10-24.1
[stack@undercloud-0 ~]$ rpm -qa | grep openstack-tripleo-heat-templates
openstack-tripleo-heat-templates-8.0.7-4.el7ost.noarch

Comment 10 errata-xmlrpc 2018-11-13 22:28:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3587


Note You need to log in before you can comment on or make changes to this bug.