Bug 1299294 - keystone errors in Overcloud deployment with IPv6 enabled
Summary: keystone errors in Overcloud deployment with IPv6 enabled
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
Target Milestone: y3
: 7.0 (Kilo)
Assignee: Jiri Stransky
QA Contact: Marius Cornea
Depends On:
TreeView+ depends on / blocked
Reported: 2016-01-18 00:40 UTC by Dan Sneddon
Modified: 2017-10-30 11:27 UTC (History)
9 users (show)

Fixed In Version: openstack-tripleo-heat-templates-0.8.6-107.el7ost
Doc Type: Bug Fix
Doc Text:
In an IPv6-based Overcloud, RabbitMQ lacked some IPv6-specific options when starting. This caused RabbitMQ to fail on some nodes, which cause other services to fail due to Pacemaker contraints. This fix adds the IPv6-specific options. Now RabbitMQ and other services start successfully.
Clone Of:
Last Closed: 2016-02-18 16:50:13 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:0264 normal SHIPPED_LIVE Red Hat Enterprise Linux OSP 7 director Bug Fix Advisory 2016-02-18 21:41:29 UTC

Description Dan Sneddon 2016-01-18 00:40:20 UTC
Description of problem:
When IPv6 is enabled on the latest poodle, the deployment is failing with an error that mentions Keystone. I'm not sure where the error is coming from.

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates.noarch     0.8.6-103.el7ost

How reproducible:

Steps to Reproduce:
1. Deploy with IPv6 enabled

Actual results:
Error: Could not prefetch keystone_tenant provider 'openstack': undefined method `collect' for nil:NilClass\u001b[0m\n\u001b[1;31mError: Could not prefetch keystone_role provider 'openstack': undefined method `collect' for nil:NilClass\u001b[0m\n\u001b[1;31m
Error: Could not prefetch keystone_user provider 'openstack': undefined method `collect' for nil:NilClass\u001b[0m\n\u001b[1;31mError: /Stage[main]/Keystone::Roles::Admin/Keystone_user_role[admin@admin]: Could not evaluate: undefined method `empty?' for nil:NilClass\u001b[0m\n\u001b[1;31

Expected results:
This shouldn't happen.

Additional info:

Comment 2 Sofer Athlan-Guyot 2016-01-18 09:25:12 UTC
This is usually due to the kesytone service has not started properly.

The issue here is that it tries to grab the domain and as the response is coming back empty it triggers this exception.

Comment 5 Gilles Dubreuil 2016-01-19 05:49:28 UTC
I seems to me OPM is in a wrong state.
Effectively Keystone module is quite old (openstack-puppet-modules-2015.1.8-41.el7ost.noarch latest puddle + poodle (OSP 7):
commit 920421a70d33e751c4ec1a665e74819d1c1354d0
Date:   Tue Jun 30 14:37:26 2015 +0000

All other modules seems way behind as well. As a matter of fact, testing with latest keystone module from stable/liberty branch doesn't work because many dependent modules are behind too.

Comment 6 Sofer Athlan-Guyot 2016-01-19 15:00:01 UTC
Oki, the problem is in that puppet is using the admin_bind_host in /etc/keystone/keystone.conf value to find the keystone server.  This variable is set to the local ip, not the vip.

Then when puppet try to get to the keystone, it works only on the one where the keystone server is started.

Pacemaker policy for keystone is a clone resource, so only one keystone is started.

Comment 7 Sofer Athlan-Guyot 2016-01-20 05:12:58 UTC
The problem is actually unrelated to keystone.  The issue here is that rabbimq is failing to start on 2 out of 3 nodes.  As there is a pacemaker constraint, saying the keystone start after rabbitmq, the keystone service was not started on 2 nodes.

The rabbimq tht seems to be missing some IPv6 information to start correctly.

Comment 9 Gilles Dubreuil 2016-01-20 06:25:44 UTC
Please ignore my comment#5, OSP 7 is Kilo. My confusion came from the fact it's pointing to a commit (the one corresponding to kilo) on liberty branch.

Comment 11 Marius Cornea 2016-02-12 12:44:30 UTC
I wasn't able to reproduce this with openstack-tripleo-heat-templates-0.8.6-120.el7ost.noarch.

Comment 13 errata-xmlrpc 2016-02-18 16:50:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.