Bug 1284734 - Keystone Authorization Failed
Keystone Authorization Failed
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-keystone (Show other bugs)
5.0 (RHEL 7)
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Adam Young
Depends On:
  Show dependency treegraph
Reported: 2015-11-23 22:52 EST by James Biao
Modified: 2016-04-26 10:44 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-11-24 00:28:04 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description James Biao 2015-11-23 22:52:29 EST
Description of problem:

It's a 3-node keystone cluster behind haproxy and authenticating with LDAP.

On a keystone client, when getting a token, sometimes it takes 14 seconds to complete, while sometimes it responded instantly.

On keystone node 2, a large number of errors below were found,

2015-11-23 23:00:48.397 16694 WARNING keystone.common.wsgi [-] Authorization failed. The request you have made requires authentication. from   ( is the keystone node itself)

2015-11-23 22:43:25.467 16690 WARNING keystone.common.wsgi [-] Could not find token, 6b190422df984d25a8bc8074e15cc5a6.
2015-11-23 22:56:32.247 16698 WARNING keystone.common.wsgi [-] Could not find project, f0470f8c72254aac8ccefc84544a0eea.

Under Keystone debug below errors can be observed and recurring continuously

2015-11-24 02:04:31.755 9065 DEBUG keystone.common.ldap.core [-] LDAP bind: dn=uid=713802ce461843cab2774611cdec05d8,ou=users,ou=serviceproviders,dc=cisco,dc=com simple_bind_s /usr/lib/python2.7/site-packages/keystone/common/ldap/core.py:783
2015-11-24 02:04:31.756 9061 DEBUG keystone.notifications [-] CADF Event: {'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'initiator': {'typeURI': 'service/security/account/user', 'host': {'agent': 'Apache-HttpClient/4.4.1 (Java/1.7.0_79)', 'address': ''}, 'id': 'openstack:a1eac7d3-8fae-4d0f-aa81-c93a40626069', 'name': u'e4c25ac9172d40998026dd5655f75313'}, 'target': {'typeURI': 'service/security/account/user', 'id': 'openstack:c9c58519-2847-4090-8b26-69b32b2906d1'}, 'observer': {'typeURI': 'service/security', 'id': 'openstack:96941ab4-145d-4b6e-8f5d-7f9b746c861e'}, 'eventType': 'activity', 'eventTime': '2015-11-24T02:04:31.755733+0000', 'action': 'authenticate', 'outcome': 'failure', 'id': 'openstack:b0ce921c-6fc8-4780-accd-39ce3a50f844'} _send_audit_notification /usr/lib/python2.7/site-packages/keystone/notifications.py:289
2015-11-24 02:04:31.756 9061 WARNING keystone.common.wsgi [-] Authorization failed. Invalid user / password from

When taking node 2 out of the cluster, clients are able to get token properly.

Version-Release number of selected component (if applicable):

-----------------sosreports uploaded------------
[collab-shell] https://gitlab.cee.redhat.com/gss-tools/collab-shell

    # ssh your_kerb@collab-shell.usersys.redhat.com
    # cd /cases/01543989

the following files have been downloaded and extracted on collab-shell:
Comment 1 James Biao 2015-11-24 00:28:04 EST
Issue resolved. Not a keystone issue. It's the problem with customer's LDAP server

Note You need to log in before you can comment on or make changes to this bug.