Bug 1404605

Summary: Undercloud installation fails during keystone-manage bootstrap
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: python-passlibAssignee: Nathan Kinder <nkinder>
Status: CLOSED ERRATA QA Contact: Marius Cornea <mcornea>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 11.0 (Ocata)CC: augol, ayoung, dbecker, dsavinea, emacchi, jdennis, jschluet, kbasil, lhh, mburns, mcornea, morazi, nkinder, panbalag, rhel-osp-director-maint, sclewis, skinjo, slinaber, srevivo, tvignaud
Target Milestone: Upstream M3Keywords: Triaged
Target Release: 11.0 (Ocata)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-passlib-1.7.0-1.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-17 19:51:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marius Cornea 2016-12-14 08:52:58 UTC 
Description of problem:
Undercloud installation fails during keystone-manage bootstrap.

Version-Release number of selected component (if applicable):
openstack-keystone-11.0.0-0.20161130231538.feeab95.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Install undercloud

Actual results:
Installation fails.

Expected results:
Installation is successful.

Additional info:
In /var/log/keystone/keystone.log:

2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi [req-00ab0bab-5cd9-4100-89fb-91ee33d724ce - - - - -] type object 'sha512_crypt' has no attribute 'hash'
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi Traceback (most recent call last):
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/wsgi.py", line 228, in __call__
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     result = method(req, **params)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/controller.py", line 164, in inner
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return f(self, request, *args, **kwargs)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/controllers.py", line 220, in create_user
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     ref, initiator=request.audit_initiator
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/manager.py", line 123, in wrapped
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     __ret_val = __f(*args, **kwargs)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 410, in wrapper
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 420, in wrapper
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return f(self, *args, **kwargs)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/core.py", line 925, in create_user
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     ref = driver.create_user(user['id'], user)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/sql/core.py", line 429, in wrapper
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return method(*args, **kwargs)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/identity/backends/sql.py", line 121, in create_user
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     user = utils.hash_user_password(user)
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/utils.py", line 129, in hash_user_password
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return dict(user, password=hash_password(password))
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi   File "/usr/lib/python2.7/site-packages/keystone/common/utils.py", line 135, in hash_password
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi     return passlib.hash.sha512_crypt.hash(
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi AttributeError: type object 'sha512_crypt' has no attribute 'hash'
2016-12-14 03:45:51.627 28868 ERROR keystone.common.wsgi
Comment 1 Emilien Macchi 2016-12-19 21:41:31 UTC 
Sounds like openstack-keystone-11.0.0-0.20161130231538.feeab95.el7ost.noarch is pretty "old", could you reproduce the bug with recent packaging?

We don't have the problem in upstream CI, so i'm curious how could you hit this one.
Comment 2 Marius Cornea 2016-12-20 09:18:38 UTC 
(In reply to Emilien Macchi from comment #1)
> Sounds like openstack-keystone-11.0.0-0.20161130231538.feeab95.el7ost.noarch
> is pretty "old", could you reproduce the bug with recent packaging?
> 
> We don't have the problem in upstream CI, so i'm curious how could you hit
> this one.

This is what we currently have in the latest downstream build. Maybe we just need a new import from RDO so we can get a working version and move forward. At this moment this is an early blocker for the downstream testing of OSP11.
Comment 4 Dimitri Savineau 2017-01-06 20:46:35 UTC 
I can reproduce the bug with the latest OSP11 puddle (2017-01-06.1).

This seems to be related to the python-passlib version (1.6) present in RHOS which is older than the upstream keystone requirement (1.7) [1]. The commit removing the compatibility 1.6/1.7 is [2]

$ rpm -qa python-passlib
python-passlib-1.6.5-1.1.el7ost.noarch
$ rpm -qa|grep keystone
python-keystoneauth1-2.16.0-0.20161207153835.3e2ea6a.el7ost.noarch
python-keystonemiddleware-4.12.0-0.20161219135615.e0e9a0e.el7ost.noarch
python-keystoneclient-3.8.0-0.20161202104120.f7c1d45.el7ost.noarch
puppet-keystone-10.0.0-0.20161209230230.07616c6.el7ost.noarch
python-keystone-11.0.0-0.20161219201930.c69df5b.el7ost.noarch
openstack-keystone-11.0.0-0.20161219201930.c69df5b.el7ost.noarch

[1] https://github.com/openstack/keystone/commit/54d2ecc32a9042c9d6c22449f459789bbc864c29
[2] https://github.com/openstack/keystone/commit/71cde670d5b7f2e9e16d860545d0c36aee115dad
Comment 9 Amit Ugol 2017-02-05 06:56:18 UTC 
Changing QA, Nathan was picked automatically.
Comment 11 errata-xmlrpc 2017-05-17 19:51:14 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245