Bug 1537598

Summary:	Director deployment of keystone integration with LDAP broken
Product:	Red Hat OpenStack	Reporter:	Ken Holden <kholden>
Component:	openstack-tripleo-heat-templates	Assignee:	Emilien Macchi <emacchi>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Jeremy Agee <jagee>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	12.0 (Pike)	CC:	acanan, coldford, hrybacki, jagee, jjoyce, jschluet, mburns, pkesavar, rhel-osp-director-maint, rmascena, scohen, slinaber, tvignaud
Target Milestone:	z2	Keywords:	TestOnly, Triaged, ZStream
Target Release:	13.0 (Queens)
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-8.0.2-0.20180410061339.b937f35.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1579023 (view as bug list)		Environment:
Last Closed:	2018-07-27 10:36:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1579023

Description Ken Holden 2018-01-23 15:08:23 UTC

Description of problem:
1.  as mentioned in bug regarding upgrading from 11 to 12, you must modify a puppet tag in the following file in order for Director 12 to deploy overcloud with LDAP integration.  This is true for fresh OSP 12 deploys as well:
/usr/share/openstack-tripleo-heat-templates/docker/services/keystone.yaml

2. after successful fresh OSP 12 deploy, the keystone v3 domain for my Active Directory environment was created, however, I could not get any users to return.  Once I restarted the keystone processes within the keystone docker container, it started working:
[root@controller1 heat-admin]# docker exec -it keystone pkill -HUP -f keystone

Version-Release number of selected component (if applicable):


How reproducible:

(overcloud) [stack@director12 ~]$ openstack domain list
+----------------------------------+------------+---------+--------------------+
| ID                               | Name       | Enabled | Description        |
+----------------------------------+------------+---------+--------------------+
| 58acbdc9da0b4ada8fdf4446ce8e0ca4 | LAB        | True    |                    |
| c31ba5db93e649f888e3d3c2aa92a929 | heat_stack | True    |                    |
| default                          | Default    | True    | The default domain |
+----------------------------------+------------+---------+--------------------+
(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

[root@controller1 heat-admin]# docker exec -it keystone pkill -HUP -f keystone

(overcloud) [stack@director12 ~]$ openstack user list --domain lab



+------------------------------------------------------------------+---------------+
| ID                                                               | Name          |
+------------------------------------------------------------------+---------------+
| 7ebf8923d6a15322c3e7b611d8e9028bd4a70715199f64bba59b83c434b3ab36 | Administrator |
| 49c22aa306719865a691b875f70de6dfcfc41da0e3bad2d82f15abdae6912c7a | Guest         |
| aae9ca159631e25687a6fdbca64fbd1933b2380fe00a0ea3c05d539c54099440 | WIN2K8SVR$    |
| 809b820c83b179dec31864db0d17ba3341422765632aec4be9efe2df2ee27502 | krbtgt        |
| 7662775af3175eda8bad21fd8766d848c31a66f6679fe85e6a6c2092364aedd0 | svc.acct1     |
| 11d4448b62a3f6b49b8ce483e7791ecba43525c2ccffbea5407fc72e19b58f35 | svc.acct2     |
| 47540c6e6c444eb03f03bebbc48730d5d1d3866811260e8f84c5d085aa40b3c6 | kholden       |
| ee7b53d02827adb2ed6f808a07c3b9e20ee9f56b0b250c3891948f6aa5400078 | svc.acct3     |
+------------------------------------------------------------------+---------------+
(overcloud) [stack@director12 ~]$ 


Steps to Reproduce:
1. on directory, run `sed -i 's/puppet_tags\: keystone_config/puppet_tags\: keystone_config,keystone_domain_config/' /usr/share/openstack-tripleo-heat-templates/docker/services/keystone.yaml`

2. deploy fresh OSP 12 using keystone_domain_specific_ldap_backend.yaml template
3. run source ~/overcloudrc.v3; openstack user list --domain DOMAIN_NAME.   this will result in no users being returned

Actual results:
(overcloud) [stack@director12 ~]$ openstack domain list
+----------------------------------+------------+---------+--------------------+
| ID                               | Name       | Enabled | Description        |
+----------------------------------+------------+---------+--------------------+
| 58acbdc9da0b4ada8fdf4446ce8e0ca4 | LAB        | True    |                    |
| c31ba5db93e649f888e3d3c2aa92a929 | heat_stack | True    |                    |
| default                          | Default    | True    | The default domain |
+----------------------------------+------------+---------+--------------------+
(overcloud) [stack@director12 ~]$ openstack user list --domain lab


Expected results:

(overcloud) [stack@director12 ~]$ openstack user list --domain lab

+------------------------------------------------------------------+---------------+
| ID                                                               | Name          |
+------------------------------------------------------------------+---------------+
| 7ebf8923d6a15322c3e7b611d8e9028bd4a70715199f64bba59b83c434b3ab36 | Administrator |
| 49c22aa306719865a691b875f70de6dfcfc41da0e3bad2d82f15abdae6912c7a | Guest         |
| aae9ca159631e25687a6fdbca64fbd1933b2380fe00a0ea3c05d539c54099440 | WIN2K8SVR$    |
| 809b820c83b179dec31864db0d17ba3341422765632aec4be9efe2df2ee27502 | krbtgt        |
| 7662775af3175eda8bad21fd8766d848c31a66f6679fe85e6a6c2092364aedd0 | svc.acct1     |
| 11d4448b62a3f6b49b8ce483e7791ecba43525c2ccffbea5407fc72e19b58f35 | svc.acct2     |
| 47540c6e6c444eb03f03bebbc48730d5d1d3866811260e8f84c5d085aa40b3c6 | kholden       |
| ee7b53d02827adb2ed6f808a07c3b9e20ee9f56b0b250c3891948f6aa5400078 | svc.acct3     |
+------------------------------------------------------------------+---------------+

Additional info:
This is how I fixed it after deployment:
1. ssh to all controllers and run sudo docker exec -it keystone pkill -HUP -f keystone

2. run source ~/overcloudrc.v3; openstack user list --domain DOMAIN_NAME.   this will result in LDAP users returned (given your LDAP configs are correct)

Comment 1 Ken Holden 2018-01-24 14:14:29 UTC

Just did a fresh OSP12 deploy and AD users were returned immediately after deploy without any restart of keystone services, however, like previous versions of OSP, i would get inconsistent responses (i.e. group or user lists would sometimes respond with users / groups and sometimes just empty results).  Once I did docker restart keystone on controllers, my list results are 100% consistent.

Comment 2 Raildo Mascena de Sousa Filho 2018-02-08 15:49:21 UTC

We verified that in a non-containerized deployment we have that keystone-restart command: https://github.com/openstack/puppet-keystone/blob/a55b9e4efe956fded7030baddc0f6a342be1d76d/manifests/ldap_backend.pp#L610 but that same command missing in the keystone docker script https://github.com/openstack/tripleo-heat-templates/blob/master/docker/services/keystone.yaml#L195, we may need to re-run the part that generates the md5 of the config and start the containers with paunch if we expect docker_puppet_tasks to modify the config files

Comment 4 Harry Rybacki 2018-05-16 19:26:08 UTC

*** Bug 1572219 has been marked as a duplicate of this bug. ***

Comment 7 Jon Schlueter 2018-06-27 19:41:05 UTC

According to our records, this should be resolved by openstack-tripleo-heat-templates-8.0.2-38.el7ost.  This build is available now.