Bug 1736486 - Redis THT templates contain malformed metadata_settings
Summary: Redis THT templates contain malformed metadata_settings
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: z4
: 14.0 (Rocky)
Assignee: Harry Rybacki
QA Contact: Jeremy Agee
URL:
Whiteboard:
Depends On:
Blocks: 1736488
TreeView+ depends on / blocked
 
Reported: 2019-08-01 18:27 UTC by Harry Rybacki
Modified: 2019-11-06 16:49 UTC (History)
2 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.3.1-0.20190513171756.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1736488 (view as bug list)
Environment:
Last Closed: 2019-11-06 16:48:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1838679 0 None None None 2019-08-01 18:27:12 UTC
OpenStack gerrit 674106 0 None MERGED Fix broken metadata_settings for redis templates 2020-02-19 15:57:55 UTC
OpenStack gerrit 678218 0 None MERGED Redis metadata using incorrect network/service 2020-02-19 15:57:55 UTC
Red Hat Product Errata RHBA-2019:3745 0 None None None 2019-11-06 16:49:19 UTC

Description Harry Rybacki 2019-08-01 18:27:13 UTC
Description of problem:

Malformed THT templates (metadata_settings specifically) for Redis are resulting in service principals not being created by noavjoin service. As a result, when during Step2 of deployment the `getcert` request fails on a permission fail.

Version-Release number of selected component (if applicable):

OSP14 and OSP13 are affected

How reproducible: 

Very


Steps to Reproduce:

1. Deploy non-HA undercloud with queens or rocky bits using FreeIPA as your CA.
2. Attempt to deploy overcloud with internal TLS via TripleO e.g.:

openstack overcloud deploy \
    --templates \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml \
    -e /home/stack/cloud-names.yaml \
    -e /home/stack/misc-bits.yaml

Actual results:

Deployment blows up during Step2 when `getcert request` is invoked to fetch a certifcate for Redis because it lacks permissions (service principal for Redis was not added to IdM).

Expected results:

Novajoin adds service principal for Redis to FreeIPA. Overcloud deploys successfully.

How to verify:

Deploy successfully non-HA environment that includes Redis service with internal TLS enabled.

Additional info:

## Overcloud deploy invocation ##

openstack overcloud deploy \
    --templates \
    --ntp-server clock1.rdu2.redhat.com \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml \
    -e /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml \
    -e /home/stack/cloud-names.yaml \
    -e /home/stack/misc-bits.yaml

## cloud-names.yaml ##

parameter_defaults:
  CloudDomain: ooo.test
  CloudName: overcloud.ooo.test
  CloudNameInternal: overcloud.internalapi.ooo.test
  CloudNameStorage: overcloud.storage.ooo.test
  CloudNameStorageManagement: overcloud.storagemgmt.ooo.test
  CloudNameCtlplane: overcloud.ctlplane.ooo.tes

## misc-bits.yaml ##

parameter_defaults:
  DnsServers: ["192.168.1.12"] # <-- FreeIPA server

## Deployment log ##

2019-08-01 18:11:32Z [overcloud-AllNodesDeploySteps-yrw4c7uy3r3v-ControllerDeployment_Step1-2mx22mczn24y.0]: SIGNAL_IN_PROGRESS Signal: deployment 0fc2d36c-fa62-4565-92fb-cf43295675ce failed (2)
2019-08-01 18:11:33Z [overcloud-AllNodesDeploySteps-yrw4c7uy3r3v-ControllerDeployment_Step1-2mx22mczn24y.0]: UPDATE_FAILED Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2019-08-01 18:11:33Z [overcloud-AllNodesDeploySteps-yrw4c7uy3r3v-ControllerDeployment_Step1-2mx22mczn24y]: UPDATE_FAILED Resource UPDATE failed: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2019-08-01 18:11:33Z [overcloud-AllNodesDeploySteps-yrw4c7uy3r3v.ControllerDeployment_Step1]: UPDATE_FAILED Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2019-08-01 18:11:33Z [overcloud-AllNodesDeploySteps-yrw4c7uy3r3v]: UPDATE_FAILED Resource UPDATE failed: Error: resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2019-08-01 18:11:33Z [AllNodesDeploySteps]: UPDATE_FAILED Error: resources.AllNodesDeploySteps.resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2019-08-01 18:11:34Z [overcloud]: UPDATE_FAILED Resource UPDATE failed: Error: resources.AllNodesDeploySteps.resources.ControllerDeployment_Step1.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2

 Stack overcloud UPDATE_FAILED

## Controller journalctl ##

Jul 29 17:41:23 overcloud-controller-0.ooo.test certmonger[19146]: 2019-07-29 17:41:23 [19146] Server at https://ipa.ooo.test/ipa/xml denied our request, giving up: 2100 (RPC failed at server. Insufficient acces
Jul 29 17:41:23 overcloud-controller-0.ooo.test certmonger[19318]: Request for certificate to be stored in file "/etc/pki/tls/certs/redis.crt" rejected by CA.
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: Could not get certificate: Execution of '/usr/bin/getcert request -I redis -f /etc/pki/tls/certs/redis.crt -c IPA -N CN=overcloud-controller-0.i
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: (/Stage[main]/Tripleo::Certmonger::Redis/Certmonger_certificate[redis]) Could not evaluate: Could not get certificate: Server at https://ipa.ooo
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: (/Stage[main]/Tripleo::Certmonger::Redis/File[/etc/pki/tls/certs/redis.crt]) Dependency Certmonger_certificate[redis] has failures: true
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: (/Stage[main]/Tripleo::Certmonger::Redis/File[/etc/pki/tls/certs/redis.crt]) Skipping because of failed dependencies
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: (/Stage[main]/Tripleo::Certmonger::Redis/File[/etc/pki/tls/private/redis.key]) Dependency Certmonger_certificate[redis] has failures: true
Jul 29 17:41:23 overcloud-controller-0.ooo.test puppet-user[18574]: (/Stage[main]/Tripleo::Certmonger::Redis/File[/etc/pki/tls/private/redis.key]) Skipping because of failed dependencies

Comment 1 Harry Rybacki 2019-08-01 18:49:10 UTC
Fix for stable/rocky proposed upstream.

Comment 4 Harry Rybacki 2019-08-05 18:01:03 UTC
Upstream review has merged. Downstream cherry-pick review posted.

Comment 5 Harry Rybacki 2019-08-05 18:24:12 UTC
Downstream build completed. Updating FIV and moving RHBZ to MODIFIED.

Comment 6 Harry Rybacki 2019-08-23 13:04:09 UTC
Original fix is incomplete. Adding external tracker to another review and moving this RHBZ to ON_DEV.

Comment 7 Harry Rybacki 2019-08-26 13:09:57 UTC
Upstream review has merged. Proposing cherry-pick to downstream branches and moving this RHBZ back to POST.

Comment 8 Harry Rybacki 2019-08-26 13:56:37 UTC
Downstream review has merged. New build created and noted in FIV. Moving RHBZ to MODIFIED.

Comment 15 errata-xmlrpc 2019-11-06 16:48:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3745


Note You need to log in before you can comment on or make changes to this bug.