Bug 2011983 - Certificates renewed with each update to the overcloud using novajoin and IPA
Summary: Certificates renewed with each update to the overcloud using novajoin and IPA
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: puppet-tripleo
Version: 16.1 (Train)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: z9
: 16.1 (Train on RHEL 8.2)
Assignee: Grzegorz Grasza
QA Contact: Joe H. Rahme
URL:
Whiteboard:
Depends On:
Blocks: 2133845
TreeView+ depends on / blocked
 
Reported: 2021-10-07 20:38 UTC by Tommy Doucet
Modified: 2024-09-18 19:39 UTC (History)
10 users (show)

Fixed In Version: puppet-tripleo-11.7.1-2.20230320124918.50f0601.el8ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2133845 (view as bug list)
Environment:
Last Closed: 2024-09-18 19:39:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 822244 0 None stable/train: MERGED puppet-tripleo: [train-only] certmonger: track change in service's private keys (I48658413f69a68dcad0a2f24ea66fe027e987f... 2022-11-01 16:49:55 UTC
OpenStack gerrit 822313 0 None stable/train: MERGED tripleo-heat-templates: [train-only] Make principal realms configuratble in certs (I0a217b4a457881367de27414faca347e50f2... 2022-11-01 16:50:01 UTC
Red Hat Issue Tracker OSP-10276 0 None None None 2021-11-15 12:37:32 UTC

Description Tommy Doucet 2021-10-07 20:38:47 UTC
Description of problem:
From https://access.redhat.com/support/cases/03039688 :
When running an update to the overcloud for any reason we noticed that all the RHOSP related certificates being tracked by certmonger are being removed. We have a rather large environment and recently we've been adding around a dozen nodes a time and we find it unnecessary to be renewing those certificates multiple times a day or everyday. Is there a way to prevent this from happening?

Version-Release number of selected component (if applicable):
-

Steps to Reproduce:
- We use novajoin to register the undercloud with IPA.
- Add the ipa_otp parameter to undercloud.conf with the password provided from novajoin before the undercloud install.
- Include the following templates
    -  /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml
    - /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml
    - /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml
    - homedir/cloudname.yaml

Actual results:
Certificates are renewed with each update to the overcloud.

Expected results:
Certificates renewed only when near expiration.

Additional info:
*** messages logs available in case comment-16.

Case from a customer in a secure environment. Part of the discussion from the stack team (case comment-20):
<dhill> almondjoy, it's more EnableInternalTLS that will control this
<almondjoy> dhill, well we don't want to disable it, just stop it from always renewing
<dhill> almondjoy, haproxy-internal-tls-certmonger.yaml - certmonger will manage the internal certs in haproxy.
<almondjoy> dhill, yeah that's the file I'm talking about
<dhill> almondjoy, resource_registry:
<dhill>   OS::TripleO::Services::HAProxyInternalTLS: ../../deployment/haproxy/haproxy-internal-tls-certmonger.yaml
<dhill>   OS::TripleO::Services::IpaClient: ../../deployment/ipa/ipaservices-baremetal-ansible.yaml
<dhill>   OS::TripleO::Services::TLSProxyBase: ../../deployment/apache/apache-baremetal-puppet.yaml
<dhill> almondjoy, interesting
<dhill> if you look at haproxy-internal-tls-certmonger.yaml
<almondjoy> oh wait
<almondjoy> which one
<dhill> almondjoy, looks like it'd be running everytime
<almondjoy> there are multiple with that name lol
<dhill> OS::TripleO::Services::HAProxyInternalTLS: ../../deployment/haproxy/haproxy-internal-tls-certmonger.yaml
<dhill> this one
<almondjoy> how can you tell it'll be running every time?
* almondjoy is looking at the jinja template
<dhill> almondjoy, well I see nothing that says "don't run everytime"
<dhill> almondjoy, do we still support both methods to create certs ?  it's just with idm right ?
<dhill> almondjoy,             - include_role:
<dhill>                 name: linux-system-roles.certificate
<dhill>               vars:
<dhill>                 certificate_requests:
<dhill>                   repeat:
<dhill>                     template:
<dhill>                       name: httpd-NETWORK
<dhill>                       dns: "{% raw %}{{ fqdn_NETWORK }}{% endraw %}"
<dhill>                       principal: "{% raw %}HTTP/{{ fqdn_NETWORK }}@{{ idm_realm }}{% endraw %}"
<dhill>                       run_after: |
<dhill>                         cp /etc/pki/tls/certs/httpd-NETWORK.crt /etc/pki/tls/certs/httpd/httpd-NETWORK.crt
<dhill>                         cp /etc/pki/tls/private/httpd-NETWORK.key /etc/pki/tls/private/httpd/httpd-NETWORK.key
<dhill>                         pkill -USR1 httpd
<dhill>                       key_size:
<dhill>                         if:
<almondjoy> dhill, according to the doc we do
<dhill>                           - key_size_override_set
<dhill>                           - {get_param: ApacheCertificateKeySize}
<dhill>                           - {get_param: CertificateKeySize}
<dhill>                       ca: ipa
<dhill>                     for_each:
<dhill>                       NETWORK: {get_attr: [ApacheNetworks, value]}
<dhill> ~
<dhill> almondjoy, I think we run that everytime
<dhill> almondjoy, so we always have working certificates

Comment 4 Tommy Doucet 2021-12-15 18:56:30 UTC
Hello, any update on this bug? Wanted to give customer on my case an update, if any.

Thank you!
Tommy Doucet
Red Hat OpenStack Support

Comment 5 Grzegorz Grasza 2021-12-20 12:50:00 UTC
Update: this is already fixed and on track for OSP 17, because of a migration to a different way of triggering certmonger certificate requests (Ansible, instead of Puppet).

There was also some work done to fix it for previous releases, part of this work was merged but some of the changes were abandoned upstream.

I re-added the 2 remaining patches for review upstream and I still have to test to make sure we have all of the dependencies downstream, so that we end up with a working solution.


Note You need to log in before you can comment on or make changes to this bug.