Bug 2011983

Description of problem:
From https://access.redhat.com/support/cases/03039688 :
When running an update to the overcloud for any reason we noticed that all the RHOSP related certificates being tracked by certmonger are being removed. We have a rather large environment and recently we've been adding around a dozen nodes a time and we find it unnecessary to be renewing those certificates multiple times a day or everyday. Is there a way to prevent this from happening?

Version-Release number of selected component (if applicable):
-

Steps to Reproduce:
- We use novajoin to register the undercloud with IPA.
- Add the ipa_otp parameter to undercloud.conf with the password provided from novajoin before the undercloud install.
- Include the following templates
    -  /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml
    - /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml
    - /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml
    - homedir/cloudname.yaml

Actual results:
Certificates are renewed with each update to the overcloud.

Expected results:
Certificates renewed only when near expiration.

Additional info:
*** messages logs available in case comment-16.

Case from a customer in a secure environment. Part of the discussion from the stack team (case comment-20):
<dhill> almondjoy, it's more EnableInternalTLS that will control this
<almondjoy> dhill, well we don't want to disable it, just stop it from always renewing
<dhill> almondjoy, haproxy-internal-tls-certmonger.yaml - certmonger will manage the internal certs in haproxy.
<almondjoy> dhill, yeah that's the file I'm talking about
<dhill> almondjoy, resource_registry:
<dhill>   OS::TripleO::Services::HAProxyInternalTLS: ../../deployment/haproxy/haproxy-internal-tls-certmonger.yaml
<dhill>   OS::TripleO::Services::IpaClient: ../../deployment/ipa/ipaservices-baremetal-ansible.yaml
<dhill>   OS::TripleO::Services::TLSProxyBase: ../../deployment/apache/apache-baremetal-puppet.yaml
<dhill> almondjoy, interesting
<dhill> if you look at haproxy-internal-tls-certmonger.yaml
<almondjoy> oh wait
<almondjoy> which one
<dhill> almondjoy, looks like it'd be running everytime
<almondjoy> there are multiple with that name lol
<dhill> OS::TripleO::Services::HAProxyInternalTLS: ../../deployment/haproxy/haproxy-internal-tls-certmonger.yaml
<dhill> this one
<almondjoy> how can you tell it'll be running every time?
* almondjoy is looking at the jinja template
<dhill> almondjoy, well I see nothing that says "don't run everytime"
<dhill> almondjoy, do we still support both methods to create certs ?  it's just with idm right ?
<dhill> almondjoy,             - include_role:
<dhill>                 name: linux-system-roles.certificate
<dhill>               vars:
<dhill>                 certificate_requests:
<dhill>                   repeat:
<dhill>                     template:
<dhill>                       name: httpd-NETWORK
<dhill>                       dns: "{% raw %}{{ fqdn_NETWORK }}{% endraw %}"
<dhill>                       principal: "{% raw %}HTTP/{{ fqdn_NETWORK }}@{{ idm_realm }}{% endraw %}"
<dhill>                       run_after: |
<dhill>                         cp /etc/pki/tls/certs/httpd-NETWORK.crt /etc/pki/tls/certs/httpd/httpd-NETWORK.crt
<dhill>                         cp /etc/pki/tls/private/httpd-NETWORK.key /etc/pki/tls/private/httpd/httpd-NETWORK.key
<dhill>                         pkill -USR1 httpd
<dhill>                       key_size:
<dhill>                         if:
<almondjoy> dhill, according to the doc we do
<dhill>                           - key_size_override_set
<dhill>                           - {get_param: ApacheCertificateKeySize}
<dhill>                           - {get_param: CertificateKeySize}
<dhill>                       ca: ipa
<dhill>                     for_each:
<dhill>                       NETWORK: {get_attr: [ApacheNetworks, value]}
<dhill> ~
<dhill> almondjoy, I think we run that everytime
<dhill> almondjoy, so we always have working certificates