Description of problem: when deploying a DCN site container-puppet-haproxy fails to start on DistributedComputeHCIScaleOut deployed with TLS-E. There are some CRL certs which are not available and puppet generated haproxy config refers to them. The exact log: 2022-05-03 01:09:02.340647 | | WARNING | ERROR: Can't run container container-puppet-haproxy stderr: + /usr/bin/puppet apply --summarize --detailed-exitcodes --color=false --modulepath=/etc/puppet/modules:/usr/share/openstack-puppet/modules --tags '"file,file_line,concat,augeas,cron,haproxy_config"' /etc/config.pp + logger -s -t puppet-user <13>May 3 01:08:46 puppet-user: Error: Facter: error while resolving custom fact "haproxy_version": undefined method `strip' for nil:NilClass <13>May 3 01:08:52 puppet-user: Warning: /etc/puppet/hiera.yaml: Use of 'hiera.yaml' version 3 is deprecated. It should be converted to version 5 <13>May 3 01:08:52 puppet-user: (file: /etc/puppet/hiera.yaml) <13>May 3 01:08:52 puppet-user: Warning: Undefined variable '::deploy_config_name'; <13>May 3 01:08:52 puppet-user: (file & line not available) <13>May 3 01:08:52 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/puppet/7.10/deprecated_language.html <13>May 3 01:08:52 puppet-user: (file & line not available) <13>May 3 01:08:53 puppet-user: Warning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications. <13>May 3 01:08:53 puppet-user: Notice: Compiled catalog for dcn1-computehciscaleout1-0.redhat.local in environment production in 0.88 seconds <13>May 3 01:08:53 puppet-user: Error: Execution of '/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg20220503-12-1npa7ww -c' returned 1: [NOTICE] (79) : haproxy version is 2.4.7-b5e51a5 <13>May 3 01:08:53 puppet-user: [NOTICE] (79) : path to executable is /usr/sbin/haproxy <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:48] : 'server dcn1-computehci1-0.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:49] : 'server dcn1-computehci1-1.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:50] : 'server dcn1-computehci1-2.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg20220503-12-1npa7ww <13>May 3 01:08:53 puppet-user: [ALERT] (79) : Fatal errors found in configuration. <13>May 3 01:08:53 puppet-user: Error: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: change from '{sha256}8afc9a0bcc462f08af54b6ac1cbfc3b8343b1feee00b4ab07d7a8c7b47065f0b' to '{sha256}f8be4e72c4471d14a08e5643725035032211626dc9cd5e7b54767c826cfd64ef' failed: Execution of '/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg20220503-12-1npa7ww -c' returned 1: [NOTICE] (79) : haproxy version is 2.4.7-b5e51a5 <13>May 3 01:08:53 puppet-user: [NOTICE] (79) : path to executable is /usr/sbin/haproxy <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:48] : 'server dcn1-computehci1-0.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:49] : 'server dcn1-computehci1-1.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : parsing [/etc/haproxy/haproxy.cfg20220503-12-1npa7ww:50] : 'server dcn1-computehci1-2.internalapi.redhat.local' : 'crl-file' : unable to load /etc/pki/CA/crl/overcloud-crl.pem. <13>May 3 01:08:53 puppet-user: [ALERT] (79) : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg20220503-12-1npa7ww <13>May 3 01:08:53 puppet-user: [ALERT] (79) : Fatal errors found in configuration. <13>May 3 01:08:53 puppet-user: Notice: /Stage[main]/Haproxy/Haproxy::Instance[haproxy]/Haproxy::Config[haproxy]/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/mode: mode changed '0644' to '0640' <13>May 3 01:08:53 puppet-user: Notice: Applied catalog in 0.19 seconds <13>May 3 01:08:53 puppet-user: Application: <13>May 3 01:08:53 puppet-user: Initial environment: production <13>May 3 01:08:53 puppet-user: Converged environment: production <13>May 3 01:08:53 puppet-user: Run mode: user <13>May 3 01:08:53 puppet-user: Changes: <13>May 3 01:08:53 puppet-user: Total: 1 <13>May 3 01:08:53 puppet-user: Events: <13>May 3 01:08:53 puppet-user: Failure: 1 <13>May 3 01:08:53 puppet-user: Success: 1 <13>May 3 01:08:53 puppet-user: Total: 2 <13>May 3 01:08:53 puppet-user: Resources: <13>May 3 01:08:53 puppet-user: Changed: 1 <13>May 3 01:08:53 puppet-user: Failed: 1 <13>May 3 01:08:53 puppet-user: Out of sync: 1 <13>May 3 01:08:53 puppet-user: Skipped: 11 <13>May 3 01:08:53 puppet-user: Total: 21 <13>May 3 01:08:53 puppet-user: Time: <13>May 3 01:08:53 puppet-user: Concat file: 0.00 <13>May 3 01:08:53 puppet-user: Concat fragment: 0.00 <13>May 3 01:08:53 puppet-user: File: 0.08 <13>May 3 01:08:53 puppet-user: Transaction evaluation: 0.18 <13>May 3 01:08:53 puppet-user: Catalog application: 0.19 <13>May 3 01:08:53 puppet-user: Config retrieval: 0.98 <13>May 3 01:08:53 puppet-user: Last run: 1651540133 <13>May 3 01:08:53 puppet-user: Total: 0.19 <13>May 3 01:08:53 puppet-user: Version: <13>May 3 01:08:53 puppet-user: Config: 1651540132 <13>May 3 01:08:53 puppet-user: Puppet: 7.10.0 + rc=6 + '[' false = false ']' + set +x 2022-05-03 01:09:02.344588 | 52540078-cf9a-b171-f034-000000006653 | FATAL | Create containers managed by Podman for /var/lib/tripleo-config/container-puppet-config/step_1 | dcn1-computehciscaleout1-0 | error={"changed": false, "msg": "Failed containers: container-puppet-haproxy"} The workaround is to set ExtraConfig for the nodes: tripleo::haproxy::crl_file: null Version-Release number of selected component (if applicable): ansible-tripleo-ipsec-11.0.1-0.20210910011424.b5559c8.el9ost.noarch ansible-role-tripleo-modify-image-1.3.1-0.20220216001439.30d23d5.el9ost.noarch ansible-tripleo-ipa-0.2.3-0.20220301190449.6b0ed82.el9ost.noarch puppet-tripleo-14.2.3-0.20220407012437.87240e8.el9ost.noarch python3-tripleo-common-15.4.1-0.20220328184445.0c754c6.el9ost.noarch tripleo-ansible-3.3.1-0.20220407091528.0bc2994.el9ost.noarch openstack-tripleo-validations-14.2.2-0.20220408101530.6614654.el9ost.noarch openstack-tripleo-common-containers-15.4.1-0.20220328184445.0c754c6.el9ost.noarch openstack-tripleo-common-15.4.1-0.20220328184445.0c754c6.el9ost.noarch openstack-tripleo-heat-templates-14.3.1-0.20220404155604.75fd885.el9ost.noarch python3-tripleoclient-16.4.1-0.20220407001042.0021766.el9ost.noarch How reproducible: Always Steps to Reproduce: 1. Deploy DCN toology with TLS-E and ceph storage at the DCN site on DistributedComputeHCIScaleOut nodes. Actual results: container-puppet-haproxy fails to start and Edge haproxy is not configured Expected results: Successful configuration of haproxy at Edge on the DistributedComputeHCIScaleOut nodes. Additional info:
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:6543