Bug 2155961

Summary: httpd-init fails to create localhost.crt, localhost.key due to "sscg" default now creates a /dhparams.pem and is not idempotent if the file /dhparams.pem already exists.
Product: Red Hat Enterprise Linux 8 Reporter: Dasharath Masirkar <dmasirka>
Component: httpdAssignee: Luboš Uhliarik <luhliari>
Status: CLOSED ERRATA QA Contact: icesalov
Severity: medium Docs Contact:
Priority: high    
Version: 8.7CC: jorton, luhliari, peter.vreman
Target Milestone: rcKeywords: AutoVerified, Regression, Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: httpd-2.4-8080020230126164620.fd72936b Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2165967 (view as bug list) Environment:
Last Closed: 2023-05-16 08:28:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2165967    

Description Dasharath Masirkar 2022-12-23 03:13:20 UTC
Description of problem:
 In RHEL8.7, the httpd-init fails to create localhost.crt, localhost.key due to "sscg" default now creates a /dhparams.pem and is not idempotent if the file /dhparams.pem already exists.

Version-Release number of selected component (if applicable):

- Red Hat Enterprise Linux release 8.7 (Ootpa)
- httpd-filesystem-2.4.37-51.module+el8.7.0+16050+02173b8e.noarch
- redhat-logos-httpd-84.5-1.el8.noarch
- mod_ssl-2.4.37-51.module+el8.7.0+16050+02173b8e.x86_64
- httpd-2.4.37-51.module+el8.7.0+16050+02173b8e.x86_64
- httpd-tools-2.4.37-51.module+el8.7.0+16050+02173b8e.x86_64

How reproducible:
Always

Steps to Reproduce:

[root@rhel87 ~]# rm -fv /etc/pki/tls/certs/localhost.crt /etc/pki/tls/private/localhost.key
removed '/etc/pki/tls/certs/localhost.crt'
removed '/etc/pki/tls/private/localhost.key'

[root@rhel87 ~]# systemctl start httpd-init
Job for httpd-init.service failed because the control process exited with error code.
See "systemctl status httpd-init.service" and "journalctl -xe" for details.


Actual results:

 - The httpd-init failed to start and do not create a required certificate and key.

[root@rhel87 ~]# systemctl status -l  httpd-init
● httpd-init.service - One-time temporary TLS key generation for httpd.service
   Loaded: loaded (/usr/lib/systemd/system/httpd-init.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2022-12-20 12:33:47 IST; 11s ago
 	Docs: man:httpd-init.service(8)
  Process: 23092 ExecStart=/usr/libexec/httpd-ssl-gencerts (code=exited, status=17)
 Main PID: 23092 (code=exited, status=17)

Dec 20 12:33:47 rhel87 systemd[1]: Starting One-time temporary TLS key generation for httpd.service...
Dec 20 12:33:47 rhel87 httpd-ssl-gencerts[23096]: Could not write to /dhparams.pem. Check directory permissions.
Dec 20 12:33:47 rhel87 systemd[1]: httpd-init.service: Main process exited, code=exited, status=17/n/a
Dec 20 12:33:47 rhel87 systemd[1]: httpd-init.service: Failed with result 'exit-code'.
Dec 20 12:33:47 rhel87 systemd[1]: Failed to start One-time temporary TLS key generation for httpd.service.

Expected results:

 - The httpd-init should start and create a required certificate and key.

# systemctl status  httpd-init
● httpd-init.service - One-time temporary TLS key generation for httpd.service
   Loaded: loaded (/usr/lib/systemd/system/httpd-init.service; static; vendor preset: disabled)
   Active: inactive (dead) since Thu 2022-12-22 04:41:59 EST; 17h ago
     Docs: man:httpd-init.service(8)
  Process: 3291255 ExecStart=/usr/libexec/httpd-ssl-gencerts (code=exited, status=0/SUCCESS)
 Main PID: 3291255 (code=exited, status=0/SUCCESS)

Dec 22 04:41:59 rhel87 systemd[1]: Starting One-time temporary TLS key generation for httpd.service...
Dec 22 04:41:59 rhel87 systemd[1]: httpd-init.service: Succeeded.
Dec 22 04:41:59 rhel87 systemd[1]: Started One-time temporary TLS key generation for httpd.service.


Additional info:
 - This issue does not exist in RHEL 8.6 only exist in RHEL 8.7

Workaround:
 - Reinstall the mod_ssl package, e.g. # dnf reinstall mod_ssl -y

Comment 3 Dasharath Masirkar 2022-12-23 08:24:21 UTC
Workaround:

rm -f /etc/pki/tls/private/localhost.key

rm -f /etc/pki/tls/certs/localhost.crt

rm -f /dhparams.pem

systemctl restart httpd

# systemctl status  httpd
● httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled; vendor preset: disabled)
   Active: reloading (reload) since Fri 2022-12-23 13:50:05 IST; 8s ago
     Docs: man:httpd.service(8)
 Main PID: 2807 (httpd)
   Status: "Reading configuration..."
    Tasks: 1 (limit: 49428)
   Memory: 8.2M
   CGroup: /system.slice/httpd.service
           └─2807 /usr/sbin/httpd -DFOREGROUND

Dec 23 13:49:54 rhel87 systemd[1]: Starting The Apache HTTP Server...
Dec 23 13:50:05 rhel87 httpd[2807]: AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using fe80::5054:ff:fef5:e9f3. Set the 'ServerName' directive globally to suppress this>
Dec 23 13:50:05 rhel87 systemd[1]: Started The Apache HTTP Server.

]# openssl s_client 127.0.0.1:443
CONNECTED(00000003)
Can't use SSL_get_servername
depth=1 C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87
verify error:num=19:self signed certificate in certificate chain
verify return:1
depth=1 C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87
verify return:1
depth=0 C = US, O = Unspecified, CN = rhel87, emailAddress = root@rhel87
verify return:1
---
Certificate chain
 0 s:C = US, O = Unspecified, CN = rhel87, emailAddress = root@rhel87
   i:C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87
 1 s:C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87
   i:C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIEqTCCApGgAwIBAgIIYCri+rwc7kgwDQYJKoZIhvcNAQELBQAwcTELMAkGA1UE
BhMCVVMxFDASBgNVBAoMC1Vuc3BlY2lmaWVkMR8wHQYDVQQLDBZjYS0xNDI3NjIz
MTkwNDc5MTI4MTM3MQ8wDQYDVQQDDAZyaGVsODcxGjAYBgkqhkiG9w0BCQEWC3Jv
b3RAcmhlbDg3MB4XDTIyMTIyMzA3NDQzNFoXDTIzMTIyMzA3NDQzNFowUDELMAkG
A1UEBhMCVVMxFDASBgNVBAoMC1Vuc3BlY2lmaWVkMQ8wDQYDVQQDDAZyaGVsODcx
GjAYBgkqhkiG9w0BCQEWC3Jvb3RAcmhlbDg3MIIBIjANBgkqhkiG9w0BAQEFAAOC
AQ8AMIIBCgKCAQEA4U4AXSxaV32WUN+0rQeD1b6AY+ovNWaOMHOOBrchGy1IErdh
lcLPris8HICR+bAjr6n6cHLpxmvy6PcYw/7y/3LeK0kemTdzy8ag5F8lLCM+AFLV
RK0wHfOK28sUVj4BZuVu8a7kRm5sVZFovuP+q/Zj9mvoa+huTifFzp8fAJqO3g4Y
AmUaF1SOlB2WO1I9H09CFLfJy3SWeuuIZxOH1VwGnBGHuGMsFT+cRqleINjn71BZ
HjAt/TvARmMNJRojdNCr7aNrkr5a1nYbRZL0v2VKHYDBLhk4hEYfKzMNJXigrKpF
U8CMMfgndAzHgz0NV/Wg7rADzfXyDdFnWQYAIQIDAQABo2YwZDAOBgNVHQ8BAf8E
BAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEwCQYDVR0TBAIwADARBgNVHREECjAI
ggZyaGVsODcwHwYDVR0jBBgwFoAUadCiKRVS/WHiEUaOw9nQD+54ioEwDQYJKoZI
hvcNAQELBQADggIBAAyZPgNvT/NoEBufU9mQGqovUcP2OBmJGoezosBOIkKg8W0O
31e8bD2auNHIXhTsLquxGU5MfqEZjiz1F0pd50MqIv/Y5zxjk+7KiLEeNuKyaT5H
C1xJGy+RCx2l0aZywuAVRcZNGrkWwkVkEMPRpPDuAgEpCkDUurEYwupDTXVPG4bS
yBWTgB1wK546MVKH4ydys/sH8A2TVWv4KCXxuBZMaICZRZbrOV+tf5amUu89E/JL
MhDm5QmfhyKdTkE5DIsDypM4Wjzdv2XGoaLXf4/7jsfkDTQHMGGMnKGMPzdbPIc9
0HbTQX59DokxUbYLtuJ7RQrj8jQUMJ4z+Bb0E1qwlaRs0hNNQjDPiGdM/P/f2kVX
1QqnixBVa2AL1Fr4DWWXKFBqE0pKSUbuOgEy9lie89gkr2RqWBMaG/a1WurghGif
SKjkqO/i4g07ll06ekqoGh3Y/v+bhfEJYpSqKbBziCq/GwBrtNuY3ou5yYVOptey
1wJPRZ+HaAis+4PBKLjUkPKLVDjdD63KSlqNoIBZ9tIzZ9D7LHI4zioo013IW6Vc
6cjiNZAksZLdGXvXYojNuGi25m/wyUjHBS1ZUxW6VcSKi3lLvqTYuaoyECxqiipy
Tvr7+kOKIpHCDBtfLIGosKAfnnP4i0s5E3P7NhO922ScGBggW5SqMZje2ZAB
-----END CERTIFICATE-----
subject=C = US, O = Unspecified, CN = rhel87, emailAddress = root@rhel87

issuer=C = US, O = Unspecified, OU = ca-1427623190479128137, CN = rhel87, emailAddress = root@rhel87

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3247 bytes and written 369 bytes
Verification error: self signed certificate in certificate chain
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 19 (self signed certificate in certificate chain)

Comment 4 Peter Vreman 2022-12-23 11:58:19 UTC
Copied from the attached case a a quick fix to call sscg with a custom dhparams file to have it craeted in a temporary directory that is also cleaned at the end

~~~
--- /usr/libexec/httpd-ssl-gencerts.221220-1    2022-07-29 03:44:13.000000000 +0000
+++ /usr/libexec/httpd-ssl-gencerts     2022-12-20 16:21:20.074712388 +0000
@@ -29,11 +29,18 @@
     exit 0
 fi

+dhparamstmpdir=$(mktemp -d)
+
 sscg -q                                                             \
+     --dhparams-file       $dhparamstmpdir/dhparams.pem             \
      --cert-file           /etc/pki/tls/certs/localhost.crt         \
      --cert-key-file       /etc/pki/tls/private/localhost.key       \
      --ca-file             /etc/pki/tls/certs/localhost.crt         \
      --lifetime            365                                      \
      --hostname            $FQDN                                    \
      --email               root@$FQDN
+res=$?
+
+rm -rf $dhparamstmpdir

+exit $res
~~~

Comment 6 Joe Orton 2023-01-04 10:14:11 UTC
Thanks for the report. I think we need to work around this in httpd by adjusting httpd-ssl-gencerts something like as suggested in comment 4, so I'm adjusting the component.

Comment 19 errata-xmlrpc 2023-05-16 08:28:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (httpd:2.4 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2789