Bug 1577570

Summary: Certmonger SCEP renewal should not use old challenges
Product: Red Hat Enterprise Linux 8 Reporter: Zhenyu Wu <Adam_5Wu>
Component: certmongerAssignee: Rob Crittenden <rcritten>
Status: CLOSED ERRATA QA Contact: ipa-qe <ipa-qe>
Severity: medium Docs Contact: David Voženílek <dvozenil>
Priority: medium    
Version: ---CC: Adam_5Wu, bernard.rodriguez, dvozenil, fedora, gfialova, j.florkowski, j.mccanta, nicholaus.daverin, pcech, pvoborni, rcritten, ssidhaye, sumenon, tscherf
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: certmonger-0.79.13-4.el8 Doc Type: Bug Fix
Doc Text:
.Certmonger can now automatically renew SCEP certificates with AD when `challengePassword` is required for enrollment Previously, requests for renewal of SCEP certificates sent by `certmonger` to an Active Directory (AD) Network Device Enrollment Service (NDES) server included the `challengePassword` used to originally obtain the certificate. However, AD treats `challengePassword` as a one-time password (OTP). As a consequence, the renewal request was rejected. This update adds the `challenge_password_otp` option to `certmonger`. When enabled, this option prevents `certmonger` from sending the OTP with the SCEP renewal request. The administrator must also add the `DisableRenewalSubjectNameMatch` entry with a value of `1` to the *HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\MSCEP* subkey in the AD registry. With this modification, AD no longer requires the signer certificate and requested certificate subject names to match. As a result, the SCEP certificate renewal is successful. To configure `certmonger` and the AD server for SCEP renewals to work: . Open `regedit` on the AD server. . In the *HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\MSCEP* subkey, add a new 32-bit REG_DWORD entry `DisableRenewalSubjectNameMatch` and set its value to `1`. . On the server where `certmonger` is running, open the `/etc/certmonger/certmonger.conf` file and add the following section: + ---- [scep] challenge_password_otp = yes ---- . Restart certmonger: + ---- # systemctl restart certmonger ----
Story Points: ---
Clone Of:
: 1990926 (view as bug list) Environment:
Last Closed: 2022-05-10 13:38:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1990926, 2150030    

Description Zhenyu Wu 2018-05-13 04:11:30 UTC
Description of problem:
During renewal of SCEP certificates, certmonger uses old challenges for the CSR. However for security reasons, challenges may be one-time-use only. Reusing old challenges can cause CSR to be rejected, breaking the auto-renew feature.

Version-Release number of selected component (if applicable):


How reproducible:
Enroll an SCEP certificate with Microsoft NDES service, which was set to use one-time challenge (the default setting), then try to renew.


Steps to Reproduce:
1. Configure an MS NDES service as external CA
   getcert add-scep-ca -c CA-NDES -u 'http://CA-NDES.local/CertSrv/mscep'
2. Obtain a one-time use challenge from MS NDES Web UI
3. Enroll a certificate
   getcert request -c CA-NDES -I Test -f /tmp/test-public.pem -k /tmp/test-private.pem -N "CN=Test Cert" -D Test.local -F /tmp/ca.pem -w -v -L <one-time-challenge>
4. Renew the certificate
   getcert resubmit -i Test -v -w

Actual results:
- Renewal failed, the following error message is logged:
  Transaction either is not permitted or is not supported by server.
- On the NDES side, the following error message is logged:
  The password in the certificate request cannot be verified. It may have been used already. Obtain a new password to submit with this request.

Expected results:
- Successful renewal

Additional info:
The problem can be worked around by the following steps:
1. Stop the certmonger service
2. Manually edit the certificate's tracking store:
vi /var/lib/certmonger/requests/2018xxxxxxxxxx
3. Remove the line like the following:
template-challenge-password=<one-time-challenge>
4. Restart the certmonger service
5. Try renewal request again, it is now successful

Comment 2 Rob Crittenden 2018-05-17 15:50:34 UTC
I'm not sure how I would know whether the password is for one-time use or not.

Re-sending the challenge password is acceptable per the SCEP specification section 
2.3: 

   A client that is performing certificate renewal as per Section 2.4
   SHOULD omit the challengePassword but MAY send the originally
   distributed password in the challengePassword attribute.  The SCEP CA
   MAY use the challengePassword in addition to the previously issued
   certificate that signs the request to authenticate the request.  The
   SCEP CA MUST NOT attempt to authenticate a client based on a self-
   signed certificate unless it has been verified through out-of-band
   means such as a certificate fingerprint.

I see your point though with a one-time password. Dropping this completely would be quite a change in behavior. Do you think adding an option to specify that the challenge password is a one-time password be sufficient?

Comment 3 Zhenyu Wu 2018-05-17 16:05:41 UTC
Yes, I think it would be great to allow specifying at initial enrollment whether challenge password should be used for future renewal. The default behavior can remain the same, but at least user could have an accessible option -- the workaround I discovered is way too hacky.

And 99% of tutorial/guideline floating on the Internet simply advice disabling one-time-use challenge, which leaves security holes, albeit on the Intranet -- when there are too many of them, small breaches can quickly escalate into major crisis.

Comment 6 j.mccanta 2019-05-12 18:37:44 UTC
I concur that having an option to specify that renewals should not use the challenge passphrase would be very helpful.  Using the SinglePassword option with windows NDES has a lot of potential for abuse.  I was not any to get Zhunyu's workaround (remove the template_challenge_passphrase after stopping certmonger) to work.  The error on the Windows 2016 server is:

The Network Device Enrollment Service cannot locate a required password in the certificate request. Either a password must be present in the certificate request or the certificate request should be signed with a valid signing certificate. The signing certificate must chain up to a trusted root in the Enterprise store. The signing certificate and the certificate request must have the same subject name or subject alternate name.

I have turned on the DisableRenewalSubjectNameMatch. Is certmonger NOT signing the renew request?

Comment 7 Rob Crittenden 2019-05-20 20:06:45 UTC
If there is a one-time password it is going to fail on renewal regardless of whether you drop it from the request or leave it. One would need to set a new password for the renewal.

If you tried his workaround and dropped the challenge password then that would explain the error message.

There isn't really a good answer to this other than to replace the password just prior to renewal. I'm not sure that adding an option to drop it entirely would be all that beneficial.

Comment 8 Nick Daverin 2020-02-03 23:19:47 UTC
From what I can see, the issue lies with certmonger not implementing the correct messageType. The PKCS signed renewal request is messageType 17, while certmonger seems to only want to send messageType 19 (authenticated with a password).

Comment 9 Florence Blanc-Renaud 2020-02-27 09:43:22 UTC
Thank you taking your time and submitting this request for Red Hat Enterprise Linux 7. Unfortunately, this bug cannot be kept even as a stretch goal and was postponed to RHEL8.

Comment 13 Rob Crittenden 2021-07-01 19:53:29 UTC
To clarify, for renewals certmonger isn't using an incorrect authentication mechanism, it just isn't using the existing certificate method (e.g. not signing the request with the existing certificate).

There are three ways to authenticate a request: using a challenge password (PKCSReq), signing with an existing certificate (RenewalReq) or passing credentials via an alternate CA. The second method is SHOULD if the CA supports it (e.g. has Renewal in its capabilities) and the third method is MAY.

From the RFC (8894):

   Note that although the above text describes several different types
   of operations, for historical reasons, most implementations always
   apply the first one, even if an existing certificate already exists.
   For this reason, support for the first case is mandatory while
   support for the latter ones are optional (see Section 2.9).

I'm looking into what it will take to use signing instead of a ChallengePassword for renewals, and setting the right request type in this case.

Comment 14 Rob Crittenden 2021-07-08 20:02:44 UTC
I was incorrect about not signing subsequent requests with the issued certificate. If there is a previously issued certificate it does use it to sign the request.

In fact, I'm not able to duplicate the original report on Fedora with certmonger-0.79.13-1.fc33.

When I run resubmit it is successful but I don't get a new certificate. I don't know if that is because it is outside the renewal period or something else.

It's very possible our AD servers are setup differently. Mine is largely vanilla other than a new cert template that has a one-year validity period.

It's unclear whether AD supports RenewalReq. It advertises Renewal as a capability so it will accept a request signed by the current cert but I don't know which spec they implemented from originally, nourse or gutmann. NDES logged an unknown message type when I tried using RenewalReq.

Comment 15 Rob Crittenden 2021-07-09 16:57:53 UTC
On further inspection AD is returning the SCEP error badRequest (2). Since there is already a certificate certmonger puts it back into MONITORING (so a no-op). The event viewer wasn't immediately updated which is why I missed it before. It is failing, like the reporter, with the error "The password in the certificate request cannot be verified. It may have been used already. Obtain a new password to submit with this request.

Removing the challenge password results in a different error. It looks like that the request is signed but apparently not to the liking of AD.

"The Network Device Enrollment Service cannot locate a required password in the certificate request. Either a password must be present in the certificate request or the certificate request should be signed with a valid signing certificate. The signing certificate must chain up to a trusted root in the Enterprise store. The signing certificate and the certificate request must have the same subject name or subject alternate name."

Comment 16 Rob Crittenden 2021-07-09 21:47:39 UTC
I worked out away to do renewals in AD. I can't say it's the best way, but it's based off 
https://social.technet.microsoft.com/Forums/ie/en-US/a2a8fb0a-46be-4ec1-9ead-04483f4ae618/ndes-automated-renewal-of-existing-certificate-via-scep-not-working?forum=winserversecurity

The procedure:

Note that you only have one chance for one of these settings. If you miss something you can delete and re-copy the template.

- Duplicate the IPSec (Offline request) template in the Certificates Templates Console. I named mine IPSecImmediateRenew.
- Open the new template and set:

General -> Template name to IPSecImmediateRenew (or whatever you want)

Compatibility -> Certificate recipient -> Windows 7 / Server 2008 R2 (this unlocks the next option)

Subject name -> Check off "Supply in request" and "Use subject information from existing certificates for autoenrollment renewal requests"

For testing purposes, under General I set the Validity period 1 hour and 0 hours for renewal period.

Now launch reg edit and go to  Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\MSCEP

Set EncryptionTemplate, GeneralPurposeTemplate and SignatureTemplate to the new template name

In PowerShell run: iisreset

Now on your RHEL system run: getcert request ... -L <OTP>

Get the serial number:

openssl x509 -text -in /path/to/your/cert.pem | grep -i "serial" -A 1

Once the certificate is issued, stop certmonger, find the request, and remove the challenge_password, per c#0. Restart certmonger.

You'll need to wait a bit to get a new certificate. I think I waited about 10 minutes.

Run getcert resubmit <options>

openssl x509 -text -in /path/to/your/cert.pem | grep -i "serial" -A 1

The serial number should have changed.

I'll work on adding a new configuration option to wipe out the challenge password/file after a certificate is issued the first time.

Note. On my AD server I did not set DisableRenewalSubjectNameMatch to 1.

I'm not sure why a subject name mismatch is being detected. I used -N=`hostname` (so CN=hostname) for the subject and that matches in both the CSR and in the certificate issued by AD.

Comment 17 bernard.rodriguez 2021-07-12 12:56:16 UTC
Hi @Rob Crittenden ,
I'm facing the same issue if I don't add the DisableRenewalSubjectNameMatch 1 . 
but the issue seems more related to the MSSCEP implementation  not matching correctly as I have the same behaviour whth my Cisco swicthes 

I'm using the workaround at the momment and would appreciate a neater solution . Let me know if you need me to test the new configuration option .


Bernard

Comment 18 Rob Crittenden 2021-07-12 20:27:35 UTC
I had no luck with DisableRenewalSubjectNameMatch (e.g. it didn't seem to affect anything). Are you saying that disabling that is sufficient for renewal and I didn't need to go through the complicated procedure of creating a new template? That would be a much simpler solution.

Comment 19 bernard.rodriguez 2021-07-13 11:21:50 UTC
(In reply to Rob Crittenden from comment #18)
> I had no luck with DisableRenewalSubjectNameMatch (e.g. it didn't seem to
> affect anything). Are you saying that disabling that is sufficient for
> renewal and I didn't need to go through the complicated procedure of
> creating a new template? That would be a much simpler solution.

The only change we asked on the MS NDES server is DisableRenewalSubjectNameMatch=1 that did the trick.
but the pki team already did some changes on the template before that to solve a similar issue with Cisco swiches scep client .  

I'll ask my colleague to go trough the template . I have no access on our pki infra myself. 

On the client :

stop certmonger service 
remove line --> template_challenge_password=<OTC>  from request file 
start cermonger service   
getcert resubmit

Comment 20 Rob Crittenden 2021-07-14 01:17:04 UTC
Cheers, the registry setting is sufficient. I re-installed my AD server and setup a new CS/NDES server and set the registry value and with my patch renewals happen just fine.

To add the value:

- fire up regedit
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\MSCEP
- add a new 32-bit DWORD value named DisableRenewalSubjectNameMatch and set the value to 1

I didn't restart IIS, but you can if you want (in PowerShell run iisreset)

On the certmonger side (with patch) add this to /etc/certmonger/certmonger.conf
[scep]
challenge_password_otp = yes

Restart certmonger

getcert add-scep-ca -c scep -u http://root-ad.vm/certsrv/mscep -v -v -v
getcert request -f /etc/pki/tls/certs/test.pem -k /etc/pki/tls/private/test.key -N `hostname` -c scep -v -w -g 2048 -L <OTP from AD CS>
openssl x509 -text -in /etc/pki/tls/certs/test.pem |grep -i serial -A 1

Now wait. I don't know how long but AD won't immediately re-issue a certificate. I ended up waiting a few hours (because dinner).

getcert resubmit -f /etc/pki/tls/certs/test.pem -v -w
openssl x509 -text -in /etc/pki/tls/certs/test.pem |grep -i serial -A 1

The second serial number should differ from the first.

I'll get a PR upstream tomorrow.

Comment 21 Rob Crittenden 2021-07-14 21:53:06 UTC
Upstream PR https://pagure.io/certmonger/pull-request/214

Comment 22 Rob Crittenden 2021-08-06 15:16:56 UTC
master: b38981c6e140ada6dd34bc817c508e8dd9714494

Comment 27 Sudhir Menon 2021-12-21 10:22:13 UTC
Marking the bug as verified  using certmonger-0.79.13-5.el8.x86_64 since there is no error 
seen as mentioned in the original bz comment i.e 'Transaction either is not permitted or is not supported by server'
Also after doing resubmit the serial number differs from the first.  

[root@master requests]# getcert list-cas
CA 'scep':
	is-default: no
	ca-type: EXTERNAL
	helper-location: /usr/libexec/certmonger/scep-submit -u http://win01/CertSrv/mscep           
	SCEP CA certificate thumbprint (MD5): 6CF04DFB 2529F13C 4829A890 E0F9E67D
	SCEP CA certificate thumbprint (SHA1): 5545DC5B 45525202 AC6E7D91 4A450456 6FA240BD

[root@master requests]# getcert request -c scep -I Test -f /tmp/test-public.pem -k /tmp/test-private.pem -N "CN=Test Cert" -D scep.test -F /tmp/ca.pem -w -v -L 6A694CA86FECEECD 
New signing request "Test" added.
State NEWLY_ADDED_READING_CERT, stuck: no.
State MONITORING, stuck: no.

[root@master requests]# getcert list
Number of certificates and requests being tracked: 1.
Request ID 'Test':
	status: MONITORING
	stuck: no
	key pair storage: type=FILE,location='/tmp/test-private.pem'
	certificate: type=FILE,location='/tmp/test-public.pem'
	CA: scep
	issuer: CN=win01,DC=scep,DC=test
	subject: CN=Test Cert
	issued: 2021-12-21 04:37:06 EST
	expires: 2023-12-21 04:37:06 EST
	dns: scep.test
	key usage: digitalSignature,keyEncipherment
	eku: iso.org.dod.internet.security.mechanisms.8.2.2
	certificate template/profile: IPSECIntermediateOffline
	profile: IPSECIntermediateOffline
	pre-save command: 
	post-save command: 
	track: yes
	auto-renew: yes

/var/lib/certmonger/requests/20211221101044
key_storage_location=/tmp/test-private.pem
cert_storage_location=/tmp/test-public.pem
cert_issuer=CN=win01,DC=scep,DC=test
cert_serial=72000000099B64352843005288000000000009
cert_subject=CN=Test Cert
cert_hostname=scep.test
cert_profile=IPSECIntermediateOffline
template_challenge_password=6A694CA86FECEECD
state=MONITORING
autorenew=1
monitor=1
ca_name=scep
submitted=19700101000000
root_cert_files=/tmp/ca.pem

[root@master requests]# openssl x509 -text -in /tmp/test-public.pem |grep -i serial -A 1
        Serial Number:
            72:00:00:00:09:9b:64:35:28:43:00:52:88:00:00:00:00:00:09

[root@master requests]# getcert resubmit -i Test -v -w
Resubmitting "Test" to "scep".
State GENERATING_CSR, stuck: no.
State READING_CERT, stuck: no.
State MONITORING, stuck: no.

[root@master requests]# openssl x509 -text -in /tmp/test-public.pem |grep -i serial -A 1
        Serial Number:
            72:00:00:00:0a:c8:09:49:6b:f1:f9:fe:44:00:00:00:00:00:0a

Comment 32 errata-xmlrpc 2022-05-10 13:38:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (certmonger bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1789

Comment 33 shaozx 2022-11-22 02:20:00 UTC
I found the same issue happening on certmonger 0.78.4 (certmonger-0.78.4-17.el7_9.x86_64) on CentOS 7.9 and google leads to this bug report. I am wondering whether you (@rcritten) could help to backport this fix to CentOS 7 / EL 7?

Comment 34 Rob Crittenden 2022-11-22 16:43:25 UTC
No plans to backport.