RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1839181 - Certmonger-0.78.4-12.el7.x86_64 upgrade broke Microsoft NDES (SCEP) Integration
Summary: Certmonger-0.78.4-12.el7.x86_64 upgrade broke Microsoft NDES (SCEP) Integration
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: certmonger
Version: 7.8
Hardware: All
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Rob Crittenden
QA Contact: ipa-qe
URL:
Whiteboard:
: 1842960 (view as bug list)
Depends On:
Blocks: 1843009
TreeView+ depends on / blocked
 
Reported: 2020-05-22 16:25 UTC by joel
Modified: 2023-12-15 17:59 UTC (History)
11 users (show)

Fixed In Version: certmonger-0.78.4-14.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1843009 (view as bug list)
Environment:
Last Closed: 2020-09-29 20:33:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FREEIPA-10446 0 None None None 2023-10-06 20:16:00 UTC
Red Hat Product Errata RHBA-2020:4009 0 None None None 2020-09-29 20:33:40 UTC

Description joel 2020-05-22 16:25:42 UTC
What problem/issue/behavior are you having trouble with?  What do you expect to see?
We have requested to have an old issue re-opened based upon recent builds of certmonger.  Our integration with Microsoft NDES using SCEP protocol started failing with the recent patch of certmonger, listed below.  The thread of information and what is being changed can be found here: https://pagure.io/certmonger/issue/103#comment-653448

This request is to track this release and get the change implemented. We have a cached version of certmonger-0.78.4-11.el7 in order to avoid this issue, but certmonger-0.78.4-12.el7 broke our integration to NDES.  Please advise on next steps.

Where are you experiencing the behavior? What environment?
Everywhere with certmonger-0.78.4-12.el7 installed.

When does the behavior occur? Frequency? Repeatedly? At certain times?
During certificate creation.

What information can you provide around timeframes and the business impact?
Certs cannot be created.

Comment 2 joel 2020-05-22 16:28:52 UTC
Hello,

One of my customers did a lot of foot work on this issue:

 "So I believe the error is occurring when certmonger tries to request the scep ca certs using the scep-submit command.


We have observed an error initially when we try to add the scep-ca as an available ca for getcert.  In order to add the scep-ca, we first need to retrieve the CA certs using the scep-submit command:


# /usr/libexec/certmonger/scep-submit -vvv -u https://my.ndes.server/certsrv/mscep -C
> GET /certsrv/mscep?operation=GetCACert HTTP/1.1
Host: my.ndes.server
Accept: */*

< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=UTF-8
< Location: https://my.ndes.server/certsrv/mscep/?operation=GetCACert
< Server: Microsoft-IIS/8.5
< Date: Fri, 08 May 2020 17:35:29 GMT
< Content-Length: 179
<
* Ignoring the response-body
* Connection #0 to host my.ndes.server left intact
* Issue another request to this URL: 'https://my.ndes.server/certsrv/mscep/?operation=GetCACert'
* Found bundle for host my.ndes.server: 0x561909f41230
* Re-using existing connection! (#0) with host my.ndes.server
* Connected to my.ndes.server (a.b.c.22) port 443 (#0)
> GET /certsrv/mscep/?operation=GetCACert HTTP/1.1
Host: my.ndes.server
Accept: */*

< HTTP/1.1 200 OK
< Server: Microsoft-IIS/8.5
< Date: Fri, 08 May 2020 17:35:29 GMT
< Content-Length: 0
<
* Connection #0 to host my.ndes.server left intact
GET "https://my.ndes.server/certsrv/mscep?operation=GetCACert"
response_code = 200
content-type = "text/html; charset=UTF-8"
code = 0
code_text = "No error"
results = ""
Internal error: no response to "https://my.ndes.server/certsrv/mscep?operation=GetCACert".
When using the earlier version of certmonger 0.78.4-11, the url has &message=0 appended, and the certs are returned at the end of the command:


< HTTP/1.1 301 Moved Permanently
< Content-Type: text/html; charset=UTF-8
< Location: https://my.ndes.server/certsrv/mscep/?operation=GetCAChain&message=0
< Server: Microsoft-IIS/8.5
< Date: Fri, 08 May 2020 17:38:10 GMT
< Content-Length: 194
<
* Ignoring the response-body
* Connection #1 to host my.ndes.server left intact
* Issue another request to this URL: 'https://my.ndes.server/certsrv/mscep/?operation=GetCAChain&message=0'
* Found bundle for host my.ndes.server: 0x55603380df80
* Re-using existing connection! (#1) with host my.ndes.server
* Connected to my.ndes.server (a.b.c.22) port 443 (#1)
> GET /certsrv/mscep/?operation=GetCAChain&message=0 HTTP/1.1
Host: my.ndes.server
Accept: */*

< HTTP/1.1 200 OK
< Server: Microsoft-IIS/8.5
< Date: Fri, 08 May 2020 17:38:10 GMT
< Content-Length: 0
<
* Connection #1 to host my.ndes.server left intact
GET "https://my.ndes.server/certsrv/mscep?operation=GetCAChain&message=0"
response_code = 200
content-type = "text/html; charset=UTF-8"
code = 0
code_text = "No error"
results = ""
-----BEGIN CERTIFICATE-----
....(certs print here)
I can "fool" this command into working with 0.78.4-12, by manually changing my url:


/usr/libexec/certmonger/scep-submit -vvv -u "https://my.ndes.server/certsrv/mscep?operation=GetCACert&message=0" -C
...
* Connection #0 to host my.ndes.server left intact
* Issue another request to this URL: 'https://my.ndes.server/certsrv/mscep/?operation=GetCACert&message=0?operation=GetCACert'
...
-----BEGIN CERTIFICATE-----
but as you can see, the URL that ends up getting used is not exactly "correct": https://my.ndes.server/certsrv/mscep/?operation=GetCACert&message=0?operation=GetCACert.  


Nevertheless, I now have the CA certs, and we can split them up into the necessary components, and use them to add the scep-ca configuration to certmonger:


getcert add-scep-ca -c my_domain_ndes_ca -u https://my.ndes.server/certsrv/mscep -R /etc/pki/CA/certs/my_domain_ndes_ca_cert-2.crt -r /etc/pki/CA/certs/my_domain_ndes_ca_cert-1.crt -I /etc/pki/CA/certs/my_domain_ndes_ca_ra_chain.crt
However, when I try to request a cert using getcert req, I receive the NEED_SCEP_ENCRYPTION_CERT error:


# getcert request -k /etc/pki/tls/private/$(hostname -f).key -f /etc/pki/tls/certs/$(hostname -f).key -I $(hostname -f)_ssl_cert -g 2048 -c my_domaon_ndes_ca -T "0.MYTemplate" -N "CN=$(hostname -f),OU=MY,O=ORG,C=US" -D $(hostname -f) -A a.b.c.94

# getcert list

Number of certificates and requests being tracked: 1.
Request ID 'hostname-00-07-5_ssl_cert':
        status: NEED_SCEP_ENCRYPTION_CERT
        ca-error: Error reading request, expected PKCS7 data.
        stuck: yes
        key pair storage: type=FILE,location='/etc/pki/tls/private/hostname-00-07-5.key'
        certificate: type=FILE,location='/etc/pki/tls/certs/hostname-00-07-5.key'
        CA: my_domain_ndes_ca
        issuer:
        subject:
        expires: unknown
        pre-save command:
        post-save command:
        track: yes
        auto-renew: yes
And based on what the man page for scep-submit, this error occurs when "The service is waiting until it can retrieve a copy of the CA's certificate".
I've interpreted this to mean that certmonger is running the scep-submit command with -C, to request the CA certs.  If that's the case, then it is most likely encountering the same error that we have seen when running scep-submit with -C.  When scep-submit is run with -C, no CA certs are returned, because of the missing &message=0 on certmonger 0.78.4-12.


As I mentioned in my initial writeup, this may be "broken" functionality in the MS CA, because message=0 should not be required as per the spec.  However, if there were a way to manually specify to certmonger to use this on MS CA's, that would be helpful as a workaround for this particular MS CA quirk.
 "


Thanks,

Comment 6 Rob Crittenden 2020-05-22 16:48:05 UTC
What version(s) of AD is this known to not work on so we can reproduce.

Comment 7 joel 2020-05-22 16:55:26 UTC
so far one cu has responded with Windows Server 2012 R2

Comment 8 Rob Crittenden 2020-05-22 16:57:57 UTC
The nourse RFC makes it clear that GetCACert is not required to send message.

https://tools.ietf.org/html/draft-nourse-scep-23#section-5.2.1

5.2.1.  GetCACert

   The OPERATION MUST be set to "GetCACert".

   The MESSAGE MAY be omitted, or it MAY be a string that represents the
   certification authority issuer identifier.  A CA Administrator
   defined string allows for multiple CAs supported by one SCEP server.

The guttman RFC is less clear.

https://datatracker.ietf.org/doc/draft-gutmann-scep/?include_text=1

4.2.  Get CA Certificate

   To get the CA certificate(s), the client sends a GetCACert message to
   the CA.  The OPERATION MUST be set to "GetCACert".  There is no
   request data associated with this message.

It seems it is safe to exclude &message=id from GetCACert and only provide it for GetCACaps. The current default identifier of 0 should be sufficient.

Comment 9 Rob Crittenden 2020-05-22 18:39:39 UTC
For reproduction purposes you want to verify that message=CA-IDENT is included in GetCACaps requests.

To do this call the helper directly against a SCEP server. The -c option means ask for the CA capabilities and -v will display the request information. For example:

# /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -cv
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACaps&message=0
response_code = 200
content-type = "text/plain"
code = 0
code_text = "No error"
...

The default CA-IDENT in certmonger is 0. This request shows that message is properly sent.  You can also add your own identifier with -i IDENT, ala

# /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -cv -i CAIdentifier
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACaps&message=CAIdentifier"
response_code = 200
content-type = "text/plain"
code = 0
code_text = "No error"
...

Similarly confirm that the GetCACert command does not include the message using option -C:

# /usr/libexec/certmonger/scep-submit -u https://child-dc/certsrv/mscep/mscep.dll -C -v
GET "https://child-dc/certsrv/mscep/mscep.dll?operation=GetCACert"
response_code = 200
content-type = "application/x-x509-ca-ra-cert"
code = 0
code_text = "No error"
...

And specifying an identifier:

# /usr/libexec/certmonger/scep-submit -u https://child-dc/certsrv/mscep/mscep.dll -C -v -i CAIdentifier
GET "https://child-dc/certsrv/mscep/mscep.dll?operation=GetCACert"
response_code = 200
content-type = "application/x-x509-ca-ra-cert"
code = 0
code_text = "No error"
...

Comment 11 Rob Crittenden 2020-06-01 16:50:53 UTC
I tested against an AD 2012 evaluation server instance and it requires message=<ident> for both the getcaps and getcacert operations.

Comment 12 Rob Crittenden 2020-06-01 17:23:40 UTC
Fixing this properly is going to create what looks like a regression in BZ https://bugzilla.redhat.com/show_bug.cgi?id=1608781 . It won't be.

From what I can tell an EJBCA CA has a name. It expects this name in the message sent for GetCACaps and GetCACert.  certmonger defaults to 0 so the message didn't match a known EJBCA CA.

In the docs about a different SCEP client, MobileIron, it writes:

Mobile Iron always use the CA identifier 'MobileIronSCEP' in all SCEP request. SCEP request from MobileIron always start with "operation=GetCACaps&message=MobileIronSCEP". Therefore the CAs name have to be set to 'MobilIronSCEP' to make it work. 

https://download.primekey.com/docs/EJBCA-Enterprise/6_12_0/SCEP.html

certmonger doesn't hardcode a value but defaults to 0. To set a specific value the SCEP CA must be creates with -i <identifier>

Which means the original bug was a combination of user configuration issue and my misreading of the spec(s).

Comment 13 Rob Crittenden 2020-06-02 13:40:04 UTC
*** Bug 1842960 has been marked as a duplicate of this bug. ***

Comment 14 Rob Crittenden 2020-06-02 14:39:12 UTC
Fix merged into upstream master: 5c21bcbc0c189777b8cad8658c47d2cfb4cbd2e5

Comment 16 Rob Crittenden 2020-06-02 16:08:02 UTC
Correction to c#9 reproduction. The identifier is also required for GetCACert, so it too will have &message=<identifier> where identifier is 0 by default.

Comment 20 Mohammad Rizwan 2020-06-18 12:06:47 UTC
version:
certmonger-0.78.4-14.el7.x86_64


[root@master twe]# /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -cv
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACaps&message=0"
response_code = 200
content-type = "text/plain"
code = 0
code_text = "No error"


[root@master twe]#  /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -cv -i CAIdentifier
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACaps&message=CAIdentifier"
response_code = 200
content-type = "text/plain"
code = 0
code_text = "No error"
[..]

[root@master twe]#  /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -C -v 
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACert&message=0"
response_code = 200
content-type = "application/x-x509-ca-ra-cert"
code = 0
code_text = "No error"
[..]


[root@master twe]#  /usr/libexec/certmonger/scep-submit -u https://interop.redwax.eu/test/simple/scep -C -v -i CAIdentifier
GET "https://interop.redwax.eu/test/simple/scep?operation=GetCACert&message=CAIdentifier"
response_code = 200
content-type = "application/x-x509-ca-ra-cert"
code = 0
code_text = "No error"
[..]


'message=CA-IDENTIFIER' included in the request. Hence based on above observations marking the bug a verified.

Comment 25 Orion Poplawski 2020-09-24 18:51:03 UTC
I'm running into this issue as well.  Is there anywhere I can get a fixed certmonger package?

Comment 27 errata-xmlrpc 2020-09-29 20:33:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (certmonger bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4009


Note You need to log in before you can comment on or make changes to this bug.