Bug 800093 - CRL preventing repository access - Client certificate did not match the global repo auth CA certificate
Summary: CRL preventing repository access - Client certificate did not match the globa...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: 6.0.0
Hardware: Unspecified
OS: Unspecified
high
high vote
Target Milestone: Unspecified
Assignee: Mike McCune
QA Contact: Og Maciel
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-03-05 17:56 UTC by James Laska
Modified: 2019-09-26 15:54 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-22 18:30:22 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description James Laska 2012-03-05 17:56:25 UTC
Description of problem:

Unable to deploy CloudForms images using katello hosted content.  It appears that attempts to access the pulp repos using a valid crt and key are blocked by the CRL.  Removing the CRL file (/etc/pki/pulp/content/40e36648.r0) works around the problem.

Version-Release number of selected component (if applicable):
 * candlepin-0.5.23-1.el6.src.rpm
 * katello-0.1.301-2.el6.src.rpm
 * katello-candlepin-cert-key-pair-1.0-1.src.rpm
 * katello-certs-tools-1.0.3-1.el6.src.rpm
 * katello-cli-0.1.100-2.el6.src.rpm
 * katello-configure-0.1.101-1.el6.src.rpm
 * katello-qpid-broker-key-pair-1.0-1.src.rpm
 * katello-qpid-client-key-pair-1.0-1.src.rpm
 * katello-selinux-0.1.8-1.el6.src.rpm
 * pulp-1.0.0-4.el6.src.rpm

How reproducible:
 * 2 of 2 attempts end up with

Steps to Reproduce:
1. # curl --insecure -L --cert /tmp/uebercert.crt --key /tmp/uebercert.key https://flatline-katello.usersys.redhat.com/pulp/repos/redhat/Stage/content/beta/rhel/server/6/6Server/x86_64/cf-tools/1.0/os/repodata/repomd.xml
  
Actual results:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /pulp/repos/redhat/Stage/content/beta/rhel/server/6/6Server/x86_64/cf-tools/1.0/os/repodata/repomd.xml
on this server.</p>

Expected results:

<?xml version="1.0" encoding="UTF-8"?>
<repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm">
  <revision>1330696594</revision>
  <data type="other_db">
    <location href="repodata/59b8470f65ad8b95a911b95c2d3ebdf4655826ea-other.sqlite.bz2"/>
    <checksum type="sha">59b8470f65ad8b95a911b95c2d3ebdf4655826ea</checksum>
    <timestamp>1330696595.47</timestamp>
    <size>18742</size>
    <open-size>132096</open-size>
    <open-checksum type="sha">b13f8d54bf12296aa3fcc9f14332bc6a29825b55</open-checksum>
    <database_version>10</database_version>
  </data>
...

Additional info:

== /var/log/httpd/ssl_kt_error_log ==
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] Cert verification failed against 1 ca cert(s) and 1 CRL(s)
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] Current Time: <Mon Mar  5 10:30:31 2012>
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] Certificate to verify: 
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] \tsubject=</CN=8a8b66ac35d3d4120135d4b82e930093>, issuer=</C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=flatline-katello.usersys.redhat.com>, subject.as_hash=<1564283010>, issuer.as_hash=<1088644680>, fingerprint=<A25F74A08C68E3D1EC32CEB930181196>, serial=<3368552365664118609>, version=<2>, check_ca=<0>, notBefore=<Mar  2 18:42:14 2012 GMT>, notAfter=<Dec  3 02:18:42 2021 GMT>
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] Using a CA Chain with 1 cert(s)
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] \tCA: subject=</C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=flatline-katello.usersys.redhat.com>, issuer=</C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=flatline-katello.usersys.redhat.com>, subject.as_hash=<1088644680>, issuer.as_hash=<1088644680>, fingerprint=<B32208DB5AD4FA547119D0EA51992B56>, serial=<17830954723505377029>, version=<2>, check_ca=<1>, notBefore=<Mar  2 14:32:16 2012 GMT>, notAfter=<Jan 18 14:32:16 2038 GMT>
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] Using a CRL Stack with 1 CRL(s)
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] \tCRL: issuer=</C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=flatline-katello.usersys.redhat.com>, issuer.as_hash=<1088644680> lastUpdate=<Mar  4 17:00:00 2012 GMT>, nextUpdate=<Mar  5 17:00:00 2012 GMT>Client certificate did not match the global repo auth CA certificate
[Mon Mar 05 10:30:31 2012] [error] [client 10.11.231.160] mod_wsgi (pid=14081): Client denied by server configuration: '/var/www/pub/repos/redhat/Stage/content/beta/rhel/server/6/6Server/x86_64/cf-tools/1.0/os/repodata/repomd.xml'.
[Mon Mar 05 10:30:31 2012] [info] [client 10.11.231.160] Connection closed to child 3 with standard shutdown (server flatline-katello.usersys.redhat.com:443)

== Suggested Beta3 workaround ==

# CRL_HASH=$(openssl x509 -subject_hash -in /etc/candlepin/certs/candlepin-ca.crt  | head -n1)
# mv /etc/pki/pulp/content/${CRL_HASH}.r0  /etc/pki/pulp/content/RENAMED_${CRL_HASH}.r0

Comment 3 John Matthews 2012-03-06 20:55:28 UTC
The cause of the CRL not working was the lack of "CRL Sign" under "X509v3 Key Usage".
Example:
X509v3 Key Usage:
  Digital Signature, Key Encipherment, Certificate Sign, CRL Sign

Change is here:
$ git diff
diff --git a/certs-tools/certs/sslToolConfig.py b/certs-tools/certs/sslToolConfig.py
index a8ae8e1..0c31f51 100644
--- a/certs-tools/certs/sslToolConfig.py
+++ b/certs-tools/certs/sslToolConfig.py
@@ -368,9 +368,9 @@ x509_extensions         = req_ca_x509_extensions
 
 [ req_ca_x509_extensions ]
 basicConstraints = CA:true
-keyUsage = digitalSignature, keyEncipherment, keyCertSign
+keyUsage = digitalSignature, keyEncipherment, keyCertSign, cRLSign
 extendedKeyUsage = serverAuth, clientAuth
-nsCertType = server
+nsCertType = server, sslCA
 # PKIX recommendations harmless if included in all certificates.
 nsComment               = "Katello SSL Tool Generated Certificate"
 subjectKeyIdentifier    = hash


With above change:
      X509v3 extensions:
            X509v3 Basic Constraints:
                CA:TRUE
            X509v3 Key Usage:
                Digital Signature, Key Encipherment, Certificate Sign, CRL Sign
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            Netscape Cert Type:
                SSL Server, SSL CA
            Netscape Comment:
                Katello SSL Tool Generated Certificate
            X509v3 Subject Key Identifier:
                97:304:FD:618:C0:23:F2:7B:9A:ED:B16:7F:B2:55:97:BA:40
            X509v3 Authority Key Identifier:
                keyid:97:304:FD:618:C0:23:F2:7B:9A:ED:B16:7F:B2:55:97:BA:40
                DirName:/C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=beta.lan
                serial:81:16:C6:95:B6:39:67:52

Without change:
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:TRUE
            X509v3 Key Usage:
                Digital Signature, Key Encipherment, Certificate Sign
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            Netscape Cert Type:
                SSL Server
            Netscape Comment:
                Katello SSL Tool Generated Certificate
            X509v3 Subject Key Identifier:
                E5:43:F7:80:B0:6A:69:EA:0F:73:E08:771:09:01:70:AF:83:FE
            X509v3 Authority Key Identifier:
                keyid:E5:43:F7:80:B0:6A:69:EA:0F:73:E08:771:09:01:70:AF:83:FE
                DirName:/C=US/ST=North Carolina/L=Raleigh/O=Red Hat/OU=Cloud BU/CN=beta.lan
                serial:B1:4A:85:8B:45:ED:3D:65


For background, this is the command I ran after patching the sslToolConfig.py to examine what the new ca would look like.

katello-ssl-tool --gen-ca -p "$(cat /etc/katello/candlepin_ca_password-file)" --set-country 'US' --set-state 'North Carolina' --set-city 'Raleigh' --set-org 'Red Hat' --set-org-unit 'Cloud BU' --set-common-name `hostname` --set-email '' --ca-key 'candlepin-cert.key' --ca-cert 'candlepin-cert.crt' --ca-cert-rpm  'katello-candlepin-cert-key-pair' -vvv --force

Comment 6 James Laska 2012-03-07 13:43:39 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Due to a bug in how certificates are constructed during System Engine configuration, the certificate restriction list (CRL) blocks all katello-hosted repository access.  This issue will be resolved in a future katello update.  In the meantime, to work around this problem, you can disable CRL support entirely using the following commands:

$ CRL_HASH=$(openssl x509 -subject_hash -in
/etc/candlepin/certs/candlepin-ca.crt  | head -n1)
$ mv /etc/pki/pulp/content/${CRL_HASH}.r0 
/etc/pki/pulp/content/DISABLED_${CRL_HASH}.r0

Comment 7 Mike McCune 2012-03-08 22:27:18 UTC
Fixed in katello-certs-tools-1.0.4-1 and above

Comment 9 Og Maciel 2012-03-09 19:53:19 UTC
Validated:
* candlepin-0.5.24-1.el6.noarch
* candlepin-tomcat6-0.5.24-1.el6.noarch
* katello-0.1.303-1.el6.noarch
* katello-all-0.1.303-1.el6.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.0.4-1.el6.noarch
* katello-cli-0.1.102-1.el6.noarch
* katello-cli-common-0.1.102-1.el6.noarch
* katello-common-0.1.303-1.el6.noarch
* katello-configure-0.1.104-1.el6.noarch
* katello-glue-candlepin-0.1.303-1.el6.noarch
* katello-glue-foreman-0.1.303-1.el6.noarch
* katello-glue-pulp-0.1.303-1.el6.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-0.1.8-1.el6.noarch
* pulp-1.0.0-4.el6.noarch
* pulp-common-1.0.0-4.el6.noarch
* pulp-selinux-server-1.0.0-4.el6.noarch

Comment 10 James Laska 2012-04-20 13:15:44 UTC
Removing technical note and requires_release_note? flag.  This issue has been fixed and requires *no* any release notes.

Comment 11 James Laska 2012-04-20 13:15:45 UTC
Deleted Technical Notes Contents.

Old Contents:
Due to a bug in how certificates are constructed during System Engine configuration, the certificate restriction list (CRL) blocks all katello-hosted repository access.  This issue will be resolved in a future katello update.  In the meantime, to work around this problem, you can disable CRL support entirely using the following commands:

$ CRL_HASH=$(openssl x509 -subject_hash -in
/etc/candlepin/certs/candlepin-ca.crt  | head -n1)
$ mv /etc/pki/pulp/content/${CRL_HASH}.r0 
/etc/pki/pulp/content/DISABLED_${CRL_HASH}.r0


Note You need to log in before you can comment on or make changes to this bug.