Created attachment 926463 [details] Create necessary certs to recreate scenario. I'm running into an issue with latest pulp REST binding unable to verify server certificate if the CA cert is a chain cert. It looks like the new pulp is using code from m2crypto library that does not support this feature. Attached are 2 small scripts that will recreate this scenario. Note: test_m2crypto.py have pieces of code yanked from pulp bindings/server.py Note2: The scripts assume pulp is installed locally. Here are the steps: 1) ./create_rhui_ssl_certs.sh - This would output a bunch of certs, the important ones are copied to ./certs dir. 2) Edit line 8 of test_m2crypto.py to point to $HOME/certs/server-ca-chain.pem 3) Edit /etc/httpd/conf.d/ssl.conf with following key-value pair: 3.1) SSLCertificateFile $HOME/certs/test-cert.pem 3.2) SSLCertificateKeyFile $HOME/certs/test-key.pem 4) Restart httpd 5) python test_m2crypto.py 6) openssl verify -verbose -CAfile $HOME/certs/server-ca-chain.pem $HOME/certs/test-cert.pem Output should look like: [root@rhua ~]# python test_m2crypto.py certificate verify failed [root@rhua ~]# openssl verify -verbose -CAfile /root/certs/server-ca-chain.pem /root/certs/test-cert.pem /root/certs/test-cert.pem: OK The version of pulp I'm using is: [root@rhua ~]# rpm -qa | grep "pulp" python-isodate-0.5.0-1.pulp.el6.noarch python-pulp-rpm-common-2.4.0-0.30.beta.el6.noarch createrepo-0.9.9-21.2.pulp.el6.noarch pulp-admin-client-2.4.0-0.30.beta.el6.noarch python-kombu-3.0.15-12.pulp.el6.noarch pulp-puppet-plugins-2.4.0-0.30.beta.el6.noarch pulp-selinux-2.4.0-0.30.beta.el6.noarch pulp-rpm-admin-extensions-2.4.0-0.30.beta.el6.noarch m2crypto-0.21.1.pulp-8.el6.x86_64 python-pulp-common-2.4.0-0.30.beta.el6.noarch python-pulp-puppet-common-2.4.0-0.30.beta.el6.noarch python-pulp-bindings-2.4.0-0.30.beta.el6.noarch python-pulp-client-lib-2.4.0-0.30.beta.el6.noarch mod_wsgi-3.4-1.pulp.el6.x86_64 pulp-server-2.4.0-0.30.beta.el6.noarch pulp-rpm-plugins-2.4.0-0.30.beta.el6.noarch pulp-puppet-admin-extensions-2.4.0-0.30.beta.el6.noarch pulp-v2-cds-server-1.0.1-1.git.3.9a1a04f.el6.noarch
Created attachment 926464 [details] Test ssl verification using chain ca cert.
Brian has determined that this does not affect Satellite 6 GA as they do not use chain certificates. Given that M2Crypto does not support this feature, it will be a significant effort to resolve this issue. I will mark this as untriaged so we can decide if we really want to keep this in 2.4.2.
When revisiting this BZ, please know that lack of CA Chain support will negatively impact RHUI customers. RHUI customers often use CA Chains when managing their SSL certificates.
This is also a common case for Satellite 6 installations, but not with the default installer. Given use cases that require this by other projects, I'm triaging this as 2.4.2, Urgent.
It turned out that the 1 in the call to set_verify wasn't a boolean, but an integer specifying how deep we should be willing to follow certificate chains. I believe that setting it to 1 meant that the peer certificate had to be signed by a root certificate authority. To fix this, I changed that line to this[0]: ssl_context.set_verify(SSL.verify_peer, depth=100) I would recommend using Apache's SSLCertificateChainFile directive[1] to give the certificate chain to the client instead of using Pulp's ca_path setting to do this. Pulp's ca_path should only need to be a path to trusted root certificates. I will write steps to reproduce this bug in another comment. [0] https://github.com/pulp/pulp/pull/1142 [1] https://httpd.apache.org/docs/2.2/mod/mod_ssl.html#sslcertificatechainfile
To reproduce this bug: 1) Create a root certificate authority (self signed v1 certificate will do). 2) Configure a host running pulp-admin to trust the root CA certificate. 3) Create an intermediate certificate authority. 4) Sign the intermediate certificate with the root certificate from step 1 (this must generate a v3 certificate with the appropriate certificate authority extensions[0]. 5) Create a server certificate for Apache to use. 6) Sign the server certificate with the intermediate certificate. 7) Configure Apache's SSLCertificateChainFile directive (in /etc/httpd/conf.d/ssl.conf) to point at the intermediate CA certificate you created in step 3. This will cause Apache to hand that CA out along with its own certificate signed by that CA. 8) Try to use pulp-admin rpm repo list (or any other command that uses the server). You should see step 8 fail before upgrading to the version that fixes this bug. Once it is fixed, step 8 should succeed. [0] For example, I generated one that had these extensions, as printed by openssl x509… -text: X509v3 extensions: X509v3 Subject Key Identifier: C7:AF:E2:AC:53:E6:5F:60:D6:52:3B:61:83:02:9F:4F:5C:11:52:DB X509v3 Authority Key Identifier: keyid:5C:88:1C:B7:DF:DC:A8:C1:19:D1:C2:26:E0:DD:4C:A7:91:EC:D7:F3 X509v3 Basic Constraints: CA:TRUE X509v3 Key Usage: Certificate Sign, CRL Sign Netscape Cert Type: SSL CA, S/MIME CA X509v3 Subject Alternative Name: email:rbarlow X509v3 Issuer Alternative Name: <EMPTY>
verified [root@pulp-24-server ~]# rpm -qa pulp-server pulp-server-2.4.1-0.7.beta.el6.noarch [root@pulp-24-server ~]# followed David's steps from the description [root@pulp-24-server ~]# python test_m2crypto.py certificate verify failed [root@pulp-24-server ~]# openssl verify -verbose -CAfile /root/certs/server-ca-chain.pem /root/certs/test-cert.pem /root/certs/test-cert.pem: OK [root@pulp-24-server ~]# Also followed steps from https://bugzilla.redhat.com/show_bug.cgi?id=1129719#c6
This is fixed in Pulp-2.4.1-1.