Bug 723447 - [libvirt] libvirtd (0.9.3-6) fails to start with rhevm's cert configured : 'cert_file="/etc/pki/vdsm/certs/vdsmcert.pem" # by vdsm'
Summary: [libvirt] libvirtd (0.9.3-6) fails to start with rhevm's cert configured : '...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt
Version: 6.1
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: rc
: ---
Assignee: Daniel Berrangé
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-07-20 08:31 UTC by David Naori
Modified: 2011-12-06 11:17 UTC (History)
17 users (show)

Fixed In Version: libvirt-0.9.3-7.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-12-06 11:17:04 UTC
Target Upstream Version:


Attachments (Terms of Use)
vdsmcert.pem, cacert.pem (2.98 KB, application/x-gzip)
2011-07-20 11:08 UTC, David Naori
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:1513 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2011-12-06 01:23:30 UTC

Internal Links: 723568

Description David Naori 2011-07-20 08:31:55 UTC
Description of problem:
Libvirtd fails to start with rhevm's certificate:

11:08:43.669: 27874: debug : virNetTLSContextNew:447 : cacert=/etc/pki/vdsm/certs/cacert.pem cacrl=(null) cert=/etc/pki/vdsm/certs/vdsmcert.pem key=/etc/pki/vdsm/keys/vdsmkey.pem requireValid=1 isServer=1
11:08:43.671: 27874: debug : virNetTLSContextSanityCheckCert:116 : isServer 1 isCA 0 certFile /etc/pki/vdsm/certs/vdsmcert.pem
11:08:43.672: 27874: debug : virNetTLSContextSanityCheckCert:161 : Cert /etc/pki/vdsm/certs/vdsmcert.pem basic constraints 0
11:08:43.672: 27874: debug : virNetTLSContextSanityCheckCert:194 : Cert /etc/pki/vdsm/certs/vdsmcert.pem key usage status -56 usage 0
11:08:43.672: 27874: error : virNetTLSContextSanityCheckCert:198 : Unable to query certificate /etc/pki/vdsm/certs/vdsmcert.pem basic constraints The requested data were not available.

conf:
'cert_file="/etc/pki/vdsm/certs/vdsmcert.pem" # by vdsm'

#cat /etc/pki/vdsm/certs/vdsmcert.pem

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1 (0x1)
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=US, O=RedHat, CN=CA-pink-vds2.qa.lab.tlv.redhat.com
        Validity
            Not Before: Mar 14 19:49:43 2011
            Not After : Mar 12 17:49:43 2021 GMT
        Subject: C=US, O=RedHat, CN=CA-pink-vds2.qa.lab.tlv.redhat.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:bd:30:35:e9:87:63:75:36:92:96:5b:20:df:17:
                    53:c9:7c:7a:3f:11:51:31:e9:e2:09:b3:fa:fc:bc:
                    b9:1b:b1:a3:16:53:22:9b:b9:68:07:7a:10:79:8f:
                    d6:8b:35:6f:a2:37:99:5c:68:fa:03:89:63:89:22:
                    dd:52:a1:38:4a:02:0e:a8:6d:81:9e:1b:85:94:b5:
                    6a:82:23:8c:25:7a:4f:19:8d:c8:f1:f9:b5:4d:56:
                    7e:0e:0f:f0:e3:db:a4:8a:14:69:c2:55:fa:4c:f3:
                    95:b2:15:2e:bb:cf:6f:97:14:41:ce:f5:b6:c4:8f:
                    3d:00:13:ef:0f:ec:0a:2f:69
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                CA:4C:6F:79:1F:8B:4A:31:6F:92:B1:7E:CD:F8:3B:64:58:A3:16:89
            Authority Information Access:
                CA Issuers - URI:http://my.ca/ca.crt

            X509v3 Authority Key Identifier:
                keyid:CA:4C:6F:79:1F:8B:4A:31:6F:92:B1:7E:CD:F8:3B:64:58:A3:16:89
                DirName:/C=US/O=RedHat/CN=CA-pink-vds2.qa.lab.tlv.redhat.com
                serial:01

            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
    Signature Algorithm: sha1WithRSAEncryption
        8c:84:a1:c7:ea:16:67:f7:15:3b:d7:79:be:d8:c8:75:88:24:
        55:b9:94:9c:4d:d2:bc:30:ae:39:d7:ca:2e:d4:83:f6:e1:c7:
        22:00:0e:a0:26:70:92:88:1b:3d:a5:60:63:33:d6:f1:71:76:
        a0:79:4f:4f:a3:b3:57:a1:bc:72:3a:76:3a:c4:e3:3e:25:25:
        c6:c8:75:c8:4b:c9:92:85:24:b3:5d:4e:ef:cb:1c:8f:2c:8d:
        d8:41:f5:05:e6:04:de:b8:00:19:5f:57:55:e1:78:ae:16:60:
        39:0e:52:ca:fe:d4:c3:7f:79:9e:d9:02:35:75:ab:d7:e5:a2:
        25:eb
-----BEGIN CERTIFICATE-----
MIIC+jCCAmOgAwIBAgIBATANBgkqhkiG9w0BAQUFADBLMQswCQYDVQQGEwJVUzEP
MA0GA1UEChMGUmVkSGF0MSswKQYDVQQDEyJDQS1waW5rLXZkczIucWEubGFiLnRs
di5yZWRoYXQuY29tMCIXETExMDMxNDE5NDk0MyswMjAwFw0yMTAzMTIxNzQ5NDNa
MEsxCzAJBgNVBAYTAlVTMQ8wDQYDVQQKEwZSZWRIYXQxKzApBgNVBAMTIkNBLXBp
bmstdmRzMi5xYS5sYWIudGx2LnJlZGhhdC5jb20wgZ8wDQYJKoZIhvcNAQEBBQAD
gY0AMIGJAoGBAL0wNemHY3U2kpZbIN8XU8l8ej8RUTHp4gmz+vy8uRuxoxZTIpu5
aAd6EHmP1os1b6I3mVxo+gOJY4ki3VKhOEoCDqhtgZ4bhZS1aoIjjCV6TxmNyPH5
tU1Wfg4P8OPbpIoUacJV+kzzlbIVLrvPb5cUQc71tsSPPQAT7w/sCi9pAgMBAAGj
gekwgeYwHQYDVR0OBBYEFMpMb3kfi0oxb5Kxfs34O2RYoxaJMC8GCCsGAQUFBwEB
BCMwITAfBggrBgEFBQcwAoYTaHR0cDovL215LmNhL2NhLmNydDBzBgNVHSMEbDBq
gBTKTG95H4tKMW+SsX7N+DtkWKMWiaFPpE0wSzELMAkGA1UEBhMCVVMxDzANBgNV
BAoTBlJlZEhhdDErMCkGA1UEAxMiQ0EtcGluay12ZHMyLnFhLmxhYi50bHYucmVk
aGF0LmNvbYIBATAPBgNVHRMBAf8EBTADAQH/MA4GA1UdDwEB/wQEAwIBBjANBgkq
hkiG9w0BAQUFAAOBgQCMhKHH6hZn9xU713m+2Mh1iCRVuZScTdK8MK4518ou1IP2
4cciAA6gJnCSiBs9pWBjM9bxcXageU9Po7NXobxyOnY6xOM+JSXGyHXIS8mShSSz
XU7vyxyPLI3YQfUF5gTeuAAZX1dV4XiuFmA5DlLK/tTDf3me2QI1davX5aIl6w==
-----END CERTIFICATE-----

Version-Release number of selected component (if applicable):
0.9.3-6

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Daniel Veillard 2011-07-20 09:11:14 UTC
It's the upstream commit 79591d4fbf666a5b1a5b93c9f30ddc81e77d593a

   Add sanity checking of basic constraints, key purpose & key usage
    
    Gnutls requires that certificates have basic constraints present
    to be used as a CA certificate. OpenSSL doesn't add this data
    by default, so add a sanity check to catch this situation. Also
    validate that the key usage and key purpose constraints contain
    correct data
    
    * src/rpc/virnettlscontext.c: Add sanity checking of certificate
      constraints

in that case gnutls_x509_crt_get_basic_constraints() returns 0 indicating
it doesn't think that this is a CA certificate.

RETURN VALUE
       If the certificate is a CA a positive value will be returned,  or  zero
       if  the certificate does not have CA flag set.  A negative value may be
       returned in case of errors.  If the certificate does  not  contain  the
       basicConstraints  extension  GNUTLS_E_REQUESTED_DATA_NOT_AVAILABLE will
       be returned.

then gnutls_x509_crt_get_key_usage() just fails on that certificate ad that
leads to the abort.
I note that we are passing NULL values for a bunch of arguments of gnutls
maybe this is working fine in recent versions but leads to troubles on the
RHEL-6 one ... we should check this

Daniel

Comment 3 Daniel Berrangé 2011-07-20 09:44:20 UTC
Please attach both your CA certificate, and the VDSM server certificate to this bug

Comment 4 Daniel Berrangé 2011-07-20 09:55:08 UTC
> query certificate /etc/pki/vdsm/certs/vdsmcert.pem basic constraints The
requested data were not available.

This error message is actually wrong. It is actually missing 'key usage' data from the certificate. In the past libvirt did not enforce presence of this data, but now it does. While the CA cert looks OK, I believe the server certificate (vdsmcert.pem) was not created following the libvirt documentation.

When creating a server certificate, you need to create a .info file for certtool containing the following information:

#cat > server.info <<EOF
organization = Name of your organization
cn = fully.qualified.server.hostname
tls_www_server
encryption_key
signing_key
EOF

The 'tls_www_server', 'encryption_key' and 'signing_key' bits ensure you get the key usage & key purpose extension fields are filled in for the certificate.

For a correctly generated server cert,  'certtool -i --infile /path/to/cert' should show

        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): FALSE
                Key Purpose (not critical):
                        TLS WWW Server.
                Key Usage (critical):
                        Digital signature.
                        Key encipherment.

While a client certificate should show

	Extensions:
		Basic Constraints (critical):
			Certificate Authority (CA): FALSE
		Key Purpose (not critical):
			TLS WWW Client.
		Key Usage (critical):
			Digital signature.
			Key encipherment.

And a CA cert should show

        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): TRUE
                Key Usage (critical):
                        Certificate signing.

Comment 5 Daniel Berrangé 2011-07-20 10:00:30 UTC
Opps, missed out the docs link for how to generate certs:

http://libvirt.org/remote.html#Remote_TLS_server_certificates

Comment 7 David Naori 2011-07-20 11:08:10 UTC
Created attachment 513977 [details]
vdsmcert.pem, cacert.pem

attached both cacert and vdsmcert.

Comment 9 Daniel Berrangé 2011-07-20 13:20:02 UTC
While the VDSM certs are missing the key usage/purpose data, re-checking the relevant RFCs shows we should have treated that as non-fatal by default. We should only treat it as a fatal problem if the key usage + purpose *is* in the certificate and marked as 'critical':
	
http://www.redhat.com/archives/libvir-list/2011-July/msg01333.html


In addition to that, to get RHEV/VDSM to follow best practice recommendations from the libvirt.org/remote.html when generating certificates, there are a couple of changes that should be made to RHEV-M's code:

In the file:

   rhevm/backend/manager/conf/ca/cert.template

There should be two extra fields set under the '[ v3_ca ]' heading, just after the basicConstraints data:

For client certificates:

  keyUsage=critical,digitalSignature,keyEncipherment
  extendedKeyUsage=critical,clientAuth

For server certificates:

  keyUsage=critical,digitalSignature,keyEncipherment
  extendedKeyUsage=critical,serverAuth


It seems like VDSM might in fact need a dual purpose certificate, which can be done with

  keyUsage=critical,digitalSignature,keyEncipherment
  extendedKeyUsage=critical,serverAuth,clientAuth

Comment 13 Daniel Berrangé 2011-07-21 12:31:35 UTC
To avoid future regressions in this area, I have created a comprehensive unit test case for libvirt's TLS cert handling

http://www.redhat.com/archives/libvir-list/2011-July/msg01490.html

Comment 14 errata-xmlrpc 2011-12-06 11:17:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html


Note You need to log in before you can comment on or make changes to this bug.