Bug 501111 - pki authentication not working with libvirtd
Summary: pki authentication not working with libvirtd
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 10
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Daniel Veillard
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2009-05-16 14:16 UTC by Jeff Layton
Modified: 2014-06-18 07:38 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-11-13 13:09:31 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
server side log (1.60 KB, text/plain)
2009-05-18 14:08 UTC, Jeff Layton
no flags Details
client side log (2.34 KB, text/plain)
2009-05-18 14:09 UTC, Jeff Layton
no flags Details
/etc/hosts from server (228 bytes, text/plain)
2009-05-18 14:13 UTC, Jeff Layton
no flags Details
libvirtd.conf (6.90 KB, text/plain)
2009-05-18 14:15 UTC, Jeff Layton
no flags Details
client side log (connecting to fqdn) (2.41 KB, text/plain)
2009-05-18 14:33 UTC, Jeff Layton
no flags Details

Description Jeff Layton 2009-05-16 14:16:57 UTC
I just tried setting up a virtualization host with TLS "auth". I've set up a CA, verified that the certificates are good.

I've configured libvirt to listen on a socket and pointed it at the right certificate and key files. When I try to connect from my client (an F11 machine) using either virt-manager or virsh, it fails. The error from virt-manager is this:

Unable to open connection to hypervisor URI 'qemu+tls://salusa/system':
<class 'libvirt.libvirtError'> unable to connect to 'salusa': Invalid argument
None

...on the server side, I see this error in /var/log/messages:

2009-05-16T10:00:32.638817-04:00 salusa libvirtd: remoteCheckCertificate: the client certificate is not trusted.
2009-05-16T10:00:32.638904-04:00 salusa libvirtd: remoteCheckCertificate: failed to verify client's certificate

I've tried setting up tls_allowed_dn_list on the server to explicitly allow this cert, but that didn't make any difference. I've also put both client and server in SELinux permissive mode, but that also didn't make any difference. Finally, I tried running libvirtd in the foreground with --verbose set, but that didn't make it log anything more helpful.

Attaching my libvirtd.conf file from the server. The cert and key files are all in their default locations.

Comment 1 Jeff Layton 2009-05-16 15:17:48 UTC
For the record, I went ahead and used the certtool util to verify the certs:

client:

$ certtool --verify-chain --infile /etc/pki/libvirt/clientcert.pem --infile /etc/pki/CA/cacert.pem  | grep Verif
	Verification output: Verified.

server:

$ certtool --verify-chain --infile /etc/pki/libvirt/servercert.pem --infile /etc/pki/CA/cacert.pem  | grep Verif
	Verification output: Verified.

Comment 2 Daniel Berrangé 2009-05-18 13:01:20 UTC
You don't appear to have attached anything as per comment #0.

Can you provide

 * /etc/libvirt/libvirtd.conf
 * /etc/hosts
 * Output from 'hostname' command
 * Full unedited output from  
   
    LIBVIRT_DEBUG=1  virsh -c qemu+tls://salusa/system


 * Full log from libvirtd, after settting log_level=1 in libvirtd.conf, and then running

   LIBVIRT_DEBUG=1 /usr/sbin/libvirtd

and stop the log after a connection attempt

Also can you also verify that running 'host salusa' returns an IP address, and that then running 'host $IP' returns 'salusa', and that 'salusa' matches the fully qualified  hostname configured for the machine as per the 'hostname' command.

Comment 3 Jeff Layton 2009-05-18 14:08:14 UTC
Created attachment 344440 [details]
server side log

Comment 4 Jeff Layton 2009-05-18 14:09:04 UTC
Created attachment 344441 [details]
client side log

Comment 5 Jeff Layton 2009-05-18 14:12:27 UTC
It turns out that my libvirtd.conf is actually blank once you strip out comments, so I won't bother attaching it.

Server-side hostname:

# hostname
salusa.poochiereds.net

# host salusa
salusa.poochiereds.net has address 192.168.1.2

# host 192.168.1.2
2.1.168.192.in-addr.arpa domain name pointer salusa.poochiereds.net.

Similar results for the client. Certificates being used specify the FQDN's of the client and server.

Comment 6 Jeff Layton 2009-05-18 14:13:12 UTC
Created attachment 344442 [details]
/etc/hosts from server

Comment 7 Jeff Layton 2009-05-18 14:15:35 UTC
Created attachment 344443 [details]
libvirtd.conf

Actually, I did make a few changes to make it so that I didn't need root privs to run virt-manager locally. Here's what I'm using now. I don't think those changes make much difference in cert handling though.

Comment 8 Daniel Berrangé 2009-05-18 14:18:52 UTC
Ahh, in your initial comment you indicate you're using the URI:

  'qemu+tls://salusa/system'


Since your hostname is salusa.poochiereds.net, you should be using that in the URI, not the short form. Can you try connecting using

  'virsh -c qemu+tls://salusa.poochiereds.net/system'

Comment 9 Jeff Layton 2009-05-18 14:33:14 UTC
Created attachment 344444 [details]
client side log (connecting to fqdn)

Sorry, should have made it clear that I tried that already. The results are the same, here's the client-side log from the fqdn connect attempt.

Comment 10 Daniel Berrangé 2009-05-18 17:09:06 UTC
Hmmm, are the clocks on the client and server hosts in sync with each other ?

Comment 11 Mark McLoughlin 2009-05-25 14:15:54 UTC
Jeff?

Comment 12 Jeff Layton 2009-05-25 14:20:14 UTC
Sorry, missed the query before. Yes, clocks are synched between client and server -- the server in this case is an ntp server as well and the client syncs to it.

Comment 13 Jan Dvorak 2009-08-14 14:54:46 UTC
I have the exactly same problem and after some investigation it seems like a bug somewhere between gnutls and libvirt. Google suggests that TLS might be broken in this case.

Comment 14 Daniel Berrangé 2009-11-13 11:17:19 UTC
We've recently come across an issue that looks very similar to your bug. Did you by chance generate your CA certificate using OpenSSL ?

Can you run

  # certtool -i --infile <CA-CERT-FILE>


And paste in the output from the 'Extensions' block.

What we expect to see is:

	Extensions:
		Basic Constraints (critical):
			Certificate Authority (CA): TRUE
		Key Usage (critical):
			Certificate signing.
		Subject Key Identifier (not critical):
			f2a0611cc5f3d8491e225bb215cff6a64192db5f


if the extensions field is empty then that would be the cause of this problem.

Comment 15 Jeff Layton 2009-11-13 13:03:49 UTC
Haven't played with this in a while so I don't know whether it's still a problem or not...

Yes, I did generate my cacert with openssl. Here's the extensions block in the output:

	Extensions:
		Basic Constraints (not critical):
			Certificate Authority (CA): FALSE
		Unknown extension 2.16.840.1.113730.1.13 (not critical):
			ASCII: ..OpenSSL Generated Certificate
			Hexdump: 161d4f70656e53534c2047656e657261746564204365727469666963617465
		Subject Key Identifier (not critical):
			b531b908954ead69b372b42a25a83b940e82f888
		Authority Key Identifier (not critical):
			b531b908954ead69b372b42a25a83b940e82f888

Comment 16 Daniel Berrangé 2009-11-13 13:09:31 UTC
Ok, this is the problem:

   Certificate Authority (CA): FALSE


your CA certificate is claiming that it is not to be used as a CA certificate. GNUTLS enforces this when checking the certificate chains on a connection, hence why its saying

  "client certificate is not trusted."

even though certtool says the cert is valid.

So if you re-create your CA  cert ensuring is has  "Certificate Authority (CA): TRUE", then it will work with libvirt.

We have another BZ open about giving a better error message diagnostic for this scenario:

https://bugzilla.redhat.com/show_bug.cgi?id=529179


Note You need to log in before you can comment on or make changes to this bug.