Bug 611822 - libvirtd produces potentially misleading advice in error output for TLS failure
libvirtd produces potentially misleading advice in error output for TLS failure
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.0
All Linux
low Severity medium
: rc
: ---
Assigned To: Justin Clift
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-07-06 11:05 EDT by Daniel Berrange
Modified: 2015-09-27 22:33 EDT (History)
9 users (show)

See Also:
Fixed In Version: libvirt-0.8.7-1.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 556599
Environment:
Last Closed: 2011-05-19 09:19:32 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Daniel Berrange 2010-07-06 11:05:56 EDT
+++ This bug was initially created as a clone of Bug #556599 +++

Created an attachment (id=385230)
Patch to display appropriate error message when libvirtd client fails to connect to libvirtd server using TLS

Customer had a typo in  client's "Distinguished Name"  specified in "tls_allowed_dn_list" in /etc/libvirt/libvirtd.conf on the libvirtd server/hypervisor.Because of the error in defining the
"tls_allowed_dn_list" field, the libvirtd client was being refused
a connection to libvirtd on the remote host.

The following error message showed up in /var/log/messages file on the 
libvirtd server.

Dec  3 14:53:09 sf4150host01 libvirtd: 14:53:09.859: error : remoteCheckCertificate: client's Distinguished Name is not on the list of allowed clients (tls_allowed_dn_list).  Use 'openssl x509 -in clientcert.pem -text' to view the Distinguished Name field in the client
certificate, or run this daemon with --verbose option.

As per the above message,he executed "openssl x509 -in clientcert.pem -text" command and  got output in below format.

"C=AU, O=Sydney Medical School, L=Sydney, ST=NSW, CN=sf4150host00.med.usyd.edu.au"

He tried using this string in "libvirtd.conf" and he was  not able to get rid of the  authentication failure issue .

Then he used  "certtool -i --infile /etc/pki/libvirt/clientcert.pem"
command to retrieve "DN" string from the certificate and output was ,

"C=AU,O=Sydney Medical School,L=Sydney,ST=NSW,CN=sf4150host00.med.usyd.edu.au"

When compared to "openssl" output we can see that there is a "space" 
character difference among these .Using the DN string retrieved using "certtool" command he was able to get it working.

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
I've reproduced this using a KVM setup.

1)Set up TLS certificates for verifying the identity of libvirtd server
and client.Refer to http://libvirt.org/remote.html for more information on setting up a Certificate Authority(CA) and TLS certificates.

2)Setup of libvirtd server:
2.1)Enable an access control list of client certificate Distinguished Names (DNs) which can connect to the TLS port on this server.In order to reproduce 
the customer reported error,make sure that there are spaces after the comma
in the fields.

vi /etc/libvirt/libvirtd.conf
# By default, no DN's are checked
tls_allowed_dn_list = ["C=US, O=Red Hat, L=Raleigh, ST=Raleigh, CN=nachandr.rdu.redhat.com"]

2.2)Enable logging.Refer to http://libvirt.org/logging.html for more information on logging in libvirt.
Note that I've used config variables in configuration files to control
logging behavior.Environment variables can also be used to control logging behavior.
# Logging controls
#

# Logging level: 0 none, 4 errors, 3 warnings, 2 informations, 1 debug
# basically 1 will log everything possible
log_level = 1


2.3)After making the above changes,start the libvirtd server in listening mode by running it with the '--listen' and '--verbose' options or by editing /etc/sysconfig/libvirtd. 
  
vi  /etc/sysconfig/libvirtd

LIBVIRTD_ARGS="--listen --verbose"

The verbose option is to get more informative log messages.
Starting libvirtd with the listen option ensures that it listens for secure
TLS connections on the public TCP/IP port.Also note that listen_tls[default value= 1] in /etc/libvirt/libvirtd.conf is turned on by default.

2.4)Start libvirtd server

/etc/init.d/libvirtd start

3)Run the following command on the libvirtd client to connect to the remote libvirtd server:
virsh -c qemu+tls://dhcp243-202.rdu.redhat.com/system list --all

  
Actual results:
libvirtd client fails to connect to the libvirtd server and the following misleading error message is seen in /var/log/messages on libvirtd server:

Dec  3 14:53:09 sf4150host01 libvirtd: 14:53:09.859: error : remoteCheckCertificate: client's Distinguished Name is not on the list of allowed clients (tls_allowed_dn_list).  Use 'openssl x509 -in clientcert.pem -text' to view the Distinguished Name field in the client
certificate, or run this daemon with --verbose option.


Expected results:
A more appropriate error message like the following should be displayed:

"remoteCheckCertificate: client's Distinguished Name is not on the list of allowed clients (tls_allowed_dn_list).Libvirt supports a Certificate Authority(CA)set up by GnuTLS certtool.Run 'certtool -i --infile /etc/pki/libvirt/clientcert.pem' to view the Distinguished Name field in the client certificate, or run this daemon with --verbose option."


Additional info:
When there are no spaces after the comma in the fields in the DN specified in
tls_allowed_dn_list,libvirtd client can successfully connect the the libvirtd server.

vi /etc/libvirt/libvirtd.conf
# By default, no DN's are checked
tls_allowed_dn_list = ["C=US,O=Red Hat,L=Raleigh,ST=Raleigh, CN=nachandr.rdu.redhat.com"]
Comment 1 Justin Clift 2011-01-07 13:08:25 EST
This was fixed in upstream libvirt some time ago:

commit 63d1b07f83382fdd82ac348810a0e766c2c9bfd1
Author: Justin Clift <jclift@redhat.com>
Date:   Thu Sep 16 01:39:08 2010 +1000

    libvirtd: improve the error message displayed on tls client auth failure

    This address BZ # 556599:

      https://bugzilla.redhat.com/show_bug.cgi?id=556599
Comment 2 Jiri Denemark 2011-01-09 18:58:07 EST
Built into libvirt-0.8.7-1.el6
Comment 3 zhanghaiyan 2011-01-10 23:48:50 EST
Verified this bug PASS with libvirt-0.8.7-1.el6.x86_64
- kernel-2.6.32-94.el6.x86_64
- qemu-kvm-0.12.1.2-2.128.el6.x86_64

The test steps are almost the same with in comment 8 except the client host and server host are not the same one.

Actual result:
On server host, # tail -f /var/log/messages
Jan 10 23:43:56 dhcp-65-132 libvirtd: 23:43:56.679: 909: error : remoteCheckCertificate:1210 : Client's Distinguished Name is not on the list of allowed clients (tls_allowed_dn_list).  Use 'certtool -i --infile clientcert.pem' to view theDistinguished Name field in the client certificate,or run this daemon with --verbose option.
Jan 10 23:43:56 dhcp-65-132 libvirtd: 23:43:56.679: 909: error : remoteCheckAccess:1250 : remoteCheckCertificate: failed to verify client's certificate
Comment 4 zhanghaiyan 2011-01-10 23:49:36 EST
Verified this bug PASS with libvirt-0.8.7-1.el6.x86_64
- kernel-2.6.32-94.el6.x86_64
- qemu-kvm-0.12.1.2-2.128.el6.x86_64

The test steps are almost the same with in https://bugzilla.redhat.com/show_bug.cgi?id=556599#c8 except the client host and server host are not the same one.

Actual result:
On server host, # tail -f /var/log/messages
Jan 10 23:43:56 dhcp-65-132 libvirtd: 23:43:56.679: 909: error : remoteCheckCertificate:1210 : Client's Distinguished Name is not on the list of allowed clients (tls_allowed_dn_list).  Use 'certtool -i --infile clientcert.pem' to view theDistinguished Name field in the client certificate,or run this daemon with --verbose option.
Jan 10 23:43:56 dhcp-65-132 libvirtd: 23:43:56.679: 909: error : remoteCheckAccess:1250 : remoteCheckCertificate: failed to verify client's certificate
Comment 6 Cui Chun 2011-02-17 00:38:45 EST
Rechecked it on the following test environment according to comment 4 on the different client and server host. It is passed.

Test environment:
libvirt-0.8.7-6.el6
qemu-kvm-0.12.1.2-2.144.el6
kernel-2.6.32-113.el6
Comment 11 errata-xmlrpc 2011-05-19 09:19:32 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0596.html

Note You need to log in before you can comment on or make changes to this bug.