Bug 1333404

Summary: libvirtd allows SSLv3 connections and poor ciphers
Product: Red Hat Enterprise Linux 7 Reporter: Martin Poole <mpoole>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.2CC: berrange, bressers, dberry, dyuan, fdelorey, jsuchane, mpoole, nmavrogi, rbalakri, rjones, rrajaram, yafu
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-2.0.0-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1333415 (view as bug list) Environment:
Last Closed: 2016-11-03 18:44:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1333415    

Description Martin Poole 2016-05-05 12:27:20 UTC
Description of problem:

because there is no mechanism to provide a gnutls cipher string it defaults to the library hard-coded selection which allows SSLv3 and numerous weak ciphers.


Version-Release number of selected component (if applicable):

  libvirt-1.2.17-13.el7_2.4

How reproducible:

Always

Steps to Reproduce:
1. Enable TLS and scan


Actual results:

  Supported Server Cipher(s):
Accepted  SSLv3    256 bits  DHE-RSA-AES256-SHA
Accepted  SSLv3    256 bits  DHE-RSA-CAMELLIA256-SHA
Accepted  SSLv3    256 bits  AES256-SHA
Accepted  SSLv3    256 bits  CAMELLIA256-SHA
Accepted  SSLv3    128 bits  DHE-RSA-AES128-SHA
Accepted  SSLv3    128 bits  DHE-RSA-CAMELLIA128-SHA
Accepted  SSLv3    128 bits  EDH-RSA-DES-CBC3-SHA
Accepted  SSLv3    128 bits  AES128-SHA
Accepted  SSLv3    128 bits  CAMELLIA128-SHA
Accepted  SSLv3    128 bits  DES-CBC3-SHA
Accepted  SSLv3    128 bits  RC4-SHA
Accepted  SSLv3    128 bits  RC4-MD5
Accepted  TLSv1.0  256 bits  ECDHE-RSA-AES256-SHA
Accepted  TLSv1.0  256 bits  DHE-RSA-AES256-SHA
Accepted  TLSv1.0  256 bits  DHE-RSA-CAMELLIA256-SHA
Accepted  TLSv1.0  256 bits  AES256-SHA
Accepted  TLSv1.0  256 bits  CAMELLIA256-SHA
Accepted  TLSv1.0  128 bits  ECDHE-RSA-AES128-SHA
Accepted  TLSv1.0  128 bits  DHE-RSA-AES128-SHA
Accepted  TLSv1.0  128 bits  ECDHE-RSA-DES-CBC3-SHA
Accepted  TLSv1.0  128 bits  DHE-RSA-CAMELLIA128-SHA
Accepted  TLSv1.0  128 bits  EDH-RSA-DES-CBC3-SHA
Accepted  TLSv1.0  128 bits  AES128-SHA
Accepted  TLSv1.0  128 bits  CAMELLIA128-SHA
Accepted  TLSv1.0  128 bits  DES-CBC3-SHA
Accepted  TLSv1.0  128 bits  ECDHE-RSA-RC4-SHA
Accepted  TLSv1.0  128 bits  RC4-SHA
Accepted  TLSv1.0  128 bits  RC4-MD5
Accepted  TLSv1.1  256 bits  ECDHE-RSA-AES256-SHA
Accepted  TLSv1.1  256 bits  DHE-RSA-AES256-SHA
Accepted  TLSv1.1  256 bits  DHE-RSA-CAMELLIA256-SHA
Accepted  TLSv1.1  256 bits  AES256-SHA
Accepted  TLSv1.1  256 bits  CAMELLIA256-SHA
Accepted  TLSv1.1  128 bits  ECDHE-RSA-AES128-SHA
Accepted  TLSv1.1  128 bits  DHE-RSA-AES128-SHA
Accepted  TLSv1.1  128 bits  ECDHE-RSA-DES-CBC3-SHA
Accepted  TLSv1.1  128 bits  DHE-RSA-CAMELLIA128-SHA
Accepted  TLSv1.1  128 bits  EDH-RSA-DES-CBC3-SHA
Accepted  TLSv1.1  128 bits  AES128-SHA
Accepted  TLSv1.1  128 bits  CAMELLIA128-SHA
Accepted  TLSv1.1  128 bits  DES-CBC3-SHA
Accepted  TLSv1.1  128 bits  ECDHE-RSA-RC4-SHA
Accepted  TLSv1.1  128 bits  RC4-SHA
Accepted  TLSv1.1  128 bits  RC4-MD5
Accepted  TLSv1.2  256 bits  ECDHE-RSA-AES256-GCM-SHA384
Accepted  TLSv1.2  256 bits  ECDHE-RSA-AES256-SHA384
Accepted  TLSv1.2  256 bits  ECDHE-RSA-AES256-SHA
Accepted  TLSv1.2  256 bits  DHE-RSA-AES256-GCM-SHA384
Accepted  TLSv1.2  256 bits  DHE-RSA-AES256-SHA256
Accepted  TLSv1.2  256 bits  DHE-RSA-AES256-SHA
Accepted  TLSv1.2  256 bits  DHE-RSA-CAMELLIA256-SHA
Accepted  TLSv1.2  256 bits  AES256-GCM-SHA384
Accepted  TLSv1.2  256 bits  AES256-SHA256
Accepted  TLSv1.2  256 bits  AES256-SHA
Accepted  TLSv1.2  256 bits  CAMELLIA256-SHA
Accepted  TLSv1.2  128 bits  ECDHE-RSA-AES128-GCM-SHA256
Accepted  TLSv1.2  128 bits  ECDHE-RSA-AES128-SHA256
Accepted  TLSv1.2  128 bits  ECDHE-RSA-AES128-SHA
Accepted  TLSv1.2  128 bits  DHE-RSA-AES128-GCM-SHA256
Accepted  TLSv1.2  128 bits  DHE-RSA-AES128-SHA256
Accepted  TLSv1.2  128 bits  DHE-RSA-AES128-SHA
Accepted  TLSv1.2  128 bits  ECDHE-RSA-DES-CBC3-SHA
Accepted  TLSv1.2  128 bits  DHE-RSA-CAMELLIA128-SHA
Accepted  TLSv1.2  128 bits  EDH-RSA-DES-CBC3-SHA
Accepted  TLSv1.2  128 bits  AES128-GCM-SHA256
Accepted  TLSv1.2  128 bits  AES128-SHA256
Accepted  TLSv1.2  128 bits  AES128-SHA
Accepted  TLSv1.2  128 bits  CAMELLIA128-SHA
Accepted  TLSv1.2  128 bits  DES-CBC3-SHA
Accepted  TLSv1.2  128 bits  ECDHE-RSA-RC4-SHA
Accepted  TLSv1.2  128 bits  RC4-SHA
Accepted  TLSv1.2  128 bits  RC4-MD5


Expected results:

Depends on fix type, but at a minimum the disabling of SSLv3 and removal of any RC4, MD5 ciphers.

Client side probably also needs similar fix.

Additional info:

libvirtd provides no priority string to gnutls (gnutls_priority_init et al) so gnutls defaults to "NORMAL".  The global settings file is not triggered because that requires use of an '@' prefix.

Comment 1 Jaroslav Suchanek 2016-05-13 14:58:19 UTC
Nikos, before any further actions in libvirt, may I ask you for your explanation/recommendation regarding "weak" ciphers in gnutls like SSLv3 which are by default enabled in gnutls?

Comment 3 Daniel Berrangé 2016-05-13 15:12:33 UTC
Libvirt simply uses gnutls_set_default_priority() which is the recommendation given in https://fedoraproject.org/wiki/Packaging:CryptoPolicies to make it use the system default policies for crypto ciphers / protocols.  So if libvirt is showing SSLv3 enabled, that's because the system policy has enabled it. Libvirt shouldn't need any configuration of its own to customize ciphers/protocols if the system policy is working correctly in RHEL

Comment 4 Jaroslav Suchanek 2016-05-17 13:56:04 UTC
(In reply to Daniel Berrange from comment #3)
> Libvirt simply uses gnutls_set_default_priority() which is the
> recommendation given in
> https://fedoraproject.org/wiki/Packaging:CryptoPolicies to make it use the
> system default policies for crypto ciphers / protocols.  So if libvirt is
> showing SSLv3 enabled, that's because the system policy has enabled it.
> Libvirt shouldn't need any configuration of its own to customize
> ciphers/protocols if the system policy is working correctly in RHEL

While the above is correct I was not able to find any straightforward way how to change the cipher system policy in RHEL (besides enforcing fips140-2, where I am not sure what impact it has on gnutls). Seems like having an option in libvirt's configuration file and using gnutls_priority_set_direct() might help.

Jano can you please take care of this one, thanks.

Comment 5 Ján Tomko 2016-05-18 12:08:13 UTC
Proposed upstream patch adding a configuration option for the priorities:
https://www.redhat.com/archives/libvir-list/2016-May/msg01345.html

Comment 6 Ján Tomko 2016-05-19 11:57:03 UTC
The upstream patch was rejected as it would only allow to work around the problem for libvirt, not other applications using gnutls.

gnutls_set_default_priority should be giving us both reasonable and configurable defaults.

Comment 7 Nikos Mavrogiannopoulos 2016-05-19 12:05:19 UTC
Not all applications need to adhere to PCI-DSS. If we fix that globally without co-ordination we will break use cases that require SSL 3.0. If it is an urgent issue in libvirt it cannot be solved by the gnutls component.

Comment 8 Daniel Berrangé 2016-05-19 12:39:46 UTC
The original bug description here is merely complaining there is no mechanism to change the default cipher/protocol selection in libvirt, not asking to actually change the out of the box defaults.

Libvirt simply uses gnutls_set_default_priority() on the basis, that this makes it honour gnutls' global defaults. Assuming gnutls global defaults can be changed by an admin deploying RHEL (eg via /etc/crypto-policies/config or an equivalent RHEL specific method), then IMHO this can be NOTABUG.

Comment 9 Nikos Mavrogiannopoulos 2016-05-19 12:50:36 UTC
(In reply to Daniel Berrange from comment #8)

> Libvirt simply uses gnutls_set_default_priority() on the basis, that this
> makes it honour gnutls' global defaults. Assuming gnutls global defaults can
> be changed by an admin deploying RHEL (eg via /etc/crypto-policies/config or
> an equivalent RHEL specific method), then IMHO this can be NOTABUG.

We do not have any equivalent RHEL specific method to configure system wide policy (that's only in Fedora and incomplete as it doesn't yet cover Java and NSS). As I mentioned before we plan to introduce that in RHEL-7.5 for all the crypto libraries (gnutls/openssl/NSS/Java) so that everything is consistent. We can discuss about bringing that sooner (RHEL-7.4), but that cannot address an urgent customer issue.

Comment 10 Daniel Berrangé 2016-05-19 13:00:22 UTC
(In reply to Nikos Mavrogiannopoulos from comment #9)
> (In reply to Daniel Berrange from comment #8)
> 
> > Libvirt simply uses gnutls_set_default_priority() on the basis, that this
> > makes it honour gnutls' global defaults. Assuming gnutls global defaults can
> > be changed by an admin deploying RHEL (eg via /etc/crypto-policies/config or
> > an equivalent RHEL specific method), then IMHO this can be NOTABUG.
> 
> We do not have any equivalent RHEL specific method to configure system wide
> policy (that's only in Fedora and incomplete as it doesn't yet cover Java
> and NSS). As I mentioned before we plan to introduce that in RHEL-7.5 for
> all the crypto libraries (gnutls/openssl/NSS/Java) so that everything is
> consistent. We can discuss about bringing that sooner (RHEL-7.4), but that
> cannot address an urgent customer issue.

If its planned for a RHEL update, then from libvirt I'd rather we just wait for that update to arrive and, if needed focus effort on accelerating that, rather than inventing yet more application specific config flags

Comment 14 Nikos Mavrogiannopoulos 2016-05-23 07:30:26 UTC
Given the discussion above I'm inclined to move this bug back to libvirt. There is no way gnutls can help libvirt have customized settings while using the default options. You may want to wait until the global defaults are satisfactory to the client with the support case, but the bug should remain to libvirt and be blocked to the global default change (which is planned for 7.5.0, thus no bug yet). I'll reassign the bug today, unless there is an objection.

Comment 16 Martin Poole 2016-05-31 08:22:10 UTC
Public security articles now published noting that it is not possible to adequately secure libvirt for remote connections.

Comment 17 Daniel Berrangé 2016-05-31 09:06:19 UTC
Re-opening, since there is definitely a gap here that we've identified - it just isn't yet 100% clear how we want to solve it long term. I'll be sending a mail to gnutls dev mailing list to discuss some possibilities in near future.

Comment 18 Daniel Berrangé 2016-06-03 16:02:23 UTC
I've sent a patch to gnutls upstream to better handle application specific overrides & fallbacks with the global priorities file

  http://lists.gnutls.org/pipermail/gnutls-devel/2016-June/008007.html

if gnutls accepts this, then we could modify libvirt to call

  gnutls_priority_set_direct(session, "@LIBVIRT,SYSTEM", NULL);

by default (similarly for QEMU + GTK-VNC). 

Combined with this enhancement to the fedora crypto-policy generator:

  https://github.com/nmav/fedora-crypto-policies/pull/3

We would enable sys admins todo per-application customization of cipher policies in the global configuration file, without needing to ever touch libvirt/qemu/gtk-vnc config files directly.

Of course, we probably still need to add support for direct customization in libvirt to facilitate use with older gnutls versions that already exist in distros, but for the long term, this proposed solution will be much nicer.

Comment 19 Daniel Berrangé 2016-06-08 13:21:23 UTC
The libvirt part of this is addressed in

8dfb796080de822bf0525ae8c83f7b704abe9375 Use @SYSTEM priority for TLS on Fedora >= 21
6d310c9cffa08ed7e1ea2d57113929dc831702bf remote: allow TLS priority to be customized
5f1837eaca4beba418ea75ce5ee9ec235e277fa2 Pass config file object through to driver open methods
416358d99df0929a3901735c557bda8f393820ea remote: allow TLS protocol/cipher priority override in URI
c7d0fbe62b0f59b08d0f140e58e48f9837fe8476 libvirtd: add config option for TLS priority
214489f550b95e4accf55896afb39a45be1175df rpc: allow priority string to be passed to TLS context
cbb2e91ecc56d218ca20c1a9ba345eaf754d6c5d configure: allow setting default TLS priority string
20c5ded9d013e8186e98e36ef62aea2e86f5c2f3 rpc: set gnutls log function at global init time
d8a8af3492194089d1f101e10305421fba2d1760 tls: remove support for gnutls 1.x.x, require 2.2.0


Fixing the QEMU side is non-trivial effort, since it'll require a complete change in the way we configure QEMU TLS

Comment 23 Daniel Berrangé 2016-06-08 14:56:52 UTC
Also one followup

19aee2cb9af340e12a4552405ba271054263ac37 libvirt.spec.in: include NORMAL as a fallback for @SYSTEM in TLS prio

Comment 26 Ján Tomko 2016-06-22 08:17:48 UTC
The most recent commit from comment 19 relevant to RHEL:
commit 6d310c9cffa08ed7e1ea2d57113929dc831702bf
Author:     Daniel P. Berrange <berrange>
CommitDate: 2016-06-08 13:48:45 +0100

    remote: allow TLS priority to be customized

git describe: v1.3.5-97-g6d310c9

Comment 28 yafu 2016-09-06 09:47:25 UTC
Verify the bug with libvirt-2.0.0-6.el7.x86_64.

Scenerio1 Test the tls_priority config in the libvirt.conf:
0.Prepare the tls env between hostA and hostB;

1.Set tls_priority in libvirt.conf:
  tls_priority="NORMAL:-VERS-SSL3.0"

2.Connect to server hypervisor:
  #virsh -c qemu+tls://hostB/system
 Welcome to virsh, the virtualization interactive terminal.

 Type:  'help' for help with commands
       'quit' to quit

 virsh # 

3.Also test tls_priority as "NORMAL:-VERS-TLS1.0","NORMAL:-VERS-TLS1.1","NORMAL:-VERS-TLS1.2",can connect to server hypervisor correctly.

4.Set tls_priority to the wrong values:
   tls_priority="NORMAL:-VERS-SSL4.0"

5.Connect to server hypervisor:
   # virsh -c qemu+tls://hostB/system
error: failed to connect to the hypervisor
error: Failed to set TLS session priority to NORMAL:-VERS-TLS4.0: The request is invalid.

6.Also test tls_prioprity as " ","-VERS-TLS1.1",".", then connect to server hypervisor, both libvirtd and virsh client works well.


Scenerio2 Test TLS protocol/cipher priority override in URI:
1.Override TLS priority in URI with right value:
  #virsh -c qemu+tls://hostB/system?tls_priority=NORMAL:-VERS-SSL3.0
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # 

2.Also test tls_priority as "NORMAL:-VERS-TLS1.0","NORMAL:-VERS-TLS1.1","NORMAL:-VERS-TLS1.2",can connect to server hypervisor correctly.

3.Override TLS priority in URI with wrong value:
 #virsh -c qemu+tls://hostB/system?tls_priority=NORMAL:-VERS-TLS1.3
error: failed to connect to the hypervisor
error: Failed to set TLS session priority to NORMAL:-VERS-TLS1.3: The request is invalid.

Comment 29 yafu 2016-09-06 09:53:36 UTC
Hi,Daniel,

Since @SYSTEM priority for TLS is not supported in rhel7.3, would you help to check the steps in comment 28 are enough for verifying this bug? Thanks a lot.

Comment 30 Daniel Berrangé 2016-09-06 09:58:08 UTC
That is fine - gnutls in RHEL-7 doesn't yet support the @BLAH style strings.

Comment 31 yafu 2016-09-06 10:56:54 UTC
Move the bug to VERIFIED according comment 28.

Comment 33 errata-xmlrpc 2016-11-03 18:44:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html

Comment 42 Ján Tomko 2018-05-15 07:13:12 UTC
*** Bug 1578050 has been marked as a duplicate of this bug. ***