Bug 1071176

(In reply to Sibiao Luo from comment #0)
> Steps to Reproduce:
> 1.boot KVM guest with spice.
> # /usr/libexec/qemu-kvm -M pc -enable-kvm -m 2048 -smp 2
> -no-kvm-pit-reinjection -name sluo -uuid
> 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc
> base=localtime,clock=host,driftfix=slew -drive
> file=/home/RHEL-7.0-20140116.1_Server_x86_64.qcow2,if=none,id=drive-system-
> disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device
> virtio-blk-pci,vectors=0,bus=pci.0,addr=0x4,scsi=off,drive=drive-system-disk,
> id=system-disk,bootindex=1 -netdev
> tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device
> virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=00:01:02:03:40:21,
> bus=pci.0,addr=0x5 -k en-us -boot menu=on -spice disable-ticketing,port=5931
> -monitor stdio

Reproducible with just
/usr/libexec/qemu-kvm -enable-kvm -m 512 -spice disable-ticketing,port=5931

And I'm indeed not able to reproduce on a different RHEL7 machine :(

The machine where this fails has fips enabled (fips=1 is set on the kernel command line and /proc/sys/crypto/fips_enabled  contains 1).

reds_send_link_ack() has a call to RSA_generate_key_ex(link->tiTicketing.rsa, SPICE_TICKET_KEY_PAIR_LENGTH, link->tiTicketing.bn,  NULL); in order to initialize link->tiTicketing.rsa which returns 0 on the machine where this crash happens (most likely to indicate some kind of failure), and which returns 1 on a machine whithout the crash.

Even after turning fips mode on on my rhel7 machine I'm not hitting this crash :(

Hmm the default openssl ciphers seem different on the machine with the crash and on the machine without the crash. The one where it crashes (yours) has 50 lines in 'openssl ciphers -v' output, and no RC4/MD5 ciphers, the one where it does not crash (mine) has 75 lines there including RC4/MD5 ciphers. Have you done some crypto-related tweaking on this box?

Ah, with fips=1 + OPENSSL_FORCE_FIPS_MODE=1 in the environment I can hit the bug

(In reply to Christophe Fergeau from comment #7)
> Ah, with fips=1 + OPENSSL_FORCE_FIPS_MODE=1 in the environment I can hit the
> bug

Hmm, forget to tell you about it. I enable FIPS mode via setting PRELINKING=no in the /etc/sysconfig/prelink and appending fips=1 to the /boot/grub2/grub.cfg configuration file.

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-76.el7.x86_64 root=/dev/mapper/rhel_dhcp--11--229-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_dhcp-11-229/swap vconsole.keymap=us rd.lvm.lv=rhel_dhcp-11-229/root crashkernel=auto rhgb quiet fips=1

# cat /proc/sys/crypto/fips_enabled
1

(In reply to Christophe Fergeau from comment #6)
> Hmm the default openssl ciphers seem different on the machine with the crash
> and on the machine without the crash. The one where it crashes (yours) has
> 50 lines in 'openssl ciphers -v' output, and no RC4/MD5 ciphers, the one
> where it does not crash (mine) has 75 lines there including RC4/MD5 ciphers.
> Have you done some crypto-related tweaking on this box?
I just enable the FIPS machine to verify a bug but meet this spice issue which cause our QEMU core dumped.

I refer here to enable FIPS machine:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-Federal_Standards_And_Regulations-Federal_Information_Processing_Standard.html

This issue is related to FIPS mode, so i modify the title.

Best Regards,
sluo

The crash is fixed by http://lists.freedesktop.org/archives/spice-devel/2014-March/016271.html
However, clients can't successfully connect in fips mode. Fixing this is a bit more involved as a 1024 bit RSA key is hardcoded in the protocol used, in the server->clientlink reply message:

#define SPICE_TICKET_KEY_PAIR_LENGTH 1024
#define SPICE_TICKET_PUBKEY_BYTES (SPICE_TICKET_KEY_PAIR_LENGTH / 8 + 34)

typedef struct SPICE_ATTR_PACKED SpiceLinkReply {
    uint32_t error;
    uint8_t pub_key[SPICE_TICKET_PUBKEY_BYTES];
    uint32_t num_common_caps;
    uint32_t num_channel_caps;
    uint32_t caps_offset;
} SpiceLinkReply;

The pubkey is sent at the same time as the AUTH_SASL cap, so even when trying to use SASL, the RSA_generate_key_ex failure will cause the connection to fail :(
I think both client and server updates will be needed in order to get fips mode to work :(

(In reply to Christophe Fergeau from comment #10)
> The crash is fixed by
> http://lists.freedesktop.org/archives/spice-devel/2014-March/016271.html
> However, clients can't successfully connect in fips mode. Fixing this is a
> bit more involved as a 1024 bit RSA key is hardcoded in the protocol used,
> in the server->clientlink reply message:

Would it be possible to add a flag/cap in link msg sent by the client? that would indicate to use a fips compliant pubkey? what would be compliant btw?

> 
> #define SPICE_TICKET_KEY_PAIR_LENGTH 1024
> #define SPICE_TICKET_PUBKEY_BYTES (SPICE_TICKET_KEY_PAIR_LENGTH / 8 + 34)
> 
> typedef struct SPICE_ATTR_PACKED SpiceLinkReply {
>     uint32_t error;
>     uint8_t pub_key[SPICE_TICKET_PUBKEY_BYTES];
>     uint32_t num_common_caps;
>     uint32_t num_channel_caps;
>     uint32_t caps_offset;
> } SpiceLinkReply;
> 
> The pubkey is sent at the same time as the AUTH_SASL cap, so even when
> trying to use SASL, the RSA_generate_key_ex failure will cause the
> connection to fail :(

yeah

> I think both client and server updates will be needed in order to get fips
> mode to work :(

well, we could make it work in disable-ticketing/SASL-only, right? just do not generate a key in this case. no?

(In reply to Marc-Andre Lureau from comment #11)
> (In reply to Christophe Fergeau from comment #10)
> > The crash is fixed by
> > http://lists.freedesktop.org/archives/spice-devel/2014-March/016271.html
> > However, clients can't successfully connect in fips mode. Fixing this is a
> > bit more involved as a 1024 bit RSA key is hardcoded in the protocol used,
> > in the server->clientlink reply message:
> 
> Would it be possible to add a flag/cap in link msg sent by the client? that
> would indicate to use a fips compliant pubkey? what would be compliant btw?

Yep, that's the approach I'm currently exploring. Now that you are asking, I don't know what would be a fips-compliant pubkey :-/ Setting the key size to 2048 bits make the call not fail, but this is not consistent with what I'm seeing in openssl source.

> yeah
> 
> > I think both client and server updates will be needed in order to get fips
> > mode to work :(
> 
> well, we could make it work in disable-ticketing/SASL-only, right? just do
> not generate a key in this case. no?

With ticketing disabled, we the server will still decrypt a ticket sent by the client, and then ignore it. This means old clients expect to find a valid RSA pubkey to use for encryption in the SpiceLinkRepyl message.
For SASL authentication, the authentication method to be used is only determined after the SpiceLinkReply message has been sent, ie after the RSA pubkey has been generated. In reds.c, we first get to reds_handle_read_link_done() which calls reds_send_link_ack() which is the function generating the RSA key. Once that is sent, we wait for the client to end back a message with the auth mechanisms to use. This is handled by the reds_handle_auth_mechanism() callback.
And it seems the server has no idea whether the client supports SASL or not before getting the auth_mechanism info :( And we need to generate the RSA pubkey as old clients not supporting SASL will expect it.

Current plan of action is to append a 'ticket_encryption' uint8_t to the SpiceReplyLink message to indicate whether we use the old RSA method to encrypt the ticket, or if we use some new way of encrypting it. I'm currently thinking the new way will be 'no encryption', and will only be available when the connection goes only through SSL as I'd tend to think someone enabling fips mode will not want an unencrypted connection.

(In reply to Christophe Fergeau from comment #12)
> Now that you are asking, I
> don't know what would be a fips-compliant pubkey :-/ Setting the key size to
> 2048 bits make the call not fail, but this is not consistent with what I'm
> seeing in openssl source.

RHEL7 openssl has:
static int FIPS_rsa_builtin_keygen(RSA *rsa, int bits, BIGNUM *e_value, BN_GENCB *cb)
{
[snip]

  if (bits != 2048 && bits != 3072)
    {
      FIPSerr(FIPS_F_RSA_BUILTIN_KEYGEN, FIPS_R_INVALID_KEY_LENGTH);
      return 0;
    }


so it wants a bigger key than the one we try to use.

This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

*** Bug 1068288 has been marked as a duplicate of this bug. ***

patches upstream:
http://lists.freedesktop.org/archives/spice-devel/2014-March/016418.html

Patch still hasn't been ack'ed upstream, just resent it:
http://lists.freedesktop.org/archives/spice-devel/2014-October/017626.html

This patch has now been acked, however this is making a slight tweak to the SPICE protocol (see patch commit log). This is fully compatible with older clients, but this is still something I"m uncomfortable pushing into RHEL late into the release cycle. As there is no customer case attached, I'll push this to 7.2.
If this is something we must have in 7.1, we can revisit that change.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2429.html