Bug 620490 (CVE-2010-2938)

Summary: CVE-2010-2938 kernel: guest crashes on non-EPT machines may crash the host as well
Product: [Other] Security Response Reporter: Eugene Teo (Security Response) <eteo>
Component: vulnerabilityAssignee: Paolo Bonzini <pbonzini>
Severity: medium Docs Contact:
Priority: medium    
Version: unspecifiedCC: bburns, ddugger, dhoward, drjones, jpirko, jzheng, leiwang, lwang, mjc, pbonzini, plyons, rkrcmar, rmusil, security-response-team
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-04-06 08:05:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 621429, 621430    
Bug Blocks: 514489    
Description Flags
patch to fix the bug none

Comment 1 Paolo Bonzini 2010-08-02 17:25:07 UTC
Created attachment 436076 [details]
image triggering the bug

Booting the attached gpxe.lkrn file on a fully-virtualized Xen guest running on RHEL5.3 or newer will cause the host to reboot.

Here are the results for various versions of RHEL:
- 5.0 -> guest hangs
- 5.1 -> guest hangs
- 5.2 -> guest crashes (will attach xm dmesg output)
- 5.3 hypervisor, 5.2 kernel -> host crashes

So the bug was introduced between 2.6.18-92 and 2.6.18-128.

To trigger the bug, you need only this in grub.conf:

title BZ620490
    kernel /gpxe.lkrn

Comment 2 Paolo Bonzini 2010-08-03 13:05:13 UTC
The bug is caused by the EPT backport, i.e. the union of

  ebd3f7b ([xen] Intel EPT 2MB patch, 2008-09-10)
  1f20de0 ([xen] Intel EPT Migration patch, 2008-08-28)
  20d1d7f ([xen] Intel EPT Patch, 2008-08-28)
  f2197fa ([xen] Intel pre EPT Patch, 2008-08-28)

Comment 3 Paolo Bonzini 2010-08-04 10:30:20 UTC
The bug is only reproducible on Intel hardware.  On AMD hardware it doesn't crash, no matter if hardware-assisted paging is enabled or not.

Next to check: 1) Intel w/EPT, 2) upstream and/or SLES.

Comment 6 Paolo Bonzini 2010-08-10 12:58:47 UTC
Serial console output for the crash:

(XEN) ************* VMCS Area **************
(XEN) 	 16-bit Guest-State Fields
(XEN) 		0x00000800: 0x0000 0x0008 0x0000 0x0000 
(XEN) 		0x00000808: 0x0000 0x0000 0x0000 0x0008 
(XEN) 	 16-bit Host-State Fields
(XEN) 		0x00000c00: 0x0000 0xe008 0x0000 0x0000 
(XEN) 		0x00000c08: 0x0000 0x0000 0xe040 
(XEN) 	 64-bit Control Fields
(XEN) 		0x00002000: 0x00000000cecf8000 0x0000000000000000 0x00000000cecf9000 0x0000000000000000 
(XEN) 		0x00002004: 0x00000000cea01000 0x0000000000000000 0x0000000000000000 0x0000000000000000 
(XEN) 		0x00002008: 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 
(XEN) 		0x0000200c: 0x0000000000000000 0x0000000000000000 ------------------ ------------------ 
(XEN) 		0x00002010: 0xffff57b853d2a2ad 0x00000000ffff57b8 0x000000215a6ef000 0x0000000000000021 
(XEN) 	 64-bit Guest-State Fields
(XEN) 		0x00002800: 0xffffffffffffffff 0x00000000ffffffff 0x0000000000000000 0x0000000000000000 
(XEN) 	 32-bit Control Fields
(XEN) 		0x00004000: 0x0000001f 0x96a1e7fa 0x00044000 0x00000000 
(XEN) 		0x00004008: 0x00000000 0x00000000 0x0003efff 0x00000000 
(XEN) 		0x00004010: 0x00000000 0x000011ff 0x00000000 0x00000020 
(XEN) 		0x00004018: 0x00000000 0x00000000 0x00000000 
(XEN) 	 32-bit RO Data Fields
(XEN) 		0x00004400: 0x0000000c 0x80000021 0x00000000 0x00050033 
(XEN) 		0x00004408: 0x00000000 0x00000000 0x00000003 0x00000000 
(XEN) 	 32-bit Guest-State Fields
(XEN) 		0x00004800: 0x0000ffff 0x0000ff62 0x0000ffff 0x0000ffff 
(XEN) 		0x00004808: 0x0000ffff 0x0000ffff 0x00000000 0x00002067 
(XEN) 		0x00004810: 0x000affff 0x0000017f 0x00010093 0x000000ff 
(XEN) 		0x00004818: 0x00010093 0x00010093 0x00010093 0x00010093 
(XEN) 		0x00004820: 0x00000082 0x0000008b 0x00000000 0x00000000 
(XEN) 		0x00004828: 0x00000000 0x00000000 
(XEN) 	 32-bit Host-State Fields
(XEN) 		0x00004c00: 0x00000000 
(XEN) 	 Natural 64-bit Control Fields
(XEN) 		0x00006000: 0xffffffffffffffff 0xffffffffffffffff 0x0000000000050033 0x0000000000000611 
(XEN) 		0x00006008: 0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000 
(XEN) 	 64-bit RO Data Fields
(XEN) 		0x00006400: 0x0000000000000000 0x0000000000010002 0x00000000003f7fd5 0x00000000cebd6000 
(XEN) 		0x00006408: 0x0000000000000595 0x0000000000000000 
(XEN) 	 Natural 64-bit Guest-State Fields
(XEN) 		0x00006800: 0x0000000080050033 0x00000000cebd5980 0x0000000000002631 0x0000000000000000 
(XEN) 		0x00006808: 0x0000000000ff000a 0x0000000000000000 0x0000000000000000 0x0000000000000000 
(XEN) 		0x00006810: 0x0000000000000000 0x0000000000000000 0x00000000000d7404 0x00000000000affff 
(XEN) 		0x00006818: 0x00000000000d9600 0x0000000000000400 0x0000000000007b7a 0x0000000000000b52 
(XEN) 		0x00006820: 0x0000000000010002 0x0000000000000000 0x0000000000000000 0x0000000000000000 
(XEN) 	 Natural 64-bit Host-State Fields
(XEN) 		0x00006c00: 0x000000008005003b 0x000000215a7eb000 0x00000000000026b0 0x0000000044a64940 
(XEN) 		0x00006c08: 0xffffffff80627580 0xffff828c80336800 0xffff8321a73de000 0xffff8300cead0080 
(XEN) 		0x00006c10: 0x0000000000000000 0x0000000000000000 0xffff8300cfde7fa0 0xffff828c801761b0 
(XEN) secondary exec control = 0x00000001
(XEN) ----[ Xen-3.1.2-210.el5  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    5
(XEN) RIP:    e008:[<ffff828c8016e1ce>] vmcs_dump_vcpu+0x16e/0x200
(XEN) RFLAGS: 0000000000010242   CONTEXT: hypervisor
(XEN) rax: 0000000000002804   rbx: 0000000080000021   rcx: 0000000000002eb9
(XEN) rdx: 000000000000000a   rsi: 000000000000000a   rdi: ffff828c803164a4
(XEN) rbp: 0000000080000021   rsp: ffff8300cfde7e88   r8:  0000000000000002
(XEN) r9:  0000000000000000   r10: 00000000ffffffff   r11: 0000000000000001
(XEN) r12: 0000000000000000   r13: ffff8300cfde7f28   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026b0
(XEN) cr3: 000000215a7eb000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen stack trace from rsp=ffff8300cfde7e88:
(XEN)    0000000000000000 ffff828c80174f10 0000000000000000 ffff828c80162b2e
(XEN)    0000000000000005 ffff828c80164056 ffff8300cebd5840 ffff8300cebd4080
(XEN)    ffff8300cf1e0080 ffff828c8015d3ac 0000000000000000 ffff8300cebd5780
(XEN)    ffff8300cebd4080 0000000000009cdb 0000000000007bc4 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff828c801761cf
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000007bc4 0000000000009cdb 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000050013 0000000000000804
(XEN)    0000000000000804 0000000000010330 000000000009c5a0 0000000000000000
(XEN)    0000000000080225 0000000000000000 0000000000000046 0000000000138f7c
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000005 ffff8300cebd4080
(XEN) Xen call trace:
(XEN)    [<ffff828c8016e1ce>] vmcs_dump_vcpu+0x16e/0x200
(XEN)    [<ffff828c80174f10>] vmx_vmexit_handler+0x1e0/0x1480
(XEN)    [<ffff828c80162b2e>] pt_update_irq+0xae/0x170
(XEN)    [<ffff828c80164056>] vlapic_has_interrupt+0x26/0x60
(XEN)    [<ffff828c8015d3ac>] cpu_has_pending_irq+0x2c/0x60
(XEN)    [<ffff828c801761cf>] vmx_asm_vmexit_handler+0x1f/0x30
(XEN) ****************************************
(XEN) Panic on CPU 5:
(XEN) FATAL TRAP: vector = 6 (invalid opcode)
(XEN) ****************************************

Comment 9 Paolo Bonzini 2010-08-10 20:43:09 UTC
The bug is that several VMCS fields are accessed even if they are not available in hardware.

Notice how, in the dump, the crash is after the "secondary exec control" field.

    print_section("Natural 64-bit Host-State Fields", 0x6c00, 0x6c16, 2);
    printk("secondary exec control = 0x%08x\n",
    printk("Guest PAT = 0x%08x%08x\n",
           (uint32_t)__vmread(GUEST_PAT_HIGH), (uint32_t)__vmread(GUEST_PAT));

The fields should be guarded by cpu_has_vmx_pat, and similarly for other fields below.  Alternatively, upstream c/s 15911 will fix the bug by using __vmread_safe instead of __vmread.

Comment 14 Eugene Teo (Security Response) 2010-08-13 04:41:05 UTC
Upstream commit:

Comment 16 Andrew Jones 2010-09-08 14:48:52 UTC
*** Bug 628472 has been marked as a duplicate of this bug. ***

Comment 17 Eugene Teo (Security Response) 2010-09-09 03:09:44 UTC
*** Bug 628474 has been marked as a duplicate of this bug. ***

Comment 18 Eugene Teo (Security Response) 2010-09-09 03:09:47 UTC
*** Bug 628475 has been marked as a duplicate of this bug. ***

Comment 24 errata-xmlrpc 2010-09-29 14:55:12 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Linux 5

Via RHSA-2010:0723 https://rhn.redhat.com/errata/RHSA-2010-0723.html

Comment 25 Paolo Bonzini 2010-09-29 15:29:57 UTC
Created attachment 450509 [details]
patch to fix the bug

Posting here the patch for ease of reference.