Bug 484227

Summary: [Intel 5.4 FEAT] virtualization feature VTd: hypervisor changes (Xen)
Product: Red Hat Enterprise Linux 5 Reporter: Don Dugger <ddugger>
Component: kernel-xenAssignee: Don Dugger <ddugger>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: high Docs Contact:
Priority: high    
Version: 5.4CC: ahe, cward, dzickus, james.brown, jane.lv, jvillalo, knoel, luyu, rpacheco, sghosh, tao, xen-maint
Target Milestone: rcKeywords: FutureFeature
Target Release: 5.4   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-09-02 08:19:33 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 331741, 460955    
Attachments:
Description Flags
[1/16] add pci definitions and access functions
none
Add public header files
none
[3/16] system changes
none
[4/16] bulk of VTd code
none
[5/16] irq injection changes
none
[6/16] Intercept I/O for assigned device
none
[7/16] memory map changes
none
[8/16] add domctl interfaces
none
[9/16] fix EPT cache attributes
none
[10/16] avoid redundant context mapping
none
[11/16] fix 32bit build
none
[12/16] fix ia64 build
none
[13/16] emulate accesses to PCI regs cf8/crc
none
[14/16] support for queue invalidation
none
[15/16] support interrupt remapping
none
[16/16] cleanup VTd2 code to match upstream
none
[1/21] add pci definitions and access funtions
none
[2/21] add VT-d public header files
none
[3/21] some system changes for VT-d
none
[4/21] add VT-d specific files
none
[5/21] IRQ injection changes for VT-d
none
[6/21] Intercept I/O for assigned device
none
[7/21] memory changes for VT-d
none
[8/21] Add domctl interfaces for VT-d
none
[9/21] Fix EPT for VT-d
none
[10/21] vtd: avoid redundant context mapping
none
[11/21] emulate accesses to PCI regs cr8/cfc
none
[12/21] VT-d2: support queue invalidation
none
[13/21] VT-d2: support interrupt remapping
none
[14/21] cleanup VTd2 code to match upstream
none
[15/21] xen: rename evtchn_lock to event_lock
none
[16/21] xen: convert pirq to per-domain
none
[17/21] xen: MSI supprt internal functions
none
[18/21] xen: MSI support interface
none
[19/21] VT-d2: enable interrupt remapping for MSI/MSI-x
none
[20/21] xen: HVM MSI passthrough support
none
[21/21] xen: hardcode UC for MMIO addresses in shadow
none
[7/21] memory changes for VT-d
none
[17/21] xen: MSI supprt internal functions
none
[12/21] VT-d2 support queue invalidation
none
[17/21] xen: MSI supprt internal functions
none
[21/21] xen: add hypercall for adding and removing PCI devices
none
[19/21] VT-d2: enable interrupt remapping for MSI/MSI-x
none
[1/4] VT-d: Enhance MTRR/PAT virtualization
none
[2/4] Utilise the GUEST_PAT and HOST_PAT vmcs area for EPT and VT-d enabled both.
none
[3/4] xen, pci: Fix the definition of PCI_PM_CTRL_NO_SOFT_RESET.
none
[4/4] vt-d: workaround for Mobile Series 4 Chipset none

Description Don Dugger 2009-02-05 15:45:18 UTC
Description of patches:

Set of patches that implement VTd (Intel's IOMMU that allows direct assignment of PCI devices to a virtual guest).  These patches represent a backport of code from the xen-unstable tree into the RHEL 5.4 code tree including support for DMA remapping, memory type, qemu passthrough support.

Upstream Status:  All code has been accepted upstream, where appropriate the individual patches show when changeset they were derived from.

Comment 1 Don Dugger 2009-02-05 15:48:08 UTC
Created attachment 331000 [details]
[1/16] add pci definitions and access functions

define standard PCI registers in pci_regs.h

declare PCI access functions, implement architecture-independent PCI
 ones in xen/drivers/pci, and also implement
 architecture-dependent ones for x86.

Upstream Status: Accepted (CS 17441, 17533, 18585)

Comment 2 Don Dugger 2009-02-05 15:50:39 UTC
Created attachment 331001 [details]
Add public header files  

add public iommu.h which define VT-d data structures and functions,
and define some XEN_DOMCTL_xxx for VT-d

Upstream Status: Accepted (CS 15883, 17195, 18627)

Comment 3 Don Dugger 2009-02-05 15:52:19 UTC
Created attachment 331002 [details]
[3/16] system changes

add VT-d structures in acpi for parse VT-d info from acpi table,
define VT-d register range in fixmap, and implement clflush


Upstream Status: Accepted (CS 15883, 15904)

Comment 4 Don Dugger 2009-02-05 15:53:33 UTC
Created attachment 331003 [details]
[4/16] bulk of VTd code

major VT-d code, implement VT-d under xen/drivers/passthrough/vtd/,
do initialization/destroy for VT-d at boot/exit, define irq structures
and functions for VT-d.


Upstream Status: Accepted (CS 15883, 15905, 17195, 17973, 18649)

Comment 5 Don Dugger 2009-02-05 15:54:54 UTC
Created attachment 331004 [details]
[5/16] irq injection changes

implement irq functions for VT-d, intercept interrupts for VT-d
in __do_IRQ_guest, and do EOI particularly.


Upstream Status: Accepted (CS 15903, 16268)

Comment 6 Don Dugger 2009-02-05 15:56:14 UTC
Created attachment 331006 [details]
[6/16] Intercept I/O for assigned device

hook pio for assigned devices, thus improve pio performance.

Upstream Status: Accepted (CS 15904, 17452)

Comment 7 Don Dugger 2009-02-05 15:59:21 UTC
Created attachment 331010 [details]
[7/16] memory map changes

map memory for VT-d page table, map mmio of assigned device in p2m/ept table,
and set UC of these mmio pages.


Upstream Status: Accepted (CS 15916, 17529, 17645)

Comment 8 Don Dugger 2009-02-05 16:00:39 UTC
Created attachment 331011 [details]
[8/16] add domctl interfaces

implement each XEN_DOMCTL_xxx of VT-d,


Upstream Status: Accepted (CS 15916, 16592, 17747)

Comment 9 Don Dugger 2009-02-05 16:02:15 UTC
Created attachment 331013 [details]
[9/16] fix EPT cache attributes

hardcode UC for mmio of assigned devices in ept table similarly to what was
done for p2m.


Upstream Status: Accepted (in a different form)
This patch differs from upstram since it is a simplification, upstream
uses a more invasive change to allow finer grain control through the
MTRR/PAT registers, more flexible but not needed for the current use
cases for VTd.

Comment 10 Don Dugger 2009-02-05 16:03:21 UTC
Created attachment 331017 [details]
[10/16] avoid redundant context mapping

In case DEV_TYPE_PCI in domain_context_mapping(), redundant context
mapping may occur for some devices.
                                                                               
For example, the NIC is 03:00.0, and its parent bridge is: 00:1e.0.
After domain_context_mapping_one() 03:00.0 and 00:1e.0, the
'secbus' is 3 and 'bus' is 0, so will domain_context_mapping_one()
03:00.0 again -- this redundant invocation returns -EINVAL because
we have created the mapping but haven't changed pdev->domain from
Dom0 to a new domain at this time and eventually the
XEN_DOMCTL_assign_device hypercall returns a failure.

This patch detects this case and avoids the redundant invocation.


Upstream Status: Accepted (CS 18964)

Comment 11 Don Dugger 2009-02-05 16:05:23 UTC
Created attachment 331018 [details]
[11/16] fix 32bit build

Add missing targets for the 32-bit build.

Upstream Status: N/A

Comment 12 Don Dugger 2009-02-05 16:06:23 UTC
Created attachment 331019 [details]
[12/16] fix ia64 build

VT-d is not supported on ia64, so just build xen/drivers/passthrough/
on x86.
                                                                               
Add pci configuration code for ia64    

The new upstream xen/pci.h has a name conflict with the modified
linux/pci.h and shows up earlier in the include search path.  Since
the usage of linux/pci.h is limited, rename to linux/linux-pci.h to
avoid duplicate names and cleanup the few users.

Upstream Status: N/A

Comment 13 Don Dugger 2009-02-05 16:07:33 UTC
Created attachment 331020 [details]
[13/16] emulate accesses to PCI regs cf8/crc

Emulate accesses to PCI window registers cf8/cfc to synchronise
with accesses by the hypervisor itself. All users of cf8/cfc go
through new access functions which take the appropriate spinlock.

Dom0 will access PCI window register cf8/cfc, now hypervisor also
accesses PCI configure space using PCI window register cf8/cfc for VT-d.
So it's possible that there is conflict, though the possibility is
very small. This patch emulates accesses to PCI window registers cf8/cfc
to synchronise with accesses by the hypervisor itself, then fixes the
issue.

BTW, there are still many changes on I/O port access between RHEL 5.4
and xen-unstable, the but they are not necessary for VT-d, we didn't
backport them, so you can find two difference between this patch and
xen-unstable: there are no check_guest_io_breakpoint() and
pv_post_outb_hook() in traps.c in RHEL 5.4.

Upstream Status: Accepted (CS 17441, 17465)

Comment 14 Don Dugger 2009-02-05 16:08:39 UTC
Created attachment 331021 [details]
[14/16] support for queue invalidation 

implement queued invalidation, which is a VT-d2 feature that allows
software to invalidate multiple translation at a time.  Enable queued
invalidation method if HW support is detected. Otherwise, register
invalidation method, invalidate one entry at a time, is used.

Upstream Status: Accepted (CS 16775, 17099)

Comment 15 Don Dugger 2009-02-05 16:12:58 UTC
Created attachment 331023 [details]
[15/16] support interrupt remapping

implement interrupt remapping which is a VT-d2 feature.

remap interrupt for assigned device if such HW is detected on VT-d platforms.

For now only support interrupt remapping for IOAPIC. After MSI/MSI-x is
supported in RHEL5.4, interrupt remapping for MSI/MSI-x will be added


Upstream Status: Accepted (CS 17014, 17099)

Comment 16 Don Dugger 2009-02-05 16:14:22 UTC
Created attachment 331024 [details]
[16/16] cleanup VTd2 code to match upstream

Mainly change ordering of struct declarations and definitions, no actual
code differences. Make it consistent with xen-unstable, and pick up fixes
from xen-unstable if needed.

Upstream Status: Accepted

Comment 18 Don Dugger 2009-02-12 20:16:53 UTC
Created attachment 331740 [details]
[1/21] add pci definitions and access funtions

define standard PCI registers in pci_regs.h
                                           
declare PCI access functions, implement architecture-independent PCI
ones in xen/drivers/pci, and also implement                         
architecture-dependent ones for x86.       
                                    
[ia64] The new upstream xen/pci.h has a name conflict with the modified
linux/pci.h and shows up earlier in the include search path.  Since    
the usage of linux/pci.h is limited, rename to linux/linux-pci.h to
avoid duplicate names and cleanup the few users.                   
                                                
Upstream Status: Accepted (CS 17441, 17533, 18585)
                                                  
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 19 Don Dugger 2009-02-12 20:18:09 UTC
Created attachment 331741 [details]
[2/21] add VT-d public header files

add public iommu.h which define VT-d data structures and functions,
and define some XEN_DOMCTL_xxx for VT-d                            
                                       
Upstream Status: Accepted (CS 15883, 17195, 18627)
                                                  
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 20 Don Dugger 2009-02-12 20:19:36 UTC
Created attachment 331742 [details]
[3/21] some system changes for VT-d

add VT-d structures in acpi for parse VT-d info from acpi table,
define VT-d register range in fixmap, and implement clflush     
                                                           

Upstream Status: Accepted (CS 15883, 15904)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 21 Don Dugger 2009-02-12 20:21:18 UTC
Created attachment 331743 [details]
[4/21] add VT-d specific files

major VT-d code, implement VT-d under xen/drivers/passthrough/vtd/,
do initialization/destroy for VT-d at boot/exit, define irq structures
and functions for VT-d.                                               
                       

Upstream Status: Accepted (CS 15883, 15905, 17195, 17973, 18649)
                                                                
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 22 Don Dugger 2009-02-12 20:22:19 UTC
Created attachment 331744 [details]
[5/21] IRQ injection changes for VT-d

implement irq functions for VT-d, intercept interrupts for VT-d
in __do_IRQ_guest, and do EOI particularly.                    
                                           

Upstream Status: Accepted (CS 15903, 16268)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 23 Don Dugger 2009-02-12 20:23:16 UTC
Created attachment 331745 [details]
[6/21] Intercept I/O for assigned device

hook pio for assigned devices, thus improve pio performance.
                                                            

Upstream Status: Accepted (CS 15904, 17452)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 24 Don Dugger 2009-02-12 20:24:24 UTC
Created attachment 331746 [details]
[7/21] memory changes for VT-d

map memory for VT-d page table, map mmio of assigned device in p2m/ept table,
and set UC of these mmio pages.                                              
                               

Upstream Status: Accepted (CS 15916, 17529, 17645)
                                                  
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 25 Don Dugger 2009-02-12 20:37:58 UTC
Created attachment 331748 [details]
[8/21] Add domctl interfaces for VT-d

implement each XEN_DOMCTL_xxx of VT-d,
                                      

Upstream Status: Accepted (CS 15916, 16592, 17747)
                                                  
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 26 Don Dugger 2009-02-12 20:38:53 UTC
Created attachment 331749 [details]
[9/21] Fix EPT for VT-d

hardcode UC for mmio of assigned devices in ept table similarly to what was
done for p2m.                                                              
             

Upstream Status: Accepted (in a different form)
This patch differs from upstram since it is a simplification, upstream
uses a more invasive change to allow finer grain control through the  
MTRR/PAT registers, more flexible but not needed for the current use
cases for VTd.                                                      
              
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 27 Don Dugger 2009-02-12 20:40:42 UTC
Created attachment 331750 [details]
[10/21] vtd: avoid redundant context mapping

In case DEV_TYPE_PCI in domain_context_mapping(), redundant context
mapping may occur for some devices.                                
                                   
For example, the NIC is 03:00.0, and its parent bridge is: 00:1e.0.
After domain_context_mapping_one() 03:00.0 and 00:1e.0, the        
'secbus' is 3 and 'bus' is 0, so will domain_context_mapping_one()
03:00.0 again -- this redundant invocation returns -EINVAL because
we have created the mapping but haven't changed pdev->domain from 
Dom0 to a new domain at this time and eventually the             
XEN_DOMCTL_assign_device hypercall returns a failure.
                                                     
This patch detects this case and avoids the redundant invocation.
                                                                 

Upstream Status: Accepted (CS 18964)
                                    
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 28 Don Dugger 2009-02-12 20:42:48 UTC
Created attachment 331751 [details]
[11/21] emulate accesses to PCI regs cr8/cfc

Emulate accesses to PCI window registers cf8/cfc to synchronise
with accesses by the hypervisor itself. All users of cf8/cfc go
through new access functions which take the appropriate spinlock.
                                                                 
This is backported from xen-unstable (c/s 17441 and 17465). Dom0 will
access PCI window register cf8/cfc, now hypervisor also accesses PCI 
configure space using PCI window register cf8/cfc for VT-d. So it's 
possible that there is conflict, though the possibility is very small.
This patch emulates accesses to PCI window registers cf8/cfc to       
synchronise with accesses by the hypervisor itself, then fixes the
issue.                                                            
      
BTW, there are still many changes on I/O port access between RHEL 5.4
and xen-unstable, the but they are not necessary for VT-d, we didn't 
backport them, so you can find two difference between this patch and
xen-unstable: there are no check_guest_io_breakpoint() and          
pv_post_outb_hook() in traps.c in RHEL 5.4.               
                                           

Upstream Status: Accepted (CS 17441, 17465)

Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 29 Don Dugger 2009-02-12 20:43:59 UTC
Created attachment 331752 [details]
[12/21] VT-d2: support queue invalidation

implement queued invalidation, which is a VT-d2 feature that allows
software to invalidate multiple translation at a time.  Enable queued
invalidation method if HW support is detected. Otherwise, register   
invalidation method, invalidate one entry at a time, is used.     
                                                             

Upstream Status: Accepted (CS 16775, 17099)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 30 Don Dugger 2009-02-12 20:45:07 UTC
Created attachment 331753 [details]
[13/21] VT-d2: support interrupt remapping

implement interrupt remapping which is a VT-d2 feature.
                                                       
remap interrupt for assigned device if such HW is detected on VT-d platforms.
                                                                             
For now only support interrupt remapping for IOAPIC. After MSI/MSI-x is
supported in RHEL5.4, interrupt remapping for MSI/MSI-x will be added  
                                                                     

Upstream Status: Accepted (CS 17014, 17099)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 31 Don Dugger 2009-02-12 20:46:21 UTC
Created attachment 331754 [details]
[14/21] cleanup VTd2 code to match upstream

Mainly change ordering of struct declarations and definitions, no actual
code differences. Make it consistent with xen-unstable, and pick up fixes
from xen-unstable if needed.                                             
                            

Upstream Status: Accepted
                         
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 32 Don Dugger 2009-02-12 20:47:31 UTC
Created attachment 331755 [details]
[15/21] xen: rename evtchn_lock to event_lock

we are to use evtchn_lock to protect passthrough IRQs besides the original
event channel state, so change the naming to reflect this.                
                                                          

Upstream Status: Accepted (CS 18584)
                                    
BZ: 484227
          
Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 33 Don Dugger 2009-02-12 20:48:24 UTC
Created attachment 331756 [details]
[16/21] xen: convert pirq to per-domain

originally, xen pirq is hardwired to the IRQ number of the interrupt in dom0,
however, since passthrough device MSI doesn't have IRQ number in dom0, there 
is no corresponding pirq number in the original logic. If we still want to  
represent MSI as xen pirq, explicit mapping hypercalls should be implemented.
                                                                             
per-domain pirq is the preparation for this pirq mapping, and it also enlarges
available pirq numbers                                                        
                      

Upstream Status: Accepted (CS 17534)
                                    
BZ: 484227
          
Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 34 Don Dugger 2009-02-12 20:49:37 UTC
Created attachment 331757 [details]
[17/21] xen: MSI supprt internal functions

Add MSI and MSI-X helper functions in Xen.
Also add additional fixmap mapping for MSI-X table.
                                                   

Upstream Status: Accepted (CS 17535, 17942)
                                           
BZ: 484227
          
Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 35 Don Dugger 2009-02-12 20:50:30 UTC
Created attachment 331758 [details]
[18/21] xen: MSI support interface

Add MSI and MSI-X helper functions in Xen.
Also add additional fixmap mapping for MSI-X table.
                                                   

Upstream Status: Accepted (CS 17535, 17942)
                                           
BZ: 484227
          
Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 36 Don Dugger 2009-02-12 20:51:32 UTC
Created attachment 331759 [details]
[19/21] VT-d2: enable interrupt remapping for MSI/MSI-x

previous interrupt remapping patch is only remap INTx, due to MSI/MSI-x
were not ready at that time. This patch enables interrupt remapping    
for MSI/MSI-x.                                                     
              
In order to keep consistent with xen-unstable, add iommu_update_ire_from_apic
and iommu_update_ire_from_msi in iommu_ops.                                  
                                           

Upstream Status: Accepted (CS 17739, 18631)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 37 Don Dugger 2009-02-12 20:52:30 UTC
Created attachment 331760 [details]
[20/21] xen: HVM MSI passthrough support

Xen hypervisor support for HVM guest passthrough MSI, add vmsi concept
similar to vioapic delivered irq, add MSI type in bind_pt_irq hypercalls
to allow device model to bind vmsi to msi pirq and the code to inject MSI
to HVM guest.                                                            
             

Upstream Status: Accepted (CS 17537)
                                    
BZ: 484227
          
Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 38 Don Dugger 2009-02-12 20:53:46 UTC
Created attachment 331761 [details]
[21/21] xen: hardcode UC for MMIO addresses in shadow

this is missed in previous memory patch. MMIO addresses has
been already hardcoded UC in ept.                          
                                 
because UC is set in MTRR by default, VT-d works for us without
this patch. But this patch is necessary for architecture cleanness.
With it, VT-d always works regardless of what is set in MTRR by    
default.                                                       
        

Upstream Status: Accepted (CS 15843, 16152)
                                           
BZ: 484227
          
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>   
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 39 Don Dugger 2009-02-18 17:42:17 UTC
Created attachment 332427 [details]
 [7/21] memory changes for VT-d

map memory for VT-d page table, map mmio of assigned device in p2m/ept table,
and set UC of these mmio pages.

Hardcode UC for MMIO addresses in shadow, MMIO addresses have
already been hardcoded UC in ept.

Because UC is set in MTRR by default, VT-d works for us without
this patch. But this patch is necessary for architecture cleanness.
With it, VT-d always works regardless of what is set in MTRR by
default.


Upstream Status: Accepted (CS 15843, 15916, 16152, 17529, 17645)

BZ: 484227

Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 40 Don Dugger 2009-02-18 17:43:56 UTC
Created attachment 332428 [details]
 [17/21] xen: MSI supprt internal functions  

Add MSI and MSI-X helper functions in Xen.
Also add additional fixmap mapping for MSI-X table.
Fixed by kraxel to not use syscall & hypercall vectors.


Upstream Status: Accepted (CS 17535, 17942)

BZ: 484227

Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 41 Don Dugger 2009-04-02 19:47:24 UTC
Created attachment 337888 [details]
[12/21] VT-d2 support queue invalidation

implement queued invalidation, which is a VT-d2 feature that allows
software to invalidate multiple translation at a time.  Enable queued
invalidation method if HW support is detected. Otherwise, register
invalidation method, invalidate one entry at a time, is used.

[Version 2 update: Remove unnecessary global variable from routine
  __iommu_flush_iec. ]

Upstream Status: Accepted (CS 16775, 17099)

BZ: 484227

Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 42 Don Dugger 2009-04-02 19:49:24 UTC
Created attachment 337889 [details]
[17/21] xen: MSI supprt internal functions

Add MSI and MSI-X helper functions in Xen.
Also add additional fixmap mapping for MSI-X table.

[Version 2 update: fix by kraxel, don't use syscall & hypercall vectors]

Upstream Status: Accepted (CS 17535, 17942)

BZ: 484227

Signed-off-by: Qing He <qing.he>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 43 Don Dugger 2009-04-02 19:53:26 UTC
Created attachment 337892 [details]
[21/21] xen: add hypercall for adding and removing PCI devices

The add hypercall will add a new PCI device and register it.  The
remove hypercall will remove the pci_dev strucure for the device.

Comment 44 Don Dugger 2009-04-02 19:55:34 UTC
Created attachment 337893 [details]
[19/21] VT-d2: enable interrupt remapping for MSI/MSI-x

[Version 3 - don't do extraneous `if' test]

previous interrupt remapping patch is only remap INTx, due to MSI/MSI-x
were not ready at that time. This patch enables interrupt remapping for
MSI/MSI-x.

In order to keep consistent with xen-unstable, add
iommu_update_ire_from_apic and iommu_update_ire_from_msi in iommu_ops.

Upstream Status: Accepted (CS 17739, 18631)

BZ: 484227

Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 45 Don Dugger 2009-04-02 20:00:08 UTC
The following is a set of 4 patches to enhance the VTd support by providing better control over the PAT and also to provide a fix for an errata in the Intel Mobile Series 4 chipset.

Comment 46 Don Dugger 2009-04-02 20:02:08 UTC
Created attachment 337895 [details]
[1/4] VT-d: Enhance MTRR/PAT virtualization

VT-d: Enhance MTRR/PAT virtualization when EPT/VT-d both enabled, and 
utilise the snoop control capability of the VT-d engine, also some
cleanup in the VT-d and EPT code.

Difference from Xen-unstable:
1)epte_get_entry_emt() is not in the xen/arch/x86/hvm/mtrr.c file,
  since the file doesn't exist in xen-3.1.
  So I put it in the p2m-ept.c file.                                            2)some justification in the epte_get_entry_emt() is removed now,
  since some of the conditions don't exist in xen-3.1.

Upstream Status: Accepted (CS 19079, 19154, 19165, 19198)

BZ: 484227
          
Signed-off-by: Xiaohui Xin <xiaohui.xin>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 47 Don Dugger 2009-04-02 20:03:26 UTC
Created attachment 337896 [details]
[2/4] Utilise the GUEST_PAT and HOST_PAT vmcs area for EPT and VT-d enabled both.

The trunk includes "disable_intercept_for_msr(MSR_IA32_CR_PAT)"
is different from xen-unstable, since the whole
disable_intercept_for_msr() is substitute with
vmx_disable_intercept_from_msr() and related much to the msr_bitmap.
And at start_vmx() time, the paging mode may not be intialised yet,
so we use vmx_function_table.hap_supported to indicate if the EPT               mode is enabled or not. 
  
Upstream Status: Accepted (CS 19072)

BZ: 484227
          
Signed-off-by: Xiaohui Xin <xiaohui.xin>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 48 Don Dugger 2009-04-02 20:04:37 UTC
Created attachment 337898 [details]
[3/4] xen, pci: Fix the definition of PCI_PM_CTRL_NO_SOFT_RESET.

Minor correcton to use the correct define value


Upstream Status: Accepted
from xen-unstable.hg:
  HG changeset patch
  User Keir Fraser <keir.fraser>                                       Date 1235991666 0
  Node ID 4dac26349877cbca0bc727286d4ddaeb190dff4b
  Parent  c9e9602cc080daef0c25869cdec21b4e323fd6dd

BZ: 484227
          
Signed-off-by: Dexuan Cui <dexuan.cui>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 49 Don Dugger 2009-04-02 20:05:49 UTC
Created attachment 337899 [details]
[4/4] vt-d: workaround for Mobile Series 4 Chipset

Incorporated VT-d workaround for a sighting on Intel Mobile Series 4
chipset found in Linux iommu. The sighting is the chipset is not
reporting write buffer flush capability correctly.

Upstream Status: Accepted (CS 19230)

BZ: 484227
                                                                                Signed-off-by: Allen Kay <allen.m.kay>
Signed-off-by: Weidong Han <weidong.han>
Signed-off-by: Gerd Hoffman <kraxel>
Signed-off-by: Don Dugger <donald.d.dugger>

Comment 52 Don Zickus 2009-04-16 18:39:16 UTC
in kernel-2.6.18-139.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.  However feel free
to provide a comment indicating that this fix has been verified.

Comment 53 John Villalovos 2009-04-28 18:04:21 UTC
*** Bug 493121 has been marked as a duplicate of this bug. ***

Comment 58 Chris Ward 2009-07-03 18:23:40 UTC
~~ Attention - RHEL 5.4 Beta Released! ~~

RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.

Questions can be posted to this bug or your customer or partner representative.

Comment 59 Chris Ward 2009-07-10 19:10:35 UTC
~~ Attention Partners - RHEL 5.4 Snapshot 1 Released! ~~

RHEL 5.4 Snapshot 1 has been released on partners.redhat.com. If you have already reported your test results, you can safely ignore this request. Otherwise, please notice that there should be a fix available now that addresses this particular request. Please test and report back your results here, at your earliest convenience. The RHEL 5.4 exception freeze is quickly approaching.

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Do not flip the bug status to VERIFIED. Instead, please set your Partner ID in the Verified field above if you have successfully verified the resolution of this issue. 

Further questions can be directed to your Red Hat Partner Manager or other appropriate customer representative.

Comment 61 errata-xmlrpc 2009-09-02 08:19:33 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-1243.html