Bug 739521

Summary:	SRIOV and Intel IOMMU support not present in realtime kernel config
Product:	Red Hat Enterprise MRG	Reporter:	Robert Stonehouse <rstonehouse>
Component:	realtime-kernel	Assignee:	Clark Williams <williams>
Status:	CLOSED NOTABUG	QA Contact:	David Sommerseth <davids>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	2.2	CC:	bhu, cburrell, lgoncalv, ovasik, sghosh, williams
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-01-31 11:41:30 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Robert Stonehouse 2011-09-19 11:43:11 UTC

Description of problem:
=======================
This is more a change request than a bug report.

The following are not enabled in the -rt kernel config (though they are present in the kernel config for rhel6u1)
  1) CONFIG_PCI_IOV (this is for the hardware feature of PCI Single Root IO Virtualisation)
  2) CONFIG_DMAR (this is for DMA Remapping support in Intel IOMMUs; support for AMD is in CONFIG_AMD_IOMMU which is already present)


The reason that I am interested in this is because I work for Solarflare which has a low-latency TCP/IP stack and hardware support called OpenOnload (see www.openonload.org). The next version of OpenOnload will have support for using SR-IOV resource exposed using the hardware, where an SR-IOV Virtual Function can be used directly by an onload TCP/IP stack.


It is my understanding that:
 1) CONFIG_PCI_IOV is purely enabling some further PCI features
 2) CONFIG_DMAR can add some latency to DMA mapping/unmapping operations; but there are existing kernel command line options to disable it.


Many of our customers are running real-time kernels and therefore will need to recompile the kernel to benefit.
Therefore I wondered if there were reasons why these two options had been excluded from the real-time kernel config?
and if there is any way of requesting a change for the next kernel update or next service pack?


Many thanks for your help



How reproducible:
=================
100% by inspection
egrep 'CONFIG_(PCI_IOV|DMAR)' /boot/config-2.6.33.9-rt31.74.el6rt.x86_64

Comment 1 Robert Stonehouse 2011-09-19 15:29:17 UTC

Perhaps I was too hasty with this.

It seems the Kconfig option excludes using this config with a real-time kernel
==============================================
config DMAR
        bool "Support for DMA Remapping Devices (EXPERIMENTAL)"
        depends on PCI_MSI && ACPI && EXPERIMENTAL && !PREEMPT_RT
                                                      ^^^^^^^^^^^
        help
          DMA remapping (DMAR) devices support enables independent address
          translations for Direct Memory Access (DMA) from devices.
          These DMA remapping devices are reported via ACPI tables
          and include PCI device scope covered by these DMA
          remapping devices.
==============================================

Perhaps I should trace where this came from and if it is still the case.

Comment 3 Clark Williams 2012-05-22 18:52:15 UTC

These options:

CONFIG_DMAR_TABLE
CONFIG_PCI_IOV

are turned on in the upcoming MRG 2.2 realtime kernel (3.2.18-rt29+)

We've had the options turned on for quite some time so I don't anticipate any real issues here, at least with the 3.2 kernels.

Comment 4 Robert Stonehouse 2013-01-31 11:41:30 UTC

I can confirm that the Intel IOMMU driver was compiled into the kernel for MRG2.2

During testing we found the following commit was necessary:

(In reply to comment #15)
> commit 606d8a8ad41c307aa763579e50f81c565c210f48
> Author: Alex Williamson <alex.williamson>
> Date:   Fri Nov 11 17:26:44 2011 -0700
> 
>     intel-iommu: Default to non-coherent for domains unattached to iommus
>     
>     commit 2e12bc29fc5a12242d68e11875db3dd58efad9ff upstream.

and we have found this commit was included in 3.2.33-rt50.66.el6rt.x86_64 (perhaps earlier?)


Hence marking as CLOSED (please correct this if I am not meant to change the status)


[And just for my reference this was Solarflare bugzilla 32511/32395]