Bug 1412215 - EPA feature support in OpenStack for Intel Resource Director Technology
Summary: EPA feature support in OpenStack for Intel Resource Director Technology
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 12.0 (Pike)
Hardware: x86_64
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: nova-maint
QA Contact: nova-maint
URL:
Whiteboard:
Depends On: 1299678 1329372 1396152
Blocks: 1476900 1521118
TreeView+ depends on / blocked
 
Reported: 2017-01-11 14:33 UTC by Chegu Vinod
Modified: 2019-11-20 14:17 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-08-28 09:50:28 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Chegu Vinod 2017-01-11 14:33:50 UTC
Certain Telco/NFV use cases do require a much finer grain (i.e. Processor Cache and memory bandwidth) of resource isolation to ensure the best predicitable performance with least interference.

Intel processors starting from Broadwell and beyond are enabling support for monitoring and controlling allocation of processor Cache and Memory bandwidth. 

- Cache Monitoring Technology (CMT)
- Cache Allocation Technology (CAT)
- Memory Bandwidth Monitoring (MBM)
- Memory Bandwidth Allocation (MBA(

Changes have (and are being ) made to the recent Linux kernel (4.4 and beyond) and as applicable to KVM  stack(libvirt etc) too. There have been efforts by Intel and the community to have corresponding EPA feature changes implemented in OpenStack.

Request here is to verify/harden and include the OpenStack related changes for OSP 12 (Pike)

Comment 1 Stephen Gordon 2017-01-24 18:53:48 UTC
(In reply to Chegu Vinod from comment #0)
> Certain Telco/NFV use cases do require a much finer grain (i.e. Processor
> Cache and memory bandwidth) of resource isolation to ensure the best
> predicitable performance with least interference.
> 
> Intel processors starting from Broadwell and beyond are enabling support for
> monitoring and controlling allocation of processor Cache and Memory
> bandwidth. 
> 
> - Cache Monitoring Technology (CMT)
> - Cache Allocation Technology (CAT)
> - Memory Bandwidth Monitoring (MBM)
> - Memory Bandwidth Allocation (MBA(
> 
> Changes have (and are being ) made to the recent Linux kernel (4.4 and
> beyond) and as applicable to KVM  stack(libvirt etc) too. There have been
> efforts by Intel and the community to have corresponding EPA feature changes
> implemented in OpenStack.
> 
> Request here is to verify/harden and include the OpenStack related changes
> for OSP 12 (Pike)

Can you elaborate? Are these existing changes that Intel is driving upstream (if so please provide links) or net new features that Intel is requesting Red Hat drive upstream? If the latter which projects are outstanding in OpenStack?

I'm familiar(ish) with CMT and CAT but this is the first I am hearing of MBM and MBA in the OpenStack context, which concerns me in terms of whether any existing design discussion has started.

Comment 3 Chegu Vinod 2017-02-06 18:56:18 UTC
(In reply to Stephen Gordon from comment #1)
> (In reply to Chegu Vinod from comment #0)
> > Certain Telco/NFV use cases do require a much finer grain (i.e. Processor
> > Cache and memory bandwidth) of resource isolation to ensure the best
> > predicitable performance with least interference.
> > 
> > Intel processors starting from Broadwell and beyond are enabling support for
> > monitoring and controlling allocation of processor Cache and Memory
> > bandwidth. 
> > 
> > - Cache Monitoring Technology (CMT)
> > - Cache Allocation Technology (CAT)
> > - Memory Bandwidth Monitoring (MBM)
> > - Memory Bandwidth Allocation (MBA(
> > 
> > Changes have (and are being ) made to the recent Linux kernel (4.4 and
> > beyond) and as applicable to KVM  stack(libvirt etc) too. There have been
> > efforts by Intel and the community to have corresponding EPA feature changes
> > implemented in OpenStack.
> > 
> > Request here is to verify/harden and include the OpenStack related changes
> > for OSP 12 (Pike)
> 
> Can you elaborate? Are these existing changes that Intel is driving upstream
> (if so please provide links) or net new features that Intel is requesting
> Red Hat drive upstream? If the latter which projects are outstanding in
> OpenStack?
> 
> I'm familiar(ish) with CMT and CAT but this is the first I am hearing of MBM
> and MBA in the OpenStack context, which concerns me in terms of whether any
> existing design discussion has started.

Hello Stephen,  We got this information from Intel a few months ago. We reached out to them again last week to get pointers to the latest...but haven't heard back yet.  Will keep you posted.  Thx, Vinod

Comment 4 Chegu Vinod 2017-02-11 16:33:05 UTC
Hi Stephen, 

Just FYI... Got an update from Intel (pl. see below)

 There are still references to upcoming kernels/libvirt etc in the case of MBM and MBA features.  So I am not sure at this point if the Open Stack changes will be all ready in time for the Pike release.

Thanks
Vinod


•	Cache Monitoring Technology (CMT)
o	Available in Linux kernel (4.4), libvirt (1.3.3) and OpenStack (Newton)
o	CMT was enabled in OpenStack ceilometer, it was documented in the measurements table. Look up the “Meters added in the Newton release section” where you will see the cpu_l3_cache meter.
o	Some work is in progress to write up some easier to consume user guides. 


•	Cache Allocation Technology (CAT)
o	Target is to land it in Linux 4.10, followed by libvirt 1.3.x release. OpenStack target is Pike. 


•	Memory Bandwidth Monitoring (MBM)
o	Target is now to have it available in Linux kernel 4.11. We were hoping to land this in 4.6, but have been required to rework the proposal. The libvirt work was completed in libvirt 1.3.5 and so too was the OpenStack work in Newton. However it is highly likely that both libvirt and OpenStack will need an update to adjust to the Linux enabling. The latest target is OpenStack Pike. 
o	For OpenStack, it was documented in a measurements table. Look up the “Meters added in the Newton release section” where you will see the memory.bandwidth.total and memory.bandwidth.local meters.
o	However, even though it looks like this is usable with OpenStack Newton it is really suitable as a Proof of Concept level right now. To do so, you would need to use the proposed code for Linux 4.6. 


•	Memory Bandwidth Allocation (MBA)
o	Target is to land feature in Linux 4.11, libvirt (TBD), and the OpenStack target is Pike.

Comment 5 Stephen Gordon 2017-02-14 15:56:03 UTC
Thanks for the update, that matches what I was able to figure out as well. I'm reflagging this RFE for re-evaluation come Queens/RHOSP 13 planning.

Comment 6 Chegu Vinod 2017-02-14 15:58:50 UTC
(In reply to Stephen Gordon from comment #5)
> Thanks for the update, that matches what I was able to figure out as well.
> I'm reflagging this RFE for re-evaluation come Queens/RHOSP 13 planning.

Sure.  I was also informed (by Intel) that the proposals around this topic may get discussed during the Boston OpenStack summit.

Comment 9 Stephen Finucane 2017-10-11 15:49:03 UTC
> Request here is to verify/harden and include the OpenStack related changes for OSP 12 (Pike)

Is this expected to happen this release? I was unable to find any upstream changes related to this feature.

Comment 10 Lee Yarwood 2017-12-06 17:57:41 UTC
Moving this to 14, I can't see any evidence of progress here for Queens.

Comment 11 Stephen Finucane 2018-08-28 09:50:28 UTC
This RFE has been open in a NEEDINFO state for many months now. As a result, I'm going to close it. Please reopen if there is additional information you can provide.


Note You need to log in before you can comment on or make changes to this bug.