Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1447612

Summary: Enhance libvirt to present virtual L3 cache info for vcpus
Product: Red Hat Enterprise Linux 7 Reporter: Jaroslav Reznik <jreznik>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Luyao Huang <lhuang>
Severity: urgent Docs Contact:
Priority: high    
Version: 7.4CC: berrange, chayang, coli, dyuan, hhuang, jdenemar, jmelvin, juzhang, kchamart, knoel, michen, mtessun, pbonzini, rbalakri, rkrcmar, salmy, snagar, virt-bugs, virt-maint, xuzhang, yalzhang
Target Milestone: rcKeywords: FutureFeature, Upstream, ZStream
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-2.0.0-10.el7_3.9 Doc Type: Enhancement
Doc Text:
Feature: Level-3 CPU cache can be enabled for virtual CPUs using <cache level'3' mode='emulate'/> sub-element of the <cpu> element in domain XML. Reason: Reporting an L3 cache in a guest can increase performance in some workloads. Result: Performance in some workloads can significantly increase after an L3 CPU cache is enabled.
Story Points: ---
Clone Of: 1428952 Environment:
Last Closed: 2017-05-25 15:37:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1428952    
Bug Blocks:    

Description Jaroslav Reznik 2017-05-03 10:24:21 UTC
This bug has been copied from bug #1428952 and has been proposed
to be backported to 7.3 z-stream (EUS).

Comment 6 Luyao Huang 2017-05-08 10:59:13 UTC
Verify this bug with libvirt-2.0.0-10.el7_3.9.x86_64, qemu-kvm-rhev-2.6.0-28.el7_3.9.x86_64:

S1: l3 + host-model

1. prepare a guest which enable l3 cache:

  <cpu mode='host-model'>
    <model fallback='allow'/>
    <cache level='3' mode='emulate'/>
  </cpu>

2. start guest and recheck guest xml and qemu cmdline:

# virsh start r7
Domain r7 started

# virsh dumpxml r7 |grep -a2 'cache'
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <cache level='3' mode='emulate'/>
  </cpu>
  <clock offset='utc'>

# ps aux|grep qemu
qemu      6106 49.2  1.1 2164540 372040 ?      Sl   06:06   0:26 ..... -cpu Opteron_G5,+vme,+ht,+monitor,+osxsave,+bmi1,+mmxext,+fxsr_opt,+cmp_legacy,+extapic,+cr8legacy,+osvw,+ibs,+skinit,+wdt,+lwp,+tce,+nodeid_msr,+topoext,+perfctr_core,+perfctr_nb,l3-cache=on

3. restart libvirtd and recheck guest:

# service libvirtd restart
Redirecting to /bin/systemctl restart libvirtd.service

# virsh dumpxml r7 |grep -a2 'cache'
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <cache level='3' mode='emulate'/>
  </cpu>
  <clock offset='utc'>

4. login guest and check the cache info:

IN GUEST:

# lscpu |grep cache
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              16384K


S2: disable + host-model

1. modify the guest xml like this:

  <cpu mode='host-model'>
    <model fallback='allow'/>
    <cache mode='disable'/>
  </cpu>

2. start guest and check xml and qemu cmdline:

# virsh start r7
Domain r7 started

# virsh dumpxml r7 |grep -a2 'cache'
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <cache mode='disable'/>
  </cpu>
  <clock offset='utc'>

# ps aux|grep qemu
qemu      6437 13.4  1.1 2148144 373056 ?      Sl   06:13   0:27 ...... -cpu Opteron_G5,+vme,+ht,+monitor,+osxsave,+bmi1,+mmxext,+fxsr_opt,+cmp_legacy,+extapic,+cr8legacy,+osvw,+ibs,+skinit,+wdt,+lwp,+tce,+nodeid_msr,+topoext,+perfctr_core,+perfctr_nb


S3: host-passthrough + l3

1. modify the guest xml like this:
# virsh dumpxml r7 |grep -a1 'cache'
  <cpu mode='host-passthrough'>
    <cache level='3' mode='emulate'/>
  </cpu>

2. start guest and check xml and qemu cmdline:

# virsh start r7
Domain r7 started

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu mode='host-passthrough'>
    <cache level='3' mode='emulate'/>
  </cpu>

# ps aux|grep qemu
qemu      6659  104  0.1 2112292 50536 ?       Sl   06:19   0:05 ..... -cpu host,l3-cache=on

3. restart libvirtd and recheck guest:

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu mode='host-passthrough'>
    <cache level='3' mode='emulate'/>
  </cpu>

4. login guest and check the cache info:

IN GUEST:

# lscpu |grep cache
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              16384K


S4: host-passthrough + passthrough

1. modify the guest xml like this:
  <cpu mode='host-passthrough'>
    <cache mode='passthrough'/>
  </cpu>


2. start guest and recheck guest xml & qemu cmdline:

# virsh start r7
Domain r7 started

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu mode='host-passthrough'>
    <cache mode='passthrough'/>
  </cpu>

# ps aux|grep qemu
qemu      6968 15.4  1.1 2147116 379864 ?      Sl   06:25   0:26 ...... -cpu host,host-cache-info=on


3. restart libvirtd and recheck guest:

# service libvirtd restart
Redirecting to /bin/systemctl restart libvirtd.service

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu mode='host-passthrough'>
    <cache mode='passthrough'/>
  </cpu>


4. login guest and check the cache info:

IN GUEST:

# lscpu |grep cache
L1d cache:             16K
L1i cache:             64K
L2 cache:              2048K
L3 cache:              12288K


S5: default cpu mode + l3

1. modify guest xml like this:

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu>
    <cache level='3' mode='emulate'/>
  </cpu>

2. start guest and recheck guest xml and qemu cmdline:

# virsh start r7
Domain r7 started

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu>
    <cache level='3' mode='emulate'/>
  </cpu>

# ps aux|grep qemu
qemu      7285  107  0.1 2114336 49244 ?       Sl   06:37   0:04 ...... -cpu qemu64,l3-cache=on

3. restart libvirtd and recheck:

# virsh dumpxml r7 |grep -a1 'cache'
  <cpu>
    <cache level='3' mode='emulate'/>
  </cpu>

4. login guest:

# lscpu |grep cache
L1d cache:             64K
L1i cache:             64K
L2 cache:              512K
L3 cache:              16384K


S6: some invalid config

1. passthrough + host-model

# virsh edit r7
error: unsupported configuration: CPU cache mode 'passthrough' can only be used with 'host-passthrough' CPUs
Failed. Try again? [y,n,i,f,?]: 

2. cache level 2

# virsh edit r7
error: unsupported configuration: CPU cache mode 'emulate' can only be used with level='3'
Failed. Try again? [y,n,i,f,?]: 

3. cache level 3 + disable

# virsh edit r7
error: unsupported configuration: unsupported CPU cache level for mode 'disable'
Failed. Try again? [y,n,i,f,?]: 

4. old qemu + l3 cache:

# virsh start r7
error: Failed to start domain r7
error: internal error: qemu unexpectedly closed the monitor: 2017-05-08T10:52:08.249316Z qemu-kvm: unrecognized feature l3-cache

5. old qemu + passthrough cache:

# virsh start r7
error: Failed to start domain r7
error: internal error: qemu unexpectedly closed the monitor: 2017-05-08T10:53:12.563395Z qemu-kvm: unrecognized feature host-cache-info

Comment 9 Luyao Huang 2017-05-09 01:46:41 UTC
Move this bug status to verify according to comment 6

Comment 11 errata-xmlrpc 2017-05-25 15:37:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1304