Bug 1816670

Summary: Support for kvm-hint-dedicated performance hint in libvirt side
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: jiyan <jiyan>
Component: libvirtAssignee: Virtualization Maintenance <virt-maint>
Status: CLOSED ERRATA QA Contact: jiyan <jiyan>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 8.2CC: dyuan, fjin, jdenemar, jiyan, lmen, virt-maint, xuzhang, yalzhang
Target Milestone: rc   
Target Release: 8.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-6.0.0-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-05 09:59:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description jiyan 2020-03-24 13:57:49 UTC
Description of problem:
Support for kvm-hint-dedicated performance hint in libvirt side

Version-Release number of selected component (if applicable):
RHEL-820AV

How reproducible:
100%

Steps to Reproduce:
1. From libvirt.org we can already see the support for kvm-hint-dedicated performance hint

link: https://libvirt.org/formatdomain.html#elementsFeatures

info: hint-dedicated	Allows a guest to enable optimizations when running on dedicated vCPUs	on, off	5.7.0 (QEMU 2.12.0)

2. QEMU side already supports this function
patch from qemu side: https://www.redhat.com/archives/libvir-list/2019-August/msg00362.html

3. We also detect the XML diff for RHEL-8.2.0 fast train - downtreams
--- 6161,6171 ----kvm: hint-dedicated
              <ref name="featurestate"/>
            </element>
          </optional>
+         <optional>
+           <element name="hint-dedicated">
+             <ref name="featurestate"/>
+           </element>
+         </optional>
        </interleave>
      </element>
    </define>


Expected results:
Based on the three points above, libvirt side also need to support this kind of func. (Maybe libvirt already supports, but there is no bug to track it, I just file this bug to track this function.)

Additional info:

Comment 1 jiyan 2020-03-24 14:00:48 UTC
Same situation for stimer feature:

info from libvirt.org:
stimer	Enable SynIC timers, optionally with Direct Mode support	on, off; direct - on,off	1.3.3 (QEMU 2.6), direct mode 5.7.0 (QEMU 4.1)

XML diff for RHEL-8.2.0 fast train
--- 6089,6095 ---- hyperv: stimer
          </optional>
          <optional>
            <element name="stimer">
!             <ref name="stimer"/>
            </element>
          </optional>
          <optional>
***************
*** 5961,5966 ****
--- 6138,6157 ----
      </element>
    </define>
  
+   <!-- Hyper-V stimer features -->
+   <define name="stimer">
+     <interleave>
+       <optional>
+         <ref name="featurestate"/>
+       </optional>
+       <optional>
+         <element name="direct">
+           <ref name="featurestate"/>
+         </element>
+       </optional>
+     </interleave>
+   </define>
+

Comment 2 jiyan 2020-03-24 14:03:40 UTC
We also detect the XML diff about ccf-assist feature for RHEL-8.2.0 fast train

info from libvirt.org
ccf-assist
Configure ccf-assist (Count Cache Flush Assist) availability for pSeries guests. Possible values for the state attribute are on and off. If the attribute is not defined, the hypervisor default will be used. Since 5.9.0 (QEMU/KVM only)

--- 5274,5284 ----hyperv: ccf-assist
            <optional>
              <ref name="msrs"/>
            </optional>
+           <optional>
+             <element name="ccf-assist">
+               <ref name="featurestate"/>
+             </element>
+           </optional>
          </interleave>
        </element>
      </optional>

Comment 6 jiyan 2020-03-27 07:01:45 UTC
The feature "ccf-assist" mentioned comment 2 is available for pSeries guests, so this feature will not be tested or tracked here.

Comment 7 jiyan 2020-03-27 07:18:42 UTC
Verified hint-dedicated kvm feature with libvirt-6.0.0-15.module+el8.2.0+6106+b6345808.x86_64

Version:
libvirt-6.0.0-15.module+el8.2.0+6106+b6345808.x86_64
qemu-kvm-4.2.0-16.module+el8.2.0+6092+4f2391c1.x86_64
kernel-4.18.0-192.el8.x86_64

Steps:
1. Start VM when the state of hint-dedicated is on with host-passthrough cpu model
# virsh domstate vm1
shut off

# virsh dumpxml vm1 --inactive | grep "<kvm" -A2
    <kvm>
      <hint-dedicated state='on'/>
    </kvm>

# virsh start vm1
Domain vm1 started

# virsh dumpxml vm1  | grep "<kvm" -A2
    <kvm>
      <hint-dedicated state='on'/>
    </kvm>

# ps -ef | grep vm1 
-cpu host,kvm-hint-dedicated=on

2. Start VM when the state of hint-dedicated is off with host-passthrough cpu model
# virsh domstate vm1
shut off

# virsh dumpxml vm1 --inactive | grep "<kvm" -A2
    <kvm>
      <hint-dedicated state='off'/>
    </kvm>

# virsh start vm1
Domain vm1 started

# virsh dumpxml vm1 | grep "<kvm" -A2
    <kvm>
      <hint-dedicated state='off'/>
    </kvm>

# ps -ef | grep vm1
-cpu host 

The test result of is hint-dedicated feature as expected.

Comment 8 jiyan 2020-03-27 07:20:15 UTC
Version:
libvirt-6.0.0-15.module+el8.2.0+6106+b6345808.x86_64
qemu-kvm-4.2.0-16.module+el8.2.0+6092+4f2391c1.x86_64
kernel-4.18.0-192.el8.x86_64

Steps:
1. Test stimer hyperv feature as follows -- failed
# virsh domstate vm1 
shut off

# virsh dumpxml vm1 --inactive | grep "<hyperv" -A2
    <hyperv>
      <stimer state='on'/>
    </hyperv>

# virsh start vm1 
error: Failed to start domain vm1
error: internal error: qemu unexpectedly closed the monitor: Hyper-V synthetic timers (hv-stimer) requires Hyper-V clocksources (hv-time)
2020-03-27T07:16:31.469115Z qemu-kvm: kvm_init_vcpu failed: Function not implemented

Hi Jiri 
Could you pls check the scenario above?
Should I use windows guest or is this function not supported?
Thank you in advance. :—)

Comment 9 Jiri Denemark 2020-03-27 13:54:01 UTC
The QEMU error messages means you have to add

  <clock offset='utc'>
    <timer name='hypervclock' present='yes'/>
  </clock>

to the domain XML.

Comment 10 jiyan 2020-03-28 06:00:00 UTC
Jiri, thank for you reply.
It seems that besides the XML in comment 9, "hv-stimer" also requires: hv-vpindex, hv-synic, hv-time.
From https://fossies.org/linux/qemu/docs/hyperv.txt, 3.9 part.

Verified hv-stimer hyperv feature with libvirt-6.0.0-15.module+el8.2.0+6106+b6345808.x86_64.

Version:
libvirt-6.0.0-15.module+el8.2.0+6106+b6345808.x86_64
qemu-kvm-4.2.0-16.module+el8.2.0+6092+4f2391c1.x86_64
kernel-4.18.0-192.el8.x86_64

Steps:
1. Test stimer hyperv feature with state=on.
# virsh domstate vm1 
shut off

# virsh dumpxml vm1 --inactive | grep "<hyperv" -A2
    <hyperv>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
    </hyperv>
  <clock offset='utc'>
    <timer name='hypervclock' present='yes'/>
  </clock>

# virsh start vm1
Domain vm1 started

# virsh dumpxml vm1 | grep "<hyperv" -A2
    <hyperv>
      <vpindex state='on'/>
      <synic state='on'/>
      <stimer state='on'/>
    </hyperv>
  <clock offset='utc'>
    <timer name='hypervclock' present='yes'/>
  </clock>

# ps -ef | grep vm1 | grep utc
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,tsx-ctrl=on,hv-time,hv-vpindex,hv-synic,hv-stimer


2. Test stimer hyperv feature with state=off.
# virsh domstate vm1
shut off

# virsh dumpxml vm1 --inactive | grep "<hyperv" -A2
    <hyperv>
      <stimer state='off'/>
    </hyperv>

# virsh start vm1 
Domain vm1 started

# virsh dumpxml vm1 | grep "<hyperv" -A2
    <hyperv>
      <stimer state='off'/>
    </hyperv>

# ps -ef | grep vm1
-cpu Cascadelake-Server,ss=on,vmx=on,hypervisor=on,tsc-adjust=on,umip=on,pku=on,md-clear=on,stibp=on,arch-capabilities=on,xsaves=on,ibpb=on,amd-ssbd=on,rdctl-no=on,ibrs-all=on,skip-l1dfl-vmentry=on,mds-no=on,tsx-ctrl=on 

All the tests are as expected, move this bug to be verified.

Comment 12 errata-xmlrpc 2020-05-05 09:59:00 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2017