Red Hat Bugzilla – Bug 1384124
cpu flag nonstop_tsc is not present in guest with host-passthrough and feature policy require invtsc
Last modified: 2017-08-01 23:32:30 EDT
Description of problem: Starting a guest with <cpu mode='host-passthrough' and feature policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the guest. Using <cpu mode='custom' match='exact'> and requiring invtsc on the same system does give nonstop_tsc in the guest. SAP HANA tests for this flag in the guest and if not found, reverts to use of system calls and performance is impacted. Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64 qemu-kvm-common-rhev-2.3.0-31.el7_2.21.x86_64 qemu-kvm-tools-rhev-2.3.0-31.el7_2.21.x86_64 qemu-img-rhev-2.3.0-31.el7_2.21.x86_64 How reproducible: always Steps to Reproduce: 1.Create a RHEL7.2 guest on a host that has CPU flag nonstop_tsc 2.Use host passthrough and require invtsc as described above 3.Start guest, check cpu flags for nonstop_tsc Actual results: no nonstop_tsc cpu flag in guest Expected results: nonstop_tsc cpu flag in guest Additional info:
Created attachment 1209659 [details] host passthrough /proc/cpuinfo from host and guest, qemu ps
(In reply to djdumas from comment #0) > Description of problem: > Starting a guest with <cpu mode='host-passthrough' and feature > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the > guest. > I know we spoke a little bit about this before. On my 7.3 system, my guest sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing Eduardo to confirm] [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
(In reply to Bandan Das from comment #3) > (In reply to djdumas from comment #0) > > Description of problem: > > Starting a guest with <cpu mode='host-passthrough' and feature > > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the > > guest. > > > I know we spoke a little bit about this before. On my 7.3 system, my guest > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing > Eduardo to confirm] > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html One piece of information I didn't initially supply is that for this workload, host-passthrough performs significantly better than 'exact' even with nonstop_tsc missing. I've accounted for other cpu flag differences and still cannot match host-passthrough performance. The hope is that passthrough plus nonstop_tsc will result in even better performance since HANA will no longer fall back to system calls.
(In reply to djdumas from comment #4) > (In reply to Bandan Das from comment #3) > > (In reply to djdumas from comment #0) > > > Description of problem: > > > Starting a guest with <cpu mode='host-passthrough' and feature > > > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the > > > guest. > > > Dave, Can you check the qemu command line produced by libvirt? > > I know we spoke a little bit about this before. On my 7.3 system, my guest > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I thought migratable=no was automatic with invtsc. > > I think this > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing > > Eduardo to confirm] > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html > > One piece of information I didn't initially supply is that for this > workload, host-passthrough performs significantly better than 'exact' even > with nonstop_tsc missing. I've accounted for other cpu flag differences and > still cannot match host-passthrough performance. The hope is that > passthrough plus nonstop_tsc will result in even better performance since > HANA will no longer fall back to system calls. Hold that thought. It might be a different problem. Adding jtomko who added the libvirt support. I thought -cpu host was required for invtsc, but maybe it's the other way around - not allowed? Jan?
(In reply to Karen Noel from comment #5) > > Dave, Can you check the qemu command line produced by libvirt? > Hi Karen, see attachment
(In reply to Karen Noel from comment #5) > (In reply to djdumas from comment #4) > > (In reply to Bandan Das from comment #3) > > > I know we spoke a little bit about this before. On my 7.3 system, my guest > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. > > I thought migratable=no was automatic with invtsc. > I would also expect that. Why would the user need to say it twice? Libvirt does not touch the migratable= option at all. (And when the invtsc feature was added, it did not support customizing the features for model='host-passthrough') > > > I think this > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing > > > Eduardo to confirm] > > > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html > > That patch was to remove it from the automatically-generated feature list shown in the live XML. Libvirt does not strip the flag if it was specified in the XML, even the attachment has: -cpu host,+invtsc,+vme,... > > Hold that thought. It might be a different problem. > > Adding jtomko who added the libvirt support. I thought -cpu host was > required for invtsc, but maybe it's the other way around - not allowed? Jan? I do not know of any reason why libvirt would need to block it.
(In reply to Ján Tomko from comment #7) > (In reply to Karen Noel from comment #5) > > (In reply to djdumas from comment #4) > > > (In reply to Bandan Das from comment #3) > > > > I know we spoke a little bit about this before. On my 7.3 system, my guest > > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. > > > > I thought migratable=no was automatic with invtsc. > > > > I would also expect that. Why would the user need to say it twice? "-cpu host" runs with migratable=on by default. > Libvirt does not touch the migratable= option at all. > (And when the invtsc feature was added, it did not support > customizing the features for model='host-passthrough') > > > > > I think this > > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing > > > > Eduardo to confirm] > > > > > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html > > > > > That patch was to remove it from the automatically-generated feature list > shown in the live XML. > Libvirt does not strip the flag if it was specified in the XML, even the > attachment has: > -cpu host,+invtsc,+vme,... > > > > > Hold that thought. It might be a different problem. > > > > Adding jtomko who added the libvirt support. I thought -cpu host was > > required for invtsc, but maybe it's the other way around - not allowed? Jan? > > I do not know of any reason why libvirt would need to block it. Libvirt doesn't, but is there a way to specify migratable=no in the guest XML ? I think that is what we want here.
(In reply to Bandan Das from comment #8) > (In reply to Ján Tomko from comment #7) > > (In reply to Karen Noel from comment #5) > > > (In reply to djdumas from comment #4) > > > > (In reply to Bandan Das from comment #3) > > > > > I know we spoke a little bit about this before. On my 7.3 system, my guest > > > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. > > > > > > I thought migratable=no was automatic with invtsc. > > > > > > > I would also expect that. Why would the user need to say it twice? > > "-cpu host" runs with migratable=on by default. > > > Libvirt does not touch the migratable= option at all. > > (And when the invtsc feature was added, it did not support > > customizing the features for model='host-passthrough') > > > > > > > I think this > > > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing > > > > > Eduardo to confirm] > > > > > > > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html > > > > > > > > That patch was to remove it from the automatically-generated feature list > > shown in the live XML. > > Libvirt does not strip the flag if it was specified in the XML, even the > > attachment has: > > -cpu host,+invtsc,+vme,... > > > > > > > > Hold that thought. It might be a different problem. > > > > > > Adding jtomko who added the libvirt support. I thought -cpu host was > > > required for invtsc, but maybe it's the other way around - not allowed? Jan? > > > > I do not know of any reason why libvirt would need to block it. > > Libvirt doesn't, but is there a way to specify migratable=no in the guest > XML ? I think that is what we want here. That's true, invtsc is available only in migratable=off mode. But requiring explicit migratable=on would be different from the behavior of all other CPU models ("-cpu Haswell,+invtsc" works, for example). "-cpu host" could use the "migratable" property to choose which features will be enabled by default on "-cpu host", but not when filtering out features that were explicitly enabled by the user. We can probably address that on QEMU without changing libvirt.
Upstream fix submitted: From: Eduardo Habkost <ehabkost@redhat.com> To: qemu-devel@nongnu.org Subject: [PATCH] target-i386: Don't cpu->migratable field when filtering features Date: Fri, 14 Oct 2016 16:28:14 -0300 Message-Id: <1476473294-11052-1-git-send-email-ehabkost@redhat.com>
Patch in 2.8.0. Move to POST.
Reproduce this issue against qemu-kvm-rhev-2.6.0-29.el7.x86_64. Steps: 1.Boot rhel7.4 guest with qemu options: -cpu host,+invtsc and check whether nonstop_tsc present in guest. Results: Cannot find nonstop_tsc from guest by cat /proc/cpuinfo and qemu prompt: QEMU 2.6.0 monitor - type 'help' for more information (qemu) warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8] warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8] warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8] warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8] Verify against qemu-kvm-rhev-2.8.0-6.el7.x86_64, nothing prompt from qemu and nonstop_tsc can be found inside guest
Verified per comment 16
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392