RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1384124 - cpu flag nonstop_tsc is not present in guest with host-passthrough and feature policy require invtsc
Summary: cpu flag nonstop_tsc is not present in guest with host-passthrough and featur...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.2
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: 7.2
Assignee: Eduardo Habkost
QA Contact: Guo, Zhiyi
URL:
Whiteboard:
Depends On:
Blocks: 1018952 1413897 1446211
TreeView+ depends on / blocked
 
Reported: 2016-10-12 15:21 UTC by djdumas
Modified: 2017-08-02 03:32 UTC (History)
21 users (show)

Fixed In Version: QEMU 2.8.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1413897 (view as bug list)
Environment:
Last Closed: 2017-08-01 23:37:14 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
host passthrough /proc/cpuinfo from host and guest, qemu ps (deleted)
2016-10-12 15:21 UTC, djdumas 		no flags Details
host passthrough /proc/cpuinfo from host and guest, qemu ps (5.35 KB, text/plain)
2016-10-12 15:24 UTC, djdumas 		no flags Details
Show Obsolete (1) View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:2392 0 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2017-08-01 20:04:36 UTC

Description djdumas 2016-10-12 15:21:26 UTC 
Description of problem:
Starting a guest with <cpu mode='host-passthrough' and feature policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the guest.  

Using <cpu mode='custom' match='exact'> and requiring invtsc on the same system does give nonstop_tsc in the guest.

SAP HANA tests for this flag in the guest and if not found, reverts to use of system calls and performance is impacted.

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.3.0-31.el7_2.21.x86_64
qemu-kvm-common-rhev-2.3.0-31.el7_2.21.x86_64
qemu-kvm-tools-rhev-2.3.0-31.el7_2.21.x86_64
qemu-img-rhev-2.3.0-31.el7_2.21.x86_64


How reproducible:
always

Steps to Reproduce:
1.Create a RHEL7.2 guest on a host that has CPU flag nonstop_tsc
2.Use host passthrough and require invtsc as described above
3.Start guest, check cpu flags for nonstop_tsc

Actual results:
no nonstop_tsc cpu flag in guest

Expected results:
nonstop_tsc cpu flag in guest

Additional info:
Comment 1 djdumas 2016-10-12 15:24:28 UTC 
Created attachment 1209659 [details]
host passthrough /proc/cpuinfo from host and guest, qemu ps
Comment 3 Bandan Das 2016-10-12 18:43:46 UTC 
(In reply to djdumas from comment #0)
> Description of problem:
> Starting a guest with <cpu mode='host-passthrough' and feature
> policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> guest.  
> 
I know we spoke a little bit about this before. On my 7.3 system, my guest sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing Eduardo to confirm]

[1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
Comment 4 djdumas 2016-10-13 15:09:49 UTC 
(In reply to Bandan Das from comment #3)
> (In reply to djdumas from comment #0)
> > Description of problem:
> > Starting a guest with <cpu mode='host-passthrough' and feature
> > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> > guest.  
> > 
> I know we spoke a little bit about this before. On my 7.3 system, my guest
> sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this
> is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> Eduardo to confirm]
> 
> [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html

One piece of information I didn't initially supply is that for this workload, host-passthrough performs significantly better than 'exact' even with nonstop_tsc missing.  I've accounted for other cpu flag differences and still cannot match host-passthrough performance. The hope is that passthrough plus nonstop_tsc will result in even better performance since HANA will no longer fall back to system calls.
Comment 5 Karen Noel 2016-10-13 17:04:01 UTC 
(In reply to djdumas from comment #4)
> (In reply to Bandan Das from comment #3)
> > (In reply to djdumas from comment #0)
> > > Description of problem:
> > > Starting a guest with <cpu mode='host-passthrough' and feature
> > > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> > > guest.  
> > > 

Dave, Can you check the qemu command line produced by libvirt?

> > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 

I thought migratable=no was automatic with invtsc.

> > I think this
> > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > Eduardo to confirm]
> > 
> > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> 
> One piece of information I didn't initially supply is that for this
> workload, host-passthrough performs significantly better than 'exact' even
> with nonstop_tsc missing.  I've accounted for other cpu flag differences and
> still cannot match host-passthrough performance. The hope is that
> passthrough plus nonstop_tsc will result in even better performance since
> HANA will no longer fall back to system calls.

Hold that thought. It might be a different problem. 

Adding jtomko who added the libvirt support. I thought -cpu host was required for invtsc, but maybe it's the other way around -  not allowed? Jan?
Comment 6 djdumas 2016-10-13 17:19:19 UTC 
(In reply to Karen Noel from comment #5)
> 
> Dave, Can you check the qemu command line produced by libvirt?
> 

Hi Karen, see attachment
Comment 7 Ján Tomko 2016-10-14 07:51:12 UTC 
(In reply to Karen Noel from comment #5)
> (In reply to djdumas from comment #4)
> > (In reply to Bandan Das from comment #3)
> > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> 
> I thought migratable=no was automatic with invtsc.
> 

I would also expect that. Why would the user need to say it twice?

Libvirt does not touch the migratable= option at all.
(And when the invtsc feature was added, it did not support
customizing the features for model='host-passthrough')

> > > I think this
> > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > Eduardo to confirm]
> > > 
> > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > 

That patch was to remove it from the automatically-generated feature list shown in the live XML.
Libvirt does not strip the flag if it was specified in the XML, even the attachment has:
-cpu host,+invtsc,+vme,...

> 
> Hold that thought. It might be a different problem. 
> 
> Adding jtomko who added the libvirt support. I thought -cpu host was
> required for invtsc, but maybe it's the other way around -  not allowed? Jan?

I do not know of any reason why libvirt would need to block it.
Comment 8 Bandan Das 2016-10-14 17:17:41 UTC 
(In reply to Ján Tomko from comment #7)
> (In reply to Karen Noel from comment #5)
> > (In reply to djdumas from comment #4)
> > > (In reply to Bandan Das from comment #3)
> > > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> > 
> > I thought migratable=no was automatic with invtsc.
> > 
> 
> I would also expect that. Why would the user need to say it twice?

"-cpu host" runs with migratable=on by default.

> Libvirt does not touch the migratable= option at all.
> (And when the invtsc feature was added, it did not support
> customizing the features for model='host-passthrough')
> 
> > > > I think this
> > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > > Eduardo to confirm]
> > > > 
> > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > > 
> 
> That patch was to remove it from the automatically-generated feature list
> shown in the live XML.
> Libvirt does not strip the flag if it was specified in the XML, even the
> attachment has:
> -cpu host,+invtsc,+vme,...
> 
> > 
> > Hold that thought. It might be a different problem. 
> > 
> > Adding jtomko who added the libvirt support. I thought -cpu host was
> > required for invtsc, but maybe it's the other way around -  not allowed? Jan?
> 
> I do not know of any reason why libvirt would need to block it.

Libvirt doesn't,  but is there a way to specify migratable=no in the guest XML ? I think that is what we want here.
Comment 9 Eduardo Habkost 2016-10-14 18:40:54 UTC 
(In reply to Bandan Das from comment #8)
> (In reply to Ján Tomko from comment #7)
> > (In reply to Karen Noel from comment #5)
> > > (In reply to djdumas from comment #4)
> > > > (In reply to Bandan Das from comment #3)
> > > > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> > > 
> > > I thought migratable=no was automatic with invtsc.
> > > 
> > 
> > I would also expect that. Why would the user need to say it twice?
> 
> "-cpu host" runs with migratable=on by default.
> 
> > Libvirt does not touch the migratable= option at all.
> > (And when the invtsc feature was added, it did not support
> > customizing the features for model='host-passthrough')
> > 
> > > > > I think this
> > > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > > > Eduardo to confirm]
> > > > > 
> > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > > > 
> > 
> > That patch was to remove it from the automatically-generated feature list
> > shown in the live XML.
> > Libvirt does not strip the flag if it was specified in the XML, even the
> > attachment has:
> > -cpu host,+invtsc,+vme,...
> > 
> > > 
> > > Hold that thought. It might be a different problem. 
> > > 
> > > Adding jtomko who added the libvirt support. I thought -cpu host was
> > > required for invtsc, but maybe it's the other way around -  not allowed? Jan?
> > 
> > I do not know of any reason why libvirt would need to block it.
> 
> Libvirt doesn't,  but is there a way to specify migratable=no in the guest
> XML ? I think that is what we want here.

That's true, invtsc is available only in migratable=off mode. But requiring explicit migratable=on would be different from the behavior of all other CPU models ("-cpu Haswell,+invtsc" works, for example). "-cpu host" could use the "migratable" property to choose which features will be enabled by default on "-cpu host", but not when filtering out features that were explicitly enabled by the user. We can probably address that on QEMU without changing libvirt.
Comment 10 Eduardo Habkost 2016-10-14 19:29:56 UTC 
Upstream fix submitted:

From: Eduardo Habkost <ehabkost>
To: qemu-devel
Subject: [PATCH] target-i386: Don't cpu->migratable field when filtering features
Date: Fri, 14 Oct 2016 16:28:14 -0300
Message-Id: <1476473294-11052-1-git-send-email-ehabkost>
Comment 11 Karen Noel 2016-12-16 17:03:47 UTC 
Patch in 2.8.0. Move to POST.
Comment 16 Guo, Zhiyi 2017-03-16 08:47:18 UTC 
Reproduce this issue against qemu-kvm-rhev-2.6.0-29.el7.x86_64.

Steps:
1.Boot rhel7.4 guest with qemu options: -cpu host,+invtsc and check whether nonstop_tsc present in guest.

Results:
Cannot find nonstop_tsc from guest by cat /proc/cpuinfo and qemu prompt:
QEMU 2.6.0 monitor - type 'help' for more information
(qemu) warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]

Verify against qemu-kvm-rhev-2.8.0-6.el7.x86_64, nothing prompt from qemu and nonstop_tsc can be found inside guest
Comment 17 Guo, Zhiyi 2017-03-16 08:48:14 UTC 
Verified per comment 16
Comment 19 errata-xmlrpc 2017-08-01 23:37:14 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 20 errata-xmlrpc 2017-08-02 01:14:54 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 21 errata-xmlrpc 2017-08-02 02:06:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 22 errata-xmlrpc 2017-08-02 02:47:39 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 23 errata-xmlrpc 2017-08-02 03:12:20 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Comment 24 errata-xmlrpc 2017-08-02 03:32:30 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392
Note You need to log in before you can comment on or make changes to this bug.