Bug 1384124

Created attachment 1209659 [details]
host passthrough /proc/cpuinfo from host and guest, qemu ps

(In reply to djdumas from comment #0)
> Description of problem:
> Starting a guest with <cpu mode='host-passthrough' and feature
> policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> guest.  
> 
I know we spoke a little bit about this before. On my 7.3 system, my guest sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing Eduardo to confirm]

[1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html

(In reply to Bandan Das from comment #3)
> (In reply to djdumas from comment #0)
> > Description of problem:
> > Starting a guest with <cpu mode='host-passthrough' and feature
> > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> > guest.  
> > 
> I know we spoke a little bit about this before. On my 7.3 system, my guest
> sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. I think this
> is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> Eduardo to confirm]
> 
> [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html

One piece of information I didn't initially supply is that for this workload, host-passthrough performs significantly better than 'exact' even with nonstop_tsc missing.  I've accounted for other cpu flag differences and still cannot match host-passthrough performance. The hope is that passthrough plus nonstop_tsc will result in even better performance since HANA will no longer fall back to system calls.

(In reply to djdumas from comment #4)
> (In reply to Bandan Das from comment #3)
> > (In reply to djdumas from comment #0)
> > > Description of problem:
> > > Starting a guest with <cpu mode='host-passthrough' and feature
> > > policy='require' name='invtsc' does not provide cpu flag nonstop_tsc in the
> > > guest.  
> > > 

Dave, Can you check the qemu command line produced by libvirt?

> > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 

I thought migratable=no was automatic with invtsc.

> > I think this
> > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > Eduardo to confirm]
> > 
> > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> 
> One piece of information I didn't initially supply is that for this
> workload, host-passthrough performs significantly better than 'exact' even
> with nonstop_tsc missing.  I've accounted for other cpu flag differences and
> still cannot match host-passthrough performance. The hope is that
> passthrough plus nonstop_tsc will result in even better performance since
> HANA will no longer fall back to system calls.

Hold that thought. It might be a different problem. 

Adding jtomko who added the libvirt support. I thought -cpu host was required for invtsc, but maybe it's the other way around -  not allowed? Jan?

(In reply to Karen Noel from comment #5)
> 
> Dave, Can you check the qemu command line produced by libvirt?
> 

Hi Karen, see attachment

(In reply to Karen Noel from comment #5)
> (In reply to djdumas from comment #4)
> > (In reply to Bandan Das from comment #3)
> > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> 
> I thought migratable=no was automatic with invtsc.
> 

I would also expect that. Why would the user need to say it twice?

Libvirt does not touch the migratable= option at all.
(And when the invtsc feature was added, it did not support
customizing the features for model='host-passthrough')

> > > I think this
> > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > Eduardo to confirm]
> > > 
> > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > 

That patch was to remove it from the automatically-generated feature list shown in the live XML.
Libvirt does not strip the flag if it was specified in the XML, even the attachment has:
-cpu host,+invtsc,+vme,...

> 
> Hold that thought. It might be a different problem. 
> 
> Adding jtomko who added the libvirt support. I thought -cpu host was
> required for invtsc, but maybe it's the other way around -  not allowed? Jan?

I do not know of any reason why libvirt would need to block it.

(In reply to Ján Tomko from comment #7)
> (In reply to Karen Noel from comment #5)
> > (In reply to djdumas from comment #4)
> > > (In reply to Bandan Das from comment #3)
> > > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> > 
> > I thought migratable=no was automatic with invtsc.
> > 
> 
> I would also expect that. Why would the user need to say it twice?

"-cpu host" runs with migratable=on by default.

> Libvirt does not touch the migratable= option at all.
> (And when the invtsc feature was added, it did not support
> customizing the features for model='host-passthrough')
> 
> > > > I think this
> > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > > Eduardo to confirm]
> > > > 
> > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > > 
> 
> That patch was to remove it from the automatically-generated feature list
> shown in the live XML.
> Libvirt does not strip the flag if it was specified in the XML, even the
> attachment has:
> -cpu host,+invtsc,+vme,...
> 
> > 
> > Hold that thought. It might be a different problem. 
> > 
> > Adding jtomko who added the libvirt support. I thought -cpu host was
> > required for invtsc, but maybe it's the other way around -  not allowed? Jan?
> 
> I do not know of any reason why libvirt would need to block it.

Libvirt doesn't,  but is there a way to specify migratable=no in the guest XML ? I think that is what we want here.

(In reply to Bandan Das from comment #8)
> (In reply to Ján Tomko from comment #7)
> > (In reply to Karen Noel from comment #5)
> > > (In reply to djdumas from comment #4)
> > > > (In reply to Bandan Das from comment #3)
> > > > > I know we spoke a little bit about this before. On my 7.3 system, my guest
> > > > > sees nonstop_tsc if I specify -cpu host,+invtsc,migratable=no. 
> > > 
> > > I thought migratable=no was automatic with invtsc.
> > > 
> > 
> > I would also expect that. Why would the user need to say it twice?
> 
> "-cpu host" runs with migratable=on by default.
> 
> > Libvirt does not touch the migratable= option at all.
> > (And when the invtsc feature was added, it did not support
> > customizing the features for model='host-passthrough')
> > 
> > > > > I think this
> > > > > is a matter of the right XML syntax or missing support in libvirt[1]. [Ccing
> > > > > Eduardo to confirm]
> > > > > 
> > > > > [1] https://www.redhat.com/archives/libvir-list/2014-September/msg01680.html
> > > > 
> > 
> > That patch was to remove it from the automatically-generated feature list
> > shown in the live XML.
> > Libvirt does not strip the flag if it was specified in the XML, even the
> > attachment has:
> > -cpu host,+invtsc,+vme,...
> > 
> > > 
> > > Hold that thought. It might be a different problem. 
> > > 
> > > Adding jtomko who added the libvirt support. I thought -cpu host was
> > > required for invtsc, but maybe it's the other way around -  not allowed? Jan?
> > 
> > I do not know of any reason why libvirt would need to block it.
> 
> Libvirt doesn't,  but is there a way to specify migratable=no in the guest
> XML ? I think that is what we want here.

That's true, invtsc is available only in migratable=off mode. But requiring explicit migratable=on would be different from the behavior of all other CPU models ("-cpu Haswell,+invtsc" works, for example). "-cpu host" could use the "migratable" property to choose which features will be enabled by default on "-cpu host", but not when filtering out features that were explicitly enabled by the user. We can probably address that on QEMU without changing libvirt.

Upstream fix submitted:

From: Eduardo Habkost <ehabkost>
To: qemu-devel
Subject: [PATCH] target-i386: Don't cpu->migratable field when filtering features
Date: Fri, 14 Oct 2016 16:28:14 -0300
Message-Id: <1476473294-11052-1-git-send-email-ehabkost>

Patch in 2.8.0. Move to POST.

Reproduce this issue against qemu-kvm-rhev-2.6.0-29.el7.x86_64.

Steps:
1.Boot rhel7.4 guest with qemu options: -cpu host,+invtsc and check whether nonstop_tsc present in guest.

Results:
Cannot find nonstop_tsc from guest by cat /proc/cpuinfo and qemu prompt:
QEMU 2.6.0 monitor - type 'help' for more information
(qemu) warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]
warning: host doesn't support requested feature: CPUID.80000007H:EDX.invtsc [bit 8]

Verify against qemu-kvm-rhev-2.8.0-6.el7.x86_64, nothing prompt from qemu and nonstop_tsc can be found inside guest

Verified per comment 16

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392