Bug 784836 - new cpu's flags, to control hyper-v related features
Summary: new cpu's flags, to control hyper-v related features
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: rc
: 7.0
Assignee: Ján Tomko
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 829843 (view as bug list)
Depends On: 824639
Blocks: 817925
TreeView+ depends on / blocked
 
Reported: 2012-01-26 11:49 UTC by Vadim Rozenfeld
Modified: 2014-06-18 00:40 UTC (History)
19 users (show)

Fixed In Version: libvirt-1.1.1-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-13 13:30:33 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1056205 None CLOSED new cpu's flags, to control hyper-v related features - hv-time 2019-10-18 08:35:44 UTC
Red Hat Bugzilla 1083525 None None None 2019-10-18 08:35:44 UTC
Red Hat Bugzilla 1083529 None CLOSED [RFE] When running a KVM Windows guest, need to turn on all the "hv_*" flags 2019-10-18 08:35:44 UTC

Internal Links: 1056205 1083525 1083529

Description Vadim Rozenfeld 2012-01-26 11:49:42 UTC
Description of problem:
We are going to introduce three new, hyper-v related, cpu's flags:
- "hv_vapic" to enable virtual APIC feature. VAPIC disabled if this parameter is not specified. 
- "hv_relaxed" to depress some ugly Windows watchdog timer, and enable "relaxed" timing
- "hv_spinlocks=xxx" to enable hyper-v spinlock support and specify recommended number of attempts to retry a spinlock failure before notifying the hypervisor. 
The minimum value for this parameter is 0xfff

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 3 Jiri Denemark 2012-01-27 08:35:19 UTC
Vadim, why do you expose these features as CPU flags? Wouldn't normal qemu options work better?

Comment 4 Vadim Rozenfeld 2012-01-27 13:22:53 UTC
Hi Jiri,

Yes, it probably looks strange, but Hyper-V advertise itself through CPUID leafs. Actually, it was the reason, why hyper-v features were exposed as CPU flags.     

Best regards,
Vadim.

Comment 7 Karen Noel 2012-07-24 17:53:31 UTC
*** Bug 829843 has been marked as a duplicate of this bug. ***

Comment 8 Martin Kletzander 2012-07-31 09:10:00 UTC
Is this the final version (is there a possibility that the design will change)?

IIUC, these flags will be specified for example as: core2duo,+hv_vapic,+hv_spinlocks=0x2fff

Are there any limitations that we should check for (except the minimum of spinlocks)?

Thanks,
Martin

Comment 11 Ján Tomko 2013-05-02 17:09:39 UTC
Upstream patch(es) proposed:
https://www.redhat.com/archives/libvir-list/2013-May/msg00173.html

Comment 15 Ján Tomko 2013-06-21 11:48:10 UTC
Support for hv_vapic and hv_spinlocks is now commited upstream:
commit 19f75d5eeb6bedd49597034832284146c7591a00
Author:     Ján Tomko <jtomko@redhat.com>
AuthorDate: 2013-06-21 12:22:18 +0200
Commit:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2013-06-21 13:24:44 +0200

    qemu: add hv_vapic and hv_spinlocks support
    
    XML:
    <features>
      <hyperv>
        <vapic state='on'/>
        <spinlocks state='on' retries='4096'/>
      </hyperv>
    </features>
    
    results in the following QEMU command line:
    qemu -cpu <cpu_model>,hv_vapic,hv_spinlocks=0x1000
    
    https://bugzilla.redhat.com/show_bug.cgi?id=784836

git describe: v1.0.6-111-g19f75d5 

hv_relaxed has been implemented earlier:
commit 09f10a12be46b1d7d5a8a610388155c22de27831
Author:     Peter Krempa <pkrempa@redhat.com>
AuthorDate: 2012-10-17 14:55:18 +0200
Commit:     Peter Krempa <pkrempa@redhat.com>
CommitDate: 2012-10-18 12:22:50 +0200

    qemu: Add support for HyperV Enlightenment feature "relaxed"
    
    This patch adds QEMU support for the "relaxed" feature implemented by
    previous patch.

git describe: v0.10.2-197-g09f10a1 contains: v1.0.0-rc1~69

Comment 16 Wayne Sun 2013-07-03 03:45:17 UTC
pkgs
libvirt-1.1.0-1.el7.x86_64
qemu-kvm-1.5.1-1.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64

steps
1. prepare a windows guest with hyperV features added
# virsh dumpxml kvm-win7-x86_64-qcow2-ide
...
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='4096'/>
    </hyperv>
  </features>
...

2. start domain
# virsh start kvm-win7-x86_64-qcow2-ide
Domain kvm-win7-x86_64-qcow2-ide started

3. check
# ps aux|grep qemu
qemu      9978 49.5  4.5 1763868 1117176 ?     Sl   11:25   2:23 /usr/libexec/qemu-kvm -name kvm-win7-x86_64-qcow2-ide -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu qemu64,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d48eb7d3-a49f-65eb-9f32-a89ce62168bc -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm-win7-x86_64-qcow2-ide.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/var/lib/libvirt/images/kvm-win7-x86_64-qcow2.img,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=31 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6f:bd:c7,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5901,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

command line -cpu is as expected.

The guest also can successfully started, so the libvirt part support is done.


Hi Vadim,

When try add the features to RHEL guest like rhel6.4 and rhel7.0, the guest will fail to start. As you mentioned in comment #13, is this ok?

Comment 17 Vadim Rozenfeld 2013-07-03 04:07:26 UTC
(In reply to Wayne Sun from comment #16)
> pkgs
> libvirt-1.1.0-1.el7.x86_64
> qemu-kvm-1.5.1-1.el7.x86_64
> kernel-3.9.0-0.55.el7.x86_64
> 
> steps
> 1. prepare a windows guest with hyperV features added
> # virsh dumpxml kvm-win7-x86_64-qcow2-ide
> ...
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>     <hyperv>
>       <relaxed state='on'/>
>       <vapic state='on'/>
>       <spinlocks state='on' retries='4096'/>
>     </hyperv>
>   </features>
> ...
> 
> 2. start domain
> # virsh start kvm-win7-x86_64-qcow2-ide
> Domain kvm-win7-x86_64-qcow2-ide started
> 
> 3. check
> # ps aux|grep qemu
> qemu      9978 49.5  4.5 1763868 1117176 ?     Sl   11:25   2:23
> /usr/libexec/qemu-kvm -name kvm-win7-x86_64-qcow2-ide -S -machine
> pc-i440fx-1.4,accel=kvm,usb=off -cpu
> qemu64,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -m 1024 -realtime mlock=off
> -smp 1,sockets=1,cores=1,threads=1 -uuid
> d48eb7d3-a49f-65eb-9f32-a89ce62168bc -no-user-config -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm-win7-x86_64-qcow2-ide.
> monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc
> base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
> file=/var/lib/libvirt/images/kvm-win7-x86_64-qcow2.img,if=none,id=drive-ide0-
> 0-0,format=qcow2,cache=none -device
> ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev
> tap,fd=29,id=hostnet0,vhost=on,vhostfd=31 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6f:bd:c7,bus=pci.0,
> addr=0x3 -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -chardev
> spicevmc,id=charchannel0,name=vdagent -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,
> name=com.redhat.spice.0 -spice
> port=5901,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2
> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
> 
> command line -cpu is as expected.
> 
> The guest also can successfully started, so the libvirt part support is done.
> 
> 
> Hi Vadim,
> 
> When try add the features to RHEL guest like rhel6.4 and rhel7.0, the guest
> will fail to start. As you mentioned in comment #13, is this ok?

Hi Wayne,
I'm afraid, that after adding Hyper-V support provided by Microsoft to Linux (RHEL6.4 and forward) the assumption that I've maid in c#13 is not valid anymore. Windows guests can run on top of "conformant" hypervisor (a hypervisor which has a very limited number of features). But it seems to be not to be working for Linux guests. 

Best regards,
Vadim.

Comment 18 Wayne Sun 2013-07-03 04:51:27 UTC
(In reply to Vadim Rozenfeld from comment #17)

> 
> Hi Wayne,
> I'm afraid, that after adding Hyper-V support provided by Microsoft to Linux
> (RHEL6.4 and forward) the assumption that I've maid in c#13 is not valid
> anymore. Windows guests can run on top of "conformant" hypervisor (a
> hypervisor which has a very limited number of features). But it seems to be
> not to be working for Linux guests. 
> 
> Best regards,
> Vadim.

Thanks to clarify this, I think this is enough to verify this bug even though depend on qemu-kvm bug is still assigned. Any problem, pls correct me.

Comment 19 Wayne Sun 2013-07-04 06:48:35 UTC
There is a problem when set spinlock off

# virsh dumpxml kvm-win7-x86_64-qcow2-ide
...
    <hyperv>
      <relaxed state='off'/>
      <vapic state='off'/>
      <spinlocks state='off'/>
    </hyperv>
...

# virsh start kvm-win7-x86_64-qcow2-ide
error: Failed to start domain kvm-win7-x86_64-qcow2-ide
error: XML error: missing HyperV spinlock retry count

set spinlocks off should not check retries value and block domain start.

Hi Jan,

should I file a new bug to track this or leave it here and open the bug?

Thanks

Comment 20 Ján Tomko 2013-07-04 11:45:38 UTC
Leave it here, please.

I've posted a patch upstream to fix it:
https://www.redhat.com/archives/libvir-list/2013-July/msg00251.html

Comment 21 Ján Tomko 2013-07-04 16:46:14 UTC
Now fixed upstream:
commit 2e4dd4107e78dfff5d470602e1d18a2a0a40444b
Author:     Ján Tomko <jtomko@redhat.com>
AuthorDate: 2013-07-04 12:12:57 +0200
Commit:     Ján Tomko <jtomko@redhat.com>
CommitDate: 2013-07-04 18:39:56 +0200

    conf: don't check hyperv spinlock retries if disabled
    
    <hyperv>
      <spinlocks state='off'/>
    </hyperv>
    
    results in:
    error: XML error: missing HyperV spinlock retry count
    
    Don't require retries when state is off and use virXPathUInt
    instead of virXPathString to simplify parsing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=784836#c19

git describe: v1.1.0-52-g2e4dd41

Comment 23 Wayne Sun 2013-07-31 07:32:17 UTC
pkgs:
libvirt-1.1.1-1.el7.x86_64
qemu-kvm-1.5.2-1.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64

steps:
1.
# virsh dumpxml kvm-win7-x86_64-raw-virtio
...
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='off'/>
    </hyperv>
...

2.
# virsh start kvm-win7-x86_64-raw-virtio
Domain kvm-win7-x86_64-raw-virtio started

tried rhel7 guest with the three features on, as mentioned in comment #22 it can start successfully.

Comment 24 Alves 2013-08-18 03:27:39 UTC
I am using Fedora 19, how do I get this patches via YUM? Or should I install a new kernel? Mine is 3.9.5-301.fc19.x86_64. qemu-kvm-1.4.2-5.fc19.x86_64
 and libvirt-daemon-kvm-1.0.5.5-1. Is this even possible?
I tried unsuccessfully to download RHEL7 Beta, but could not find it. I have a license from Red Hat, but it does not seem to work for this purpose.
My virtual machine is Windows 2008 and it eats up far less CPU with  
     <hyperv>
      <relaxed state='on'/>
    </hyperv>

I want to try the other two hyper-v additions.

Comment 25 Cole Robinson 2013-08-18 17:19:29 UTC
(In reply to Alves from comment #24)
> I am using Fedora 19, how do I get this patches via YUM? Or should I install
> a new kernel? Mine is 3.9.5-301.fc19.x86_64. qemu-kvm-1.4.2-5.fc19.x86_64
>  and libvirt-daemon-kvm-1.0.5.5-1. Is this even possible?
> I tried unsuccessfully to download RHEL7 Beta, but could not find it. I have
> a license from Red Hat, but it does not seem to work for this purpose.
> My virtual machine is Windows 2008 and it eats up far less CPU with  
>      <hyperv>
>       <relaxed state='on'/>
>     </hyperv>
> 
> I want to try the other two hyper-v additions.

QEMU and kernel in F19 should be new enough, but you'll need a newer libvirt on the host. Check out https://fedoraproject.org/wiki/Virtualization_Preview_Repository

Comment 26 Ludek Smid 2014-06-13 13:30:33 UTC
This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.


Note You need to log in before you can comment on or make changes to this bug.