Bug 784836

Vadim, why do you expose these features as CPU flags? Wouldn't normal qemu options work better?

Hi Jiri,

Yes, it probably looks strange, but Hyper-V advertise itself through CPUID leafs. Actually, it was the reason, why hyper-v features were exposed as CPU flags.     

Best regards,
Vadim.

*** Bug 829843 has been marked as a duplicate of this bug. ***

Is this the final version (is there a possibility that the design will change)?

IIUC, these flags will be specified for example as: core2duo,+hv_vapic,+hv_spinlocks=0x2fff

Are there any limitations that we should check for (except the minimum of spinlocks)?

Thanks,
Martin

Upstream patch(es) proposed:
https://www.redhat.com/archives/libvir-list/2013-May/msg00173.html

Support for hv_vapic and hv_spinlocks is now commited upstream:
commit 19f75d5eeb6bedd49597034832284146c7591a00
Author:     Ján Tomko <jtomko>
AuthorDate: 2013-06-21 12:22:18 +0200
Commit:     Ján Tomko <jtomko>
CommitDate: 2013-06-21 13:24:44 +0200

    qemu: add hv_vapic and hv_spinlocks support
    
    XML:
    <features>
      <hyperv>
        <vapic state='on'/>
        <spinlocks state='on' retries='4096'/>
      </hyperv>
    </features>
    
    results in the following QEMU command line:
    qemu -cpu <cpu_model>,hv_vapic,hv_spinlocks=0x1000
    
    https://bugzilla.redhat.com/show_bug.cgi?id=784836

git describe: v1.0.6-111-g19f75d5 

hv_relaxed has been implemented earlier:
commit 09f10a12be46b1d7d5a8a610388155c22de27831
Author:     Peter Krempa <pkrempa>
AuthorDate: 2012-10-17 14:55:18 +0200
Commit:     Peter Krempa <pkrempa>
CommitDate: 2012-10-18 12:22:50 +0200

    qemu: Add support for HyperV Enlightenment feature "relaxed"
    
    This patch adds QEMU support for the "relaxed" feature implemented by
    previous patch.

git describe: v0.10.2-197-g09f10a1 contains: v1.0.0-rc1~69

pkgs
libvirt-1.1.0-1.el7.x86_64
qemu-kvm-1.5.1-1.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64

steps
1. prepare a windows guest with hyperV features added
# virsh dumpxml kvm-win7-x86_64-qcow2-ide
...
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='4096'/>
    </hyperv>
  </features>
...

2. start domain
# virsh start kvm-win7-x86_64-qcow2-ide
Domain kvm-win7-x86_64-qcow2-ide started

3. check
# ps aux|grep qemu
qemu      9978 49.5  4.5 1763868 1117176 ?     Sl   11:25   2:23 /usr/libexec/qemu-kvm -name kvm-win7-x86_64-qcow2-ide -S -machine pc-i440fx-1.4,accel=kvm,usb=off -cpu qemu64,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid d48eb7d3-a49f-65eb-9f32-a89ce62168bc -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm-win7-x86_64-qcow2-ide.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/var/lib/libvirt/images/kvm-win7-x86_64-qcow2.img,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,fd=29,id=hostnet0,vhost=on,vhostfd=31 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6f:bd:c7,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5901,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5

command line -cpu is as expected.

The guest also can successfully started, so the libvirt part support is done.


Hi Vadim,

When try add the features to RHEL guest like rhel6.4 and rhel7.0, the guest will fail to start. As you mentioned in comment #13, is this ok?

(In reply to Wayne Sun from comment #16)
> pkgs
> libvirt-1.1.0-1.el7.x86_64
> qemu-kvm-1.5.1-1.el7.x86_64
> kernel-3.9.0-0.55.el7.x86_64
> 
> steps
> 1. prepare a windows guest with hyperV features added
> # virsh dumpxml kvm-win7-x86_64-qcow2-ide
> ...
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>     <hyperv>
>       <relaxed state='on'/>
>       <vapic state='on'/>
>       <spinlocks state='on' retries='4096'/>
>     </hyperv>
>   </features>
> ...
> 
> 2. start domain
> # virsh start kvm-win7-x86_64-qcow2-ide
> Domain kvm-win7-x86_64-qcow2-ide started
> 
> 3. check
> # ps aux|grep qemu
> qemu      9978 49.5  4.5 1763868 1117176 ?     Sl   11:25   2:23
> /usr/libexec/qemu-kvm -name kvm-win7-x86_64-qcow2-ide -S -machine
> pc-i440fx-1.4,accel=kvm,usb=off -cpu
> qemu64,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -m 1024 -realtime mlock=off
> -smp 1,sockets=1,cores=1,threads=1 -uuid
> d48eb7d3-a49f-65eb-9f32-a89ce62168bc -no-user-config -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/kvm-win7-x86_64-qcow2-ide.
> monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc
> base=utc -no-shutdown -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive
> file=/var/lib/libvirt/images/kvm-win7-x86_64-qcow2.img,if=none,id=drive-ide0-
> 0-0,format=qcow2,cache=none -device
> ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev
> tap,fd=29,id=hostnet0,vhost=on,vhostfd=31 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6f:bd:c7,bus=pci.0,
> addr=0x3 -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -chardev
> spicevmc,id=charchannel0,name=vdagent -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,
> name=com.redhat.spice.0 -spice
> port=5901,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x2
> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
> 
> command line -cpu is as expected.
> 
> The guest also can successfully started, so the libvirt part support is done.
> 
> 
> Hi Vadim,
> 
> When try add the features to RHEL guest like rhel6.4 and rhel7.0, the guest
> will fail to start. As you mentioned in comment #13, is this ok?

Hi Wayne,
I'm afraid, that after adding Hyper-V support provided by Microsoft to Linux (RHEL6.4 and forward) the assumption that I've maid in c#13 is not valid anymore. Windows guests can run on top of "conformant" hypervisor (a hypervisor which has a very limited number of features). But it seems to be not to be working for Linux guests. 

Best regards,
Vadim.

(In reply to Vadim Rozenfeld from comment #17)

> 
> Hi Wayne,
> I'm afraid, that after adding Hyper-V support provided by Microsoft to Linux
> (RHEL6.4 and forward) the assumption that I've maid in c#13 is not valid
> anymore. Windows guests can run on top of "conformant" hypervisor (a
> hypervisor which has a very limited number of features). But it seems to be
> not to be working for Linux guests. 
> 
> Best regards,
> Vadim.

Thanks to clarify this, I think this is enough to verify this bug even though depend on qemu-kvm bug is still assigned. Any problem, pls correct me.

There is a problem when set spinlock off

# virsh dumpxml kvm-win7-x86_64-qcow2-ide
...
    <hyperv>
      <relaxed state='off'/>
      <vapic state='off'/>
      <spinlocks state='off'/>
    </hyperv>
...

# virsh start kvm-win7-x86_64-qcow2-ide
error: Failed to start domain kvm-win7-x86_64-qcow2-ide
error: XML error: missing HyperV spinlock retry count

set spinlocks off should not check retries value and block domain start.

Hi Jan,

should I file a new bug to track this or leave it here and open the bug?

Thanks

Leave it here, please.

I've posted a patch upstream to fix it:
https://www.redhat.com/archives/libvir-list/2013-July/msg00251.html

Now fixed upstream:
commit 2e4dd4107e78dfff5d470602e1d18a2a0a40444b
Author:     Ján Tomko <jtomko>
AuthorDate: 2013-07-04 12:12:57 +0200
Commit:     Ján Tomko <jtomko>
CommitDate: 2013-07-04 18:39:56 +0200

    conf: don't check hyperv spinlock retries if disabled
    
    <hyperv>
      <spinlocks state='off'/>
    </hyperv>
    
    results in:
    error: XML error: missing HyperV spinlock retry count
    
    Don't require retries when state is off and use virXPathUInt
    instead of virXPathString to simplify parsing.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=784836#c19

git describe: v1.1.0-52-g2e4dd41

pkgs:
libvirt-1.1.1-1.el7.x86_64
qemu-kvm-1.5.2-1.el7.x86_64
kernel-3.9.0-0.55.el7.x86_64

steps:
1.
# virsh dumpxml kvm-win7-x86_64-raw-virtio
...
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='off'/>
    </hyperv>
...

2.
# virsh start kvm-win7-x86_64-raw-virtio
Domain kvm-win7-x86_64-raw-virtio started

tried rhel7 guest with the three features on, as mentioned in comment #22 it can start successfully.

I am using Fedora 19, how do I get this patches via YUM? Or should I install a new kernel? Mine is 3.9.5-301.fc19.x86_64. qemu-kvm-1.4.2-5.fc19.x86_64
 and libvirt-daemon-kvm-1.0.5.5-1. Is this even possible?
I tried unsuccessfully to download RHEL7 Beta, but could not find it. I have a license from Red Hat, but it does not seem to work for this purpose.
My virtual machine is Windows 2008 and it eats up far less CPU with  
     <hyperv>
      <relaxed state='on'/>
    </hyperv>

I want to try the other two hyper-v additions.

(In reply to Alves from comment #24)
> I am using Fedora 19, how do I get this patches via YUM? Or should I install
> a new kernel? Mine is 3.9.5-301.fc19.x86_64. qemu-kvm-1.4.2-5.fc19.x86_64
>  and libvirt-daemon-kvm-1.0.5.5-1. Is this even possible?
> I tried unsuccessfully to download RHEL7 Beta, but could not find it. I have
> a license from Red Hat, but it does not seem to work for this purpose.
> My virtual machine is Windows 2008 and it eats up far less CPU with  
>      <hyperv>
>       <relaxed state='on'/>
>     </hyperv>
> 
> I want to try the other two hyper-v additions.

QEMU and kernel in F19 should be new enough, but you'll need a newer libvirt on the host. Check out https://fedoraproject.org/wiki/Virtualization_Preview_Repository

This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.