Bug 770285

Summary: cpu-compare fails inside virtualized hosts
Product: Red Hat Enterprise Linux 6 Reporter: Idan Mansano <imansano>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.3CC: abaron, acathrow, bazulay, berrange, clalancette, crobinso, dallan, danken, dougsland, dwmw2, dyasny, dyuan, honzhang, itamar, jdenemar, jforbes, laine, libvirt-maint, mzhan, oramraz, oschreib, owasserm, rwu, veillard, virt-maint, yupzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.0-0rc0.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 07:06:50 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
virsh -r capatablities
none
requested physical host information
none
libvirt startup in guest none

Description Idan Mansano 2011-12-25 16:08:04 UTC
Description of problem:

When adding the ovirt-engine (which runs on vm) as a host for the same system, the host is moving to "Unassigned" state and cannot move to state "Up".
I am getting the following error:

libvirtError: Requested operation is not valid: cannot get host CPU capabilities

Attached a file with the results of: virsh -r capabilities.

Version of packages:
fedora-release-16-1.noarch
libvirt-0.9.6-2.fc16.x86_64
qemu-kvm-0.15.1-3.fc16.x86_64

Comment 1 Idan Mansano 2011-12-25 16:08:43 UTC
Created attachment 549488 [details]
virsh -r capatablities

Comment 2 Dan Kenigsberg 2011-12-25 21:04:12 UTC
Idan, please add the `virsh compare-cpu` command line that causes this libvirt error, and supply information about running guest kernel and host.

Comment 3 Idan Mansano 2011-12-26 08:38:49 UTC
1.xml File:
<cpu match="minimum"><model>pentium</model></cpu>

virsh -r cpu-compare 1.xml:
error: Failed to compare host CPU with 1.xml
error: Requested operation is not valid: cannot get host CPU capabilities

"uname -a" on the guest (the described problematic unassigned host):
Linux idan-fc16-2.eng.lab.tlv.redhat.com 3.1.1-1.fc16.x86_64 #1 SMP Fri Nov 11 21:47:56 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

"uname -a" on the host running this guest:
Linux buri04.eng.lab.tlv.redhat.com 2.6.32-220.el6.x86_64 #1 SMP Wed Nov 9 08:03:13 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

Comment 4 Daniel Berrangé 2012-02-08 12:20:44 UTC
I'm not able to reproduce this problem myself. Can you provide a few more things from the *physical* host:

 - virsh capabilities   
 - /proc/cpuinfo 
 - virsh dumpxml $GUESTNAME
 - /var/log/libvirt/qemu/$GUESTNAME.log

Comment 5 Dan Kenigsberg 2012-02-08 13:41:52 UTC
Created attachment 560262 [details]
requested physical host information

Comment 6 Jiri Denemark 2012-02-08 13:53:26 UTC
Can you also attach debug logs generated while libvirtd starts in the guest?

Comment 7 Dan Kenigsberg 2012-02-08 14:13:26 UTC
Created attachment 560269 [details]
libvirt startup in guest

Comment 8 Jiri Denemark 2012-02-08 14:29:12 UTC
So the error is 2012-02-08 14:10:07.168+0000: 5470: error : x86Decode:1346 : internal error Cannot find suitable CPU model for given data

/proc/cpuinfo in the guest says:
processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 44
model name	: Westmere E56xx/L56xx/X56xx (Nehalem-C)
stepping	: 1
cpu MHz		: 2400.238
cache size	: 4096 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm constant_tsc nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt aes hypervisor lahf_lm
bogomips	: 4800.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

That is the virtual CPU is missing "sep" which is included in most of our CPU model and should really be there since Westmere model includes it as well. Not sure what is masking it off.

Comment 9 Dan Kenigsberg 2012-03-11 21:01:24 UTC
*** Bug 795836 has been marked as a duplicate of this bug. ***

Comment 11 Dan Kenigsberg 2012-05-13 23:11:02 UTC
Orit, libvirt runs the following command line with Fedora 16's qemu-kvm. Any idea why the guest's CPU is missing the "sep" flag?

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -S -M rhel6.2.0 -cpu Westmere -enable-kvm -m 3072 -smp 2,sockets=2,cores=1,threads=1 -name omer-f16 -uuid 4a30208c-baee-4660-8db6-172aa25454af -smbios type=1,manufacturer=Red Hat,product=RHEV Hypervisor,version=6Server-6.2.0.3.el6,serial=30353036-3837-4247-3831-303946353246_78:E7:D1:22:46:E0,uuid=4a30208c-baee-4660-8db6-172aa25454af -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/omer-f16.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2012-02-08T09:34:42,driftfix=slew -no-shutdown -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive file=/rhev/data-center/f79b0b28-c82f-11e0-8739-78e7d1e48c4c/5bab6470-8825-4e3a-b408-ebcde93678b6/images/201699c9-1101-4303-841f-11c2eae191c6/6cd50f97-3cf8-44a0-af84-4686a9db2743,if=none,id=drive-virtio-disk0,format=qcow2,serial=03-841f-11c2eae191c6,cache=none,werror=stop,rerror=stop,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:23:12:49,bus=pci.0,addr=0x3,bootindex=2 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/omer-f16.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -usb -spice port=5923,tls-port=5932,addr=10.35.16.6,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=inputs -k en-us -vga qxl -global qxl-vga.vram_size=67108864

Comment 12 Dan Kenigsberg 2012-05-13 23:13:20 UTC
(In reply to comment #11)
> Orit, libvirt runs the following command line with Fedora 16's qemu-kvm. Any
> idea why the guest's CPU is missing the "sep" flag?

Doh, the /guest/ is Fedora 16. The host is RHEL-6.2.

Comment 13 Orit Wasserman 2012-05-14 07:53:37 UTC
(In reply to comment #11)
> Orit, libvirt runs the following command line with Fedora 16's qemu-kvm. Any
> idea why the guest's CPU is missing the "sep" flag?
> 
> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
> QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -S -M rhel6.2.0 -cpu Westmere
> -enable-kvm -m 3072 -smp 2,sockets=2,cores=1,threads=1 -name omer-f16 -uuid
> 4a30208c-baee-4660-8db6-172aa25454af -smbios type=1,manufacturer=Red
> Hat,product=RHEV
> Hypervisor,version=6Server-6.2.0.3.el6,serial=30353036-3837-4247-3831-303946353246_78:E7:D1:22:46:E0,uuid=4a30208c-baee-4660-8db6-172aa25454af
> -nodefconfig -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/omer-f16.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc
> base=2012-02-08T09:34:42,driftfix=slew -no-shutdown -device
> virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive
> file=/rhev/data-center/f79b0b28-c82f-11e0-8739-78e7d1e48c4c/5bab6470-8825-4e3a-b408-ebcde93678b6/images/201699c9-1101-4303-841f-11c2eae191c6/6cd50f97-3cf8-44a0-af84-4686a9db2743,if=none,id=drive-virtio-disk0,format=qcow2,serial=03-841f-11c2eae191c6,cache=none,werror=stop,rerror=stop,aio=native
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device
> ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev
> tap,fd=27,id=hostnet0,vhost=on,vhostfd=28 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:23:12:49,bus=pci.0,addr=0x3,bootindex=2
> -chardev
> socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/omer-f16.com.redhat.rhevm.vdsm,server,nowait
> -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm
> -chardev spicevmc,id=charchannel1,name=vdagent -device
> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0
> -usb -spice
> port=5923,tls-port=5932,addr=10.35.16.6,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=main,tls-channel=inputs
> -k en-us -vga qxl -global qxl-vga.vram_size=67108864

RHEL6.2 kernel sometimes disable this flag , does the host has the flag ?

Comment 14 Dan Kenigsberg 2012-05-14 13:30:53 UTC
Orit, no "sep" does not show up neither in host nor in guest.

Just verified that the bug exists in RHEL:

guest:
libvirt-0.9.10-14.el6.x86_64

host:
libvirt-0.9.4-23.el6_2.8.x86_64
qemu-kvm-0.12.1.2-2.209.el6_2.4.x86_64
kernel-2.6.32-220.13.1.el6.x86_64

Moving to RHEL.

Comment 16 Dan Kenigsberg 2012-05-14 14:24:02 UTC
(05:14:30 PM) danken: jdenemar: still, I find it odd that cpu-info explodes instead of returning "no match".
(05:15:53 PM) jdenemar: danken: ok, I guess that could be fixed; but I don't see how that could help you... no cpu would match
(05:16:16 PM) danken: jdenemar: it's a guest. I don't have nested virt anyway
(05:16:27 PM) danken: jdenemar: I only want to test libvirt networking

Comment 20 Jiri Denemark 2012-07-12 12:19:10 UTC
When I was originally discussing this issue with Dan, I thought it was logical to return INCOMPATIBLE instead of an error. However, I'm not so sure anymore. Giving the error seems equally logical. Anyway, I sent a trivial patch (I thought it would need to be more complicated) upstream: https://www.redhat.com/archives/libvir-list/2012-July/msg00549.html We'll see what the community says.

Comment 21 Jiri Denemark 2012-07-16 11:44:43 UTC
Fixed upstream with v0.9.13-67-g87c8623:

commit 87c8623161f19c4f37844991deb1477e53c2b310
Author: Jiri Denemark <jdenemar>
Date:   Thu Jul 12 11:49:15 2012 +0200

    qemu: Do not fail virConnectCompareCPU if host CPU is not known
    
    When host CPU could not be properly detected, virConnectCompareCPU will
    just report that any CPU is incompatible with host CPU instead of
    failing.

Comment 23 hongming 2012-08-06 07:50:30 UTC
Hi Jiri

I would like to know what scenario the host CPU could not be properly detected. How do I reproduce it and verify the patch ? Could you give me some guidance. Thanks a lot in advance.

Comment 24 Jiri Denemark 2012-08-06 13:18:55 UTC
An easy reproducer is to run libvirt in a VM:

- pick a RHEL host (I tried with 6.3)
- install a new guest
- start libvirtd inside that guest
- run "virsh capabilities" and check if the XML contains /domain/cpu/model
  element; if not, libvirt failed to detect host CPU and you can also confirm
  it by searching for "internal error Cannot find suitable CPU model for given
  data" error in libvirtd logs

Comment 25 hongming 2012-08-07 07:45:13 UTC
Verify it as follows

Versions
libvirt-0.10.0-0rc0.el6.x86_64


1. Setup one rhel 6 guest.

2. Start the libvirtd inside the guest.

# rpm -q libvirt
libvirt-0.10.0-0rc0.el6.x86_64

# service libvirtd status
libvirtd (pid  2333) is running...

3.# virsh capabilities > capabilities.xml

# cat capabilities.xml
<capabilities>

  <host>
    <uuid>4fce166a-3797-477a-c9aa-69ce716b92f5</uuid>
    <cpu>
      <arch>x86_64</arch>
    </cpu>
  ......

  </host>
</capabilities>

4.# virsh cpu-compare capabilities.xml
CPU described in capabilities.xml is incompatible with host CPU.

Result
It is expected. So move its status to VERIFIED.

Comment 26 errata-xmlrpc 2013-02-21 07:06:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html