Bug 1092363

Summary:	[RHEL7] Virsh cmd maxvcpus returns 255 for kvm type, but the maximum number of vcpus supported by kvm is 160.
Product:	Red Hat Enterprise Linux 7	Reporter:	Yang Yang <yanyang>
Component:	libvirt	Assignee:	Ján Tomko <jtomko>
Status:	CLOSED ERRATA	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	low	Docs Contact:
Priority:	low
Version:	7.0	CC:	dyuan, hannsj_uhl, honzhang, jiahu, mzhan, rbalakri, rbian, xuzhang
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	libvirt-1.2.8-3.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1127675 (view as bug list)		Environment:
Last Closed:	2015-03-05 07:34:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yang Yang 2014-04-29 07:29:50 UTC

Description:
 Virsh cmd maxvcpus returns 255 for kvm type, but the maximum number of vcpus supported by kvm is 160

Product Version:
libvirt-1.1.1-29.el7.x86_64

How producible:
Always

Steps:
1. Edit the domain with 255 vcpus
# virsh edit test-queues
 change the vcpu to 255 like <vcpu placement='static'>255</vcpu>
Domain test-queues XML configuration edited.

# virsh dumpxml test-queues | grep domain
<domain type='kvm'>

# virsh dumpxml test-queues | grep vcpu
  <vcpu placement='static'>255</vcpu>

2. Start domain
# virsh start test-queues
error: Failed to start domain test-queues
error: internal error: process exited while connecting to monitor: Warning: Number of SMP cpus requested (255) exceeds the recommended cpus supported by KVM (160)
Number of SMP cpus requested (255) exceeds the maximum cpus supported by KVM (160)

3. # virsh maxvcpus kvm
255

Actual Results:
As above shows

Expected Results:
In step 2, it is said that the maximum cpus supported by KVM (160). However, in step 3, the maxvcpus cpus for kvm printed by virsh is 255.
If the number of vcpu is larger than 160, domain can not be started, so the result of step 3 should be modified to 160.

Comment 1 Hu Jianwei 2014-07-23 09:25:08 UTC

virsh capabilities also can hit this issue.

[root@localhost ~]# virsh capabilities 
<capabilities>

  <host>
    <uuid>00c1fcdb-9fd8-e111-852c-24be051881ce</uuid>
...(clipped)

  <guest>
    <os_type>hvm</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/libexec/qemu-kvm</emulator>
      <machine canonical='pc-i440fx-rhel7.0.0' maxCpus='255'>pc</machine>
      <machine canonical='pc-q35-rhel7.0.0' maxCpus='255'>q35</machine>
      <machine maxCpus='255'>rhel6.0.0</machine>
      <machine maxCpus='255'>rhel6.1.0</machine>
      <machine maxCpus='255'>rhel6.2.0</machine>
      <machine maxCpus='255'>rhel6.3.0</machine>
      <machine maxCpus='255'>rhel6.4.0</machine>
      <machine maxCpus='255'>rhel6.5.0</machine>
      <machine maxCpus='1'>none</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/libexec/qemu-kvm</emulator>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
      <pae/>
      <nonpae/>
    </features>
  </guest>

  <guest>
    <os_type>hvm</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/libexec/qemu-kvm</emulator>
      <machine canonical='pc-i440fx-rhel7.0.0' maxCpus='255'>pc</machine>
      <machine canonical='pc-q35-rhel7.0.0' maxCpus='255'>q35</machine>
      <machine maxCpus='255'>rhel6.0.0</machine>
      <machine maxCpus='255'>rhel6.1.0</machine>
      <machine maxCpus='255'>rhel6.2.0</machine>
      <machine maxCpus='255'>rhel6.3.0</machine>
      <machine maxCpus='255'>rhel6.4.0</machine>
      <machine maxCpus='255'>rhel6.5.0</machine>
      <machine maxCpus='1'>none</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/libexec/qemu-kvm</emulator>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>

</capabilities>

Comment 2 Ján Tomko 2014-07-25 13:13:52 UTC

For 'virsh maxvcpus', libvirt gets the maximum from the kernel:
#ifdef KVM_CAP_MAX_VCPUS
    /* at first try KVM_CAP_MAX_VCPUS to determine the maximum count */
    if ((ret = ioctl(fd, KVM_CHECK_EXTENSION, KVM_CAP_MAX_VCPUS)) > 0)
        goto cleanup;
#endif /* KVM_CAP_MAX_VCPUS */

    /* as a fallback get KVM_CAP_NR_VCPUS (the recommended maximum number of
     * vcpus). Note that on most machines this is set to 160. */
    if ((ret = ioctl(fd, KVM_CHECK_EXTENSION, KVM_CAP_NR_VCPUS)) > 0)
        goto cleanup;

Upstream QEMU only reports a warning when the VCPU number exceeds the recommended number (CAP_NR_VCPUS), but QEMU in RHEL-7.0 treats the recommended number as a maximum.

A RHEL-only patch for libvirt reporting only the NR_VCPUS value will be needed.

The maxCpus values in 'virsh capabilites' come directly from QEMU's 'query-machines' command.

Comment 3 Ján Tomko 2014-09-17 17:02:08 UTC

Downstream-only patch posted:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2014-September/msg00757.html

Comment 5 hongming 2014-09-28 08:25:57 UTC

Verify it as follows. Move its status to VERIFIED.



[root@localhost ~]# uname -r
3.10.0-170.el7.x86_64
[root@localhost ~]# rpm -q libvirt qemu-kvm
libvirt-1.2.8-3.el7.x86_64
qemu-kvm-1.5.3-73.el7.x86_64
[root@localhost ~]# virsh capabilities
<capabilities>

  <host>
    <uuid>00edea21-665c-e211-aa0c-10604b5cb9e7</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>SandyBridge</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='4' threads='2'/>
      <feature name='invtsc'/>
      <feature name='erms'/>
      <feature name='smep'/>
      <feature name='fsgsbase'/>
      <feature name='rdrand'/>
      <feature name='f16c'/>
      <feature name='osxsave'/>
      <feature name='pcid'/>
      <feature name='pdcm'/>
      <feature name='xtpr'/>
      <feature name='tm2'/>
      <feature name='est'/>
      <feature name='smx'/>
      <feature name='vmx'/>
      <feature name='ds_cpl'/>
      <feature name='monitor'/>
      <feature name='dtes64'/>
      <feature name='pbe'/>
      <feature name='tm'/>
      <feature name='ht'/>
      <feature name='ss'/>
      <feature name='acpi'/>
      <feature name='ds'/>
      <feature name='vme'/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
    </cpu>
    <power_management>
      <suspend_mem/>
      <suspend_disk/>
      <suspend_hybrid/>
    </power_management>
    <migration_features>
      <live/>
      <uri_transports>
        <uri_transport>tcp</uri_transport>
      </uri_transports>
    </migration_features>
    <topology>
      <cells num='1'>
        <cell id='0'>
          <memory unit='KiB'>8330192</memory>
          <pages unit='KiB' size='4'>2082548</pages>
          <pages unit='KiB' size='2048'>0</pages>
          <distances>
            <sibling id='0' value='10'/>
          </distances>
          <cpus num='8'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0,4'/>
            <cpu id='1' socket_id='0' core_id='1' siblings='1,5'/>
            <cpu id='2' socket_id='0' core_id='2' siblings='2,6'/>
            <cpu id='3' socket_id='0' core_id='3' siblings='3,7'/>
            <cpu id='4' socket_id='0' core_id='0' siblings='0,4'/>
            <cpu id='5' socket_id='0' core_id='1' siblings='1,5'/>
            <cpu id='6' socket_id='0' core_id='2' siblings='2,6'/>
            <cpu id='7' socket_id='0' core_id='3' siblings='3,7'/>
          </cpus>
        </cell>
      </cells>
    </topology>
    <secmodel>
      <model>selinux</model>
      <doi>0</doi>
      <baselabel type='kvm'>system_u:system_r:svirt_t:s0</baselabel>
      <baselabel type='qemu'>system_u:system_r:svirt_tcg_t:s0</baselabel>
    </secmodel>
    <secmodel>
      <model>dac</model>
      <doi>0</doi>
      <baselabel type='kvm'>+0:+0</baselabel>
      <baselabel type='qemu'>+0:+0</baselabel>
    </secmodel>
  </host>

  <guest>
    <os_type>hvm</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/libexec/qemu-kvm</emulator>
      <machine canonical='pc-i440fx-rhel7.0.0' maxCpus='240'>pc</machine>
      <machine canonical='pc-q35-rhel7.0.0' maxCpus='240'>q35</machine>
      <machine maxCpus='240'>rhel6.0.0</machine>
      <machine maxCpus='240'>rhel6.1.0</machine>
      <machine maxCpus='240'>rhel6.2.0</machine>
      <machine maxCpus='240'>rhel6.3.0</machine>
      <machine maxCpus='240'>rhel6.4.0</machine>
      <machine maxCpus='240'>rhel6.5.0</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/libexec/qemu-kvm</emulator>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='off' toggle='no'/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
      <pae/>
      <nonpae/>
    </features>
  </guest>

  <guest>
    <os_type>hvm</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/libexec/qemu-kvm</emulator>
      <machine canonical='pc-i440fx-rhel7.0.0' maxCpus='240'>pc</machine>
      <machine canonical='pc-q35-rhel7.0.0' maxCpus='240'>q35</machine>
      <machine maxCpus='240'>rhel6.0.0</machine>
      <machine maxCpus='240'>rhel6.1.0</machine>
      <machine maxCpus='240'>rhel6.2.0</machine>
      <machine maxCpus='240'>rhel6.3.0</machine>
      <machine maxCpus='240'>rhel6.4.0</machine>
      <machine maxCpus='240'>rhel6.5.0</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/libexec/qemu-kvm</emulator>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <deviceboot/>
      <disksnapshot default='off' toggle='no'/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>

</capabilities>


[root@localhost ~]# virsh maxvcpus kvm
240


[root@localhost ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     r                              shut off

[root@localhost ~]# virsh edit r
Domain r XML configuration edited.

[root@localhost ~]# virsh dumpxml r|grep vcpu
  <vcpu placement='static'>240</vcpu>
[root@localhost ~]# virsh start r
Domain r started

[root@localhost ~]# virsh destroy r
Domain r destroyed

[root@localhost ~]# virsh edit r
Domain r XML configuration edited.

[root@localhost ~]# virsh dumpxml r|grep vcpu
  <vcpu placement='static'>241</vcpu>
[root@localhost ~]# virsh start r
error: Failed to start domain r
error: unsupported configuration: Maximum CPUs greater than specified machine type limit

Comment 7 errata-xmlrpc 2015-03-05 07:34:57 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html