RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1451282 - Fail to start a q35 guest with vcpu > 255
Summary: Fail to start a q35 guest with vcpu > 255
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.4
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Ján Tomko
QA Contact: Luyao Huang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-05-16 09:57 UTC by Luyao Huang
Modified: 2017-08-02 01:32 UTC (History)
8 users (show)

Fixed In Version: libvirt-3.2.0-7.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-02 00:08:25 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1289153 1 None None None 2021-01-20 06:05:38 UTC
Red Hat Product Errata RHEA-2017:1846 0 normal SHIPPED_LIVE libvirt bug fix and enhancement update 2017-08-01 18:02:50 UTC

Internal Links: 1289153

Description Luyao Huang 2017-05-16 09:57:47 UTC 
Description of problem:
fail to start a q35 guest with vcpu > 255

Version-Release number of selected component (if applicable):
libvirt-3.2.0-5.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. check guest maxvcpu:
# virsh capabilities
...
      <machine maxCpus='384'>pc-q35-rhel7.4.0</machine>
...

2. start a guest with pc-q35-rhel7.4.0 machine type and vcpu > 255
# virsh dumpxml r7-q35

  <vcpu placement='static' current='5'>256</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-q35-rhel7.4.0'>hvm</type>

3. start guest:

# virsh start r7-q35
error: Failed to start domain r7-q35
error: internal error: qemu unexpectedly closed the monitor: 2017-05-16T09:54:57.074325Z qemu-kvm: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0)
2017-05-16T09:54:57.074761Z qemu-kvm: -chardev pty,id=charredir0: char device redirected to /dev/pts/3 (label charredir0)
2017-05-16T09:54:57.160767Z qemu-kvm: current -smp configuration requires Extended Interrupt Mode enabled. You can add an IOMMU using: -device intel-iommu,intremap=on,eim=on

3.

Actual results:

When start guest which vcpus > 255, need add -device intel-iommu,intremap=on,eim=on, but libvirt didn't support config xml to gen a qemu command line like this

Expected results:

Need support -device intel-iommu,intremap=on,eim=on in guest xml level

Additional info:
Comment 2 Jaroslav Suchanek 2017-05-16 13:42:43 UTC 

*** This bug has been marked as a duplicate of bug 1289153 ***
Comment 3 Ján Tomko 2017-05-17 15:04:57 UTC 
Reopening, since the other bug is private.
Comment 4 Ján Tomko 2017-05-18 09:12:27 UTC 
Upstream patches:
https://www.redhat.com/archives/libvir-list/2017-May/msg00639.html
Comment 5 Ján Tomko 2017-05-26 06:25:10 UTC 
Pushed upstream as:
commit dc61d927589b2b122868e6abea86b73caa682226
Author:     Ján Tomko <jtomko>
CommitDate: 2017-05-26 08:11:25 +0200

    conf: add eim attribute to <iommu><driver>
    
    Add an attribute to control extended interrupt mode.

commit 381e638d81fd13475112ec9ae2cc4fdbe546ed4d
Author:     Ján Tomko <jtomko>
CommitDate: 2017-05-26 08:16:29 +0200

    qemu: format eim on intel-iommu command line
    
    This option turns on extended interrupt mode,
    which allows more than 255 vCPUs.

git describe: v3.3.0-139-g381e638
Comment 8 Luyao Huang 2017-06-12 08:45:26 UTC 
Verify this bug with libvirt-3.2.0-9.el7.x86_64:

1. configure a guest vcpu > 255:

# virsh edit r7-q35
error: unsupported configuration: more than 255 vCPUs require extended interrupt mode enabled on the iommu device
Failed. Try again? [y,n,i,f,?]: 

2. add ioapic and iommu device in xml:

  <vcpu placement='static' current='5'>384</vcpu>
...
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <ioapic driver='qemu'/>
  </features>
...
    <iommu model='intel'>
      <driver intremap='on' eim='on'/>
    </iommu>

3. start guest:

# virsh start r7-q35
Domain r7-q35 started

4. no error and warning in guest log

5. login guest and check guest cpu number:

# lscpu -e
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE
0   0    0      0    0:0:0:0       yes
1   0    1      1    1:1:1:1       yes
2   0    2      2    2:2:2:2       yes
3   0    3      3    3:3:3:3       yes
4   0    4      4    4:4:4:4       yes

6. hot-plug to 384:

# virsh setvcpus r7-q35 384



7. check vcpu in guest:

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                384
On-line CPU(s) list:   0-383
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             384

8. retest with start a guest with 384 current vcpus: pass
Comment 9 errata-xmlrpc 2017-08-02 00:08:25 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Comment 10 errata-xmlrpc 2017-08-02 01:32:35 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1846
Note You need to log in before you can comment on or make changes to this bug.