RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1913808 - virt-install requires manual xml editing for more than 255 vcpus
Summary: virt-install requires manual xml editing for more than 255 vcpus
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: virt-manager
Version: ---
Hardware: x86_64
OS: All
unspecified
medium
Target Milestone: rc
: 8.0
Assignee: Virtualization Maintenance
QA Contact: Hongzhou Liu
URL:
Whiteboard:
Depends On:
Blocks: 1788991
TreeView+ depends on / blocked
 
Reported: 2021-01-07 16:27 UTC by Brian Payton
Modified: 2022-05-10 14:26 UTC (History)
14 users (show)

Fixed In Version: virt-manager-3.2.0-1.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-10 13:54:55 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:
pm-rhel: mirror+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2022:1862 0 None None None 2022-05-10 13:55:41 UTC

Description Brian Payton 2021-01-07 16:27:44 UTC
Description of problem:

The vpu maximum for virt-install is 255.  More than 255 vcpus requires manually editing the resulting vm xml to add the following lines.

virsh edit demo_6TB

  <features>
    <acpi/>
    <apic/>
->    <pae/>
->    <ioapic driver='qemu'/>
  </features>
...
  <devices>
->   <iommu model='intel'>
->      <driver intremap='on' eim='on'/>
->    </iommu>

Version-Release number of selected component (if applicable):


How reproducible: Easy


Steps to Reproduce:
1. Create a vm with 384 vcpus using virt-install
2. Create a vm with 255 vcpus using virt-install
3. Edit the xml as above changing the vcpus to 384

Actual results:

[root@ah-071 ~]# virt-install --name step8b --import --noreboot --noautoconsole --console pty,target_type=serial --graphics none --cpu host-passthrough --network bridge=virbr0 --os-type=linux --os-variant=rhel8.1 --vcpus 384 --memory 8390655 --disk /tmp/step8b.qcow2,format=qcow2

Starting install...
ERROR    unsupported configuration: more than 255 vCPUs require extended interrupt mode enabled on the iommu device
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start step8b
otherwise, please restart your installation.


Expected results:


Additional info:

We cannot find a virt-install command line option to add these lines to the xml.  This would be very helpful, especially since we can run 12TB 512 vms with the manual editing.  We have a problem where we use numa cell and memnode definitions for the 32 socket 12TB of memory.  To use these now we need to first configure vcpu pinning for 255 vcpus across 32 nodes then manually edit the xml for the changes above and a new set of 512 vcpupin statements.

Any comments?

Thank you for your help/

Regards,

Brian

Comment 1 Pavel Hrdina 2021-01-27 13:44:01 UTC
Hi, thanks for the report. The support to configure <iommu> is implemented in virt-manager 3.0.0 so it can be backported:

commit 25419db9caf0e45578e98d85af096fab52a3c7cc
Author: Menno Lageman <menno.lageman>
Date:   Tue Jul 7 09:55:53 2020 +0200

    virtinst: add support for configuring the IOMMU

Once that is done the <iommu> device can be configured using this:

    --iommu model=intel,driver.intremap=on,driver.eim=on

Comment 2 Brian Payton 2021-02-08 19:19:00 UTC
Thank you for the update on the commit.  Any ideas about adding "ioapic driver='qemu'" from the virt-install command line?

Regards,

Brian

Comment 3 Brian Payton 2021-09-15 20:30:57 UTC
We have virt-manager > 3.0.0 install on a system.

#rpm -q virt-install virt-manager
virt-install-3.2.0-655.1.noarch
virt-manager-3.2.0-655.1.noarch

When we tried to use the new --iommu option, the command failed with the following.  Any suggestion to resolve the ERROR?  We cannot find one.

 #  virt-install \
> --name demo_uefi_iommu \
> --disk /tmp/demo_uefi_iommu.qcow2 \
> --import \
> --noreboot \
> --console pty,target_type=serial \
> --graphics none \
> --cpu host-passthrough \
> --network bridge=virbr0 \
> --os-type=linux \
> --os-variant=fedora33 \
> --features apic=on,apic.eoi=on,pae=on \
> --iommu model=intel,driver.intremap=on,driver.eim=on \
> --machine q35 \
> --features smm=on \
> --qemu-commandline='-global isa-debugcon.iobase=0x402 -debugcon file:/tmp/demo_efi_iommu.ovmf.log ' \
> --qemu-commandline='-machine smbios-ep=3-0' \
> --qemu-commandline='-global mch.extended-tseg-mbytes=128' \
> --boot menu=on,uefi,loader.secure='no' \
> --cpu host-passthrough \
> --vcpus 255 \
> --memory 7864320

Starting install...
ERROR    unsupported configuration: IOMMU interrupt remapping requires split I/O APIC (ioapic driver='qemu')
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start demo_uefi_iommu
otherwise, please restart your installation.

      Thank you for your help.  Have a good day.

Regards,

Brian

Comment 4 Pavel Hrdina 2022-01-26 09:00:13 UTC
Moving this BZ to MODIFIED as the original report was fixed by rebase to virt-manager 3.2.0.

The issue for the ioapic driver mentioned in Comment 3 is tracked by different BZ 2037202 that is already fixed as well.

Comment 7 Hongzhou Liu 2022-02-07 07:47:33 UTC
Verify this bug with virt-manager-3.2.0-3.el8.noarch

1. Create a vm with 384 vcpus using virt-install

# virt-install --machine=q35 --noreboot --name=rhel12 --memory=4096 --graphics type=vnc --vcpus=384 --location http://download.eng.pek2.redhat.com/released/rhel-6-7-8/rhel-8/RHEL-8/8.4.0/BaseOS/x86_64/os/ --iommu model=intel,driver.intremap=on,driver.eim=on,driver.caching_mode=on --features  apic=on,ioapic.driver=qemu
Using rhel8.4 default --disk size=20

Result: vm can be installed successfully

check the xml config for vcpu:

# virsh dumpxml rhel12 | grep cpu
  <vcpu placement='static'>384</vcpu>
  <cpu mode='custom' match='exact' check='full'>
  </cpu>


2. Create a vm with 255 vcpus using virt-install

# virt-install --machine=q35 --noreboot --name=rhel11 --memory=4096 --graphics type=vnc --vcpus=255 --location http://download.eng.pek2.redhat.com/released/rhel-6-7-8/rhel-8/RHEL-8/8.4.0/BaseOS/x86_64/os/ --iommu model=intel,driver.intremap=on,driver.eim=on,driver.caching_mode=on --features  apic=on,ioapic.driver=qemu
Using rhel8.4 default --disk size=20

Result: vm can be installed successfully

check the xml config for vcpu:
# virsh dumpxml rhel11 | grep cpu
  <vcpu placement='static'>255</vcpu>
  <cpu mode='custom' match='exact' check='full'>
  </cpu>


3. Edit the xml as above changing the vcpus to 384
Use virsh edit change vcpu form 255 to 384
restart vm 
Chack the xml config for vcpu:
# virsh dumpxml rhel11 | grep vcpu
  <vcpu placement='static'>384</vcpu>

All the results are as expected, I changed the status to verified, Thanks!

Comment 8 Brian Payton 2022-02-07 13:00:53 UTC
Thanks you for the verification.  That looks great!  This is a huge help.  I am excited to test on our large system with 768 vcpus, vcpu pinning and memnodes in the virt-install command.

Have a great day.

Regards,

Brian

Comment 9 Brian Payton 2022-02-28 16:33:00 UTC
Thank you very much for this fix.  With both RHEL 8.6 sand RHELW 9.0 we arae able to create the following from the command line with virt-install.  This is a huge help with our testing and usability.  We found the next opportunity for virt-install are options to define emulatorpin and iothreadpin.

Have a great day.

Brian

virt-install \
--name demo_legacy \
--disk /tmp/demo_legacy.qcow2 \
--import \
--noreboot \
--console pty,target_type=serial \
--graphics none \
--video vga \
--cpu host-passthrough \
--network bridge=virbr0 \
--os-type=linux \
--machine=q35 \
--vcpus sockets=32,cores=11,threads=2 \
--os-variant=fedora34 \
--features  apic=on,ioapic.driver=qemu \
--qemu-commandline='-global isa-debugcon.iobase=0x402 -debugcon file:/tmp/demo_legacy.ovmf.log ' \
--qemu-commandline='-global mch.extended-tseg-mbytes=128' \
--iommu model=intel,driver.intremap=on,driver.eim=on,driver.caching_mode=on \
--cpu host-passthrough,\
numa.cell0.memory=536870912,numa.cell0.cpus=0-21,\
numa.cell1.memory=536870912,numa.cell1.cpus=22-43,\
numa.cell2.memory=536870912,numa.cell2.cpus=44-65,\
numa.cell3.memory=536870912,numa.cell3.cpus=66-87,\
numa.cell4.memory=536870912,numa.cell4.cpus=88-109,\
numa.cell5.memory=536870912,numa.cell5.cpus=110-131,\
numa.cell6.memory=536870912,numa.cell6.cpus=132-153,\
numa.cell7.memory=536870912,numa.cell7.cpus=154-175,\
numa.cell8.memory=536870912,numa.cell8.cpus=176-197,\
numa.cell9.memory=536870912,numa.cell9.cpus=198-219,\
numa.cell10.memory=536870912,numa.cell10.cpus=220-241,\
numa.cell11.memory=536870912,numa.cell11.cpus=242-263,\
numa.cell12.memory=536870912,numa.cell12.cpus=264-285,\
numa.cell13.memory=536870912,numa.cell13.cpus=286-307,\
numa.cell14.memory=536870912,numa.cell14.cpus=308-329,\
numa.cell15.memory=536870912,numa.cell15.cpus=330-351,\
numa.cell16.memory=536870912,numa.cell16.cpus=352-373,\
numa.cell17.memory=536870912,numa.cell17.cpus=374-395,\
numa.cell18.memory=536870912,numa.cell18.cpus=396-417,\
numa.cell19.memory=536870912,numa.cell19.cpus=418-439,\
numa.cell20.memory=536870912,numa.cell20.cpus=440-461,\
numa.cell21.memory=536870912,numa.cell21.cpus=462-483,\
numa.cell22.memory=536870912,numa.cell22.cpus=484-505,\
numa.cell23.memory=536870912,numa.cell23.cpus=506-527,\
numa.cell24.memory=536870912,numa.cell24.cpus=528-549,\
numa.cell25.memory=536870912,numa.cell25.cpus=550-571,\
numa.cell26.memory=536870912,numa.cell26.cpus=572-593,\
numa.cell27.memory=536870912,numa.cell27.cpus=594-615,\
numa.cell28.memory=536870912,numa.cell28.cpus=616-637,\
numa.cell29.memory=536870912,numa.cell29.cpus=638-659,\
numa.cell30.memory=536870912,numa.cell30.cpus=660-681,\
numa.cell31.memory=536870912,numa.cell31.cpus=682-703 \
--numatune 0-31,\
memnode0.cellid=0,memnode0.mode=strict,memnode0.nodeset=0,\
memnode1.cellid=1,memnode1.mode=strict,memnode1.nodeset=1,\
memnode2.cellid=2,memnode2.mode=strict,memnode2.nodeset=2,\
memnode3.cellid=3,memnode3.mode=strict,memnode3.nodeset=3,\
memnode4.cellid=4,memnode4.mode=strict,memnode4.nodeset=4,\
memnode5.cellid=5,memnode5.mode=strict,memnode5.nodeset=5,\
memnode6.cellid=6,memnode6.mode=strict,memnode6.nodeset=6,\
memnode7.cellid=7,memnode7.mode=strict,memnode7.nodeset=7,\
memnode8.cellid=8,memnode8.mode=strict,memnode8.nodeset=8,\
memnode9.cellid=9,memnode9.mode=strict,memnode9.nodeset=9,\
memnode10.cellid=10,memnode10.mode=strict,memnode10.nodeset=10,\
memnode11.cellid=11,memnode11.mode=strict,memnode11.nodeset=11,\
memnode12.cellid=12,memnode12.mode=strict,memnode12.nodeset=12,\
memnode13.cellid=13,memnode13.mode=strict,memnode13.nodeset=13,\
memnode14.cellid=14,memnode14.mode=strict,memnode14.nodeset=14,\
memnode15.cellid=15,memnode15.mode=strict,memnode15.nodeset=15,\
memnode16.cellid=16,memnode16.mode=strict,memnode16.nodeset=16,\
memnode17.cellid=17,memnode17.mode=strict,memnode17.nodeset=17,\
memnode18.cellid=18,memnode18.mode=strict,memnode18.nodeset=18,\
memnode19.cellid=19,memnode19.mode=strict,memnode19.nodeset=19,\
memnode20.cellid=20,memnode20.mode=strict,memnode20.nodeset=20,\
memnode21.cellid=21,memnode21.mode=strict,memnode21.nodeset=21,\
memnode22.cellid=22,memnode22.mode=strict,memnode22.nodeset=22,\
memnode23.cellid=23,memnode23.mode=strict,memnode23.nodeset=23,\
memnode24.cellid=24,memnode24.mode=strict,memnode24.nodeset=24,\
memnode25.cellid=25,memnode25.mode=strict,memnode25.nodeset=25,\
memnode26.cellid=26,memnode26.mode=strict,memnode26.nodeset=26,\
memnode27.cellid=27,memnode27.mode=strict,memnode27.nodeset=27,\
memnode28.cellid=28,memnode28.mode=strict,memnode28.nodeset=28,\
memnode29.cellid=29,memnode29.mode=strict,memnode29.nodeset=29,\
memnode30.cellid=30,memnode30.mode=strict,memnode30.nodeset=30,\
memnode31.cellid=31,memnode31.mode=strict,memnode31.nodeset=31 \
--cputune \
vcpupin0.vcpu=0,vcpupin0.cpuset=1,897,\
vcpupin1.vcpu=1,vcpupin1.cpuset=1,897,\
vcpupin2.vcpu=2,vcpupin2.cpuset=2,898,\
vcpupin3.vcpu=3,vcpupin3.cpuset=2,898,\
vcpupin4.vcpu=4,vcpupin4.cpuset=3,899,\
vcpupin5.vcpu=5,vcpupin5.cpuset=3,899,\
vcpupin6.vcpu=6,vcpupin6.cpuset=4,900,\
vcpupin7.vcpu=7,vcpupin7.cpuset=4,900,\
vcpupin8.vcpu=8,vcpupin8.cpuset=5,901,\
vcpupin9.vcpu=9,vcpupin9.cpuset=5,901,\
vcpupin10.vcpu=10,vcpupin10.cpuset=6,902,\
vcpupin11.vcpu=11,vcpupin11.cpuset=6,902,\
vcpupin12.vcpu=12,vcpupin12.cpuset=7,903,\
vcpupin13.vcpu=13,vcpupin13.cpuset=7,903,\
vcpupin14.vcpu=14,vcpupin14.cpuset=8,904,\
vcpupin15.vcpu=15,vcpupin15.cpuset=8,904,\
vcpupin16.vcpu=16,vcpupin16.cpuset=9,906,\
vcpupin17.vcpu=17,vcpupin17.cpuset=9,906,\
vcpupin18.vcpu=18,vcpupin18.cpuset=10,906,\
vcpupin19.vcpu=19,vcpupin19.cpuset=10,906,\
vcpupin20.vcpu=20,vcpupin20.cpuset=11,907,\
vcpupin21.vcpu=21,vcpupin21.cpuset=11,907,\
vcpupin22.vcpu=22,vcpupin22.cpuset=29,925,\
vcpupin23.vcpu=23,vcpupin23.cpuset=29,925,\
vcpupin24.vcpu=24,vcpupin24.cpuset=30,926,\
vcpupin25.vcpu=25,vcpupin25.cpuset=30,926,\
vcpupin26.vcpu=26,vcpupin26.cpuset=31,927,\
vcpupin27.vcpu=27,vcpupin27.cpuset=31,927,\
vcpupin28.vcpu=28,vcpupin28.cpuset=32,928,\
vcpupin29.vcpu=29,vcpupin29.cpuset=32,928,\
vcpupin30.vcpu=30,vcpupin30.cpuset=33,929,\
vcpupin31.vcpu=31,vcpupin31.cpuset=33,929,\
vcpupin32.vcpu=32,vcpupin32.cpuset=34,930,\
vcpupin33.vcpu=33,vcpupin33.cpuset=34,930,\
vcpupin34.vcpu=34,vcpupin34.cpuset=35,931,\
vcpupin35.vcpu=35,vcpupin35.cpuset=35,931,\
vcpupin36.vcpu=36,vcpupin36.cpuset=36,932,\
vcpupin37.vcpu=37,vcpupin37.cpuset=36,932,\
vcpupin38.vcpu=38,vcpupin38.cpuset=37,934,\
vcpupin39.vcpu=39,vcpupin39.cpuset=37,934,\
vcpupin40.vcpu=40,vcpupin40.cpuset=38,934,\
vcpupin41.vcpu=41,vcpupin41.cpuset=38,934,\
vcpupin42.vcpu=42,vcpupin42.cpuset=39,935,\
vcpupin43.vcpu=43,vcpupin43.cpuset=39,935,\
vcpupin44.vcpu=44,vcpupin44.cpuset=57,953,\
vcpupin45.vcpu=45,vcpupin45.cpuset=57,953,\
vcpupin46.vcpu=46,vcpupin46.cpuset=58,954,\
vcpupin47.vcpu=47,vcpupin47.cpuset=58,954,\
vcpupin48.vcpu=48,vcpupin48.cpuset=59,955,\
vcpupin49.vcpu=49,vcpupin49.cpuset=59,955,\
vcpupin50.vcpu=50,vcpupin50.cpuset=60,956,\
vcpupin51.vcpu=51,vcpupin51.cpuset=60,956,\
vcpupin52.vcpu=52,vcpupin52.cpuset=61,957,\
vcpupin53.vcpu=53,vcpupin53.cpuset=61,957,\
vcpupin54.vcpu=54,vcpupin54.cpuset=62,958,\
vcpupin55.vcpu=55,vcpupin55.cpuset=62,958,\
vcpupin56.vcpu=56,vcpupin56.cpuset=63,959,\
vcpupin57.vcpu=57,vcpupin57.cpuset=63,959,\
vcpupin58.vcpu=58,vcpupin58.cpuset=64,960,\
vcpupin59.vcpu=59,vcpupin59.cpuset=64,960,\
vcpupin60.vcpu=60,vcpupin60.cpuset=65,962,\
vcpupin61.vcpu=61,vcpupin61.cpuset=65,962,\
vcpupin62.vcpu=62,vcpupin62.cpuset=66,962,\
vcpupin63.vcpu=63,vcpupin63.cpuset=66,962,\
vcpupin64.vcpu=64,vcpupin64.cpuset=67,963,\
vcpupin65.vcpu=65,vcpupin65.cpuset=67,963,\
vcpupin66.vcpu=66,vcpupin66.cpuset=85,981,\
vcpupin67.vcpu=67,vcpupin67.cpuset=85,981,\
vcpupin68.vcpu=68,vcpupin68.cpuset=86,982,\
vcpupin69.vcpu=69,vcpupin69.cpuset=86,982,\
vcpupin70.vcpu=70,vcpupin70.cpuset=87,983,\
vcpupin71.vcpu=71,vcpupin71.cpuset=87,983,\
vcpupin72.vcpu=72,vcpupin72.cpuset=88,984,\
vcpupin73.vcpu=73,vcpupin73.cpuset=88,984,\
vcpupin74.vcpu=74,vcpupin74.cpuset=89,985,\
vcpupin75.vcpu=75,vcpupin75.cpuset=89,985,\
vcpupin76.vcpu=76,vcpupin76.cpuset=90,986,\
vcpupin77.vcpu=77,vcpupin77.cpuset=90,986,\
vcpupin78.vcpu=78,vcpupin78.cpuset=91,987,\
vcpupin79.vcpu=79,vcpupin79.cpuset=91,987,\
vcpupin80.vcpu=80,vcpupin80.cpuset=92,988,\
vcpupin81.vcpu=81,vcpupin81.cpuset=92,988,\
vcpupin82.vcpu=82,vcpupin82.cpuset=93,990,\
vcpupin83.vcpu=83,vcpupin83.cpuset=93,990,\
vcpupin84.vcpu=84,vcpupin84.cpuset=94,990,\
vcpupin85.vcpu=85,vcpupin85.cpuset=94,990,\
vcpupin86.vcpu=86,vcpupin86.cpuset=95,991,\
vcpupin87.vcpu=87,vcpupin87.cpuset=95,991,\
vcpupin88.vcpu=88,vcpupin88.cpuset=113,1009,\
vcpupin89.vcpu=89,vcpupin89.cpuset=113,1009,\
vcpupin90.vcpu=90,vcpupin90.cpuset=114,1010,\
vcpupin91.vcpu=91,vcpupin91.cpuset=114,1010,\
vcpupin92.vcpu=92,vcpupin92.cpuset=115,1011,\
vcpupin93.vcpu=93,vcpupin93.cpuset=115,1011,\
vcpupin94.vcpu=94,vcpupin94.cpuset=116,1012,\
vcpupin95.vcpu=95,vcpupin95.cpuset=116,1012,\
vcpupin96.vcpu=96,vcpupin96.cpuset=117,1013,\
vcpupin97.vcpu=97,vcpupin97.cpuset=117,1013,\
vcpupin98.vcpu=98,vcpupin98.cpuset=118,1014,\
vcpupin99.vcpu=99,vcpupin99.cpuset=118,1014,\
vcpupin100.vcpu=100,vcpupin100.cpuset=119,1015,\
vcpupin101.vcpu=101,vcpupin101.cpuset=119,1015,\
vcpupin102.vcpu=102,vcpupin102.cpuset=120,1016,\
vcpupin103.vcpu=103,vcpupin103.cpuset=120,1016,\
vcpupin104.vcpu=104,vcpupin104.cpuset=121,1018,\
vcpupin105.vcpu=105,vcpupin105.cpuset=121,1018,\
vcpupin106.vcpu=106,vcpupin106.cpuset=122,1018,\
vcpupin107.vcpu=107,vcpupin107.cpuset=122,1018,\
vcpupin108.vcpu=108,vcpupin108.cpuset=123,1019,\
vcpupin109.vcpu=109,vcpupin109.cpuset=123,1019,\
vcpupin110.vcpu=110,vcpupin110.cpuset=141,1037,\
vcpupin111.vcpu=111,vcpupin111.cpuset=141,1037,\
vcpupin112.vcpu=112,vcpupin112.cpuset=142,1038,\
vcpupin113.vcpu=113,vcpupin113.cpuset=142,1038,\
vcpupin114.vcpu=114,vcpupin114.cpuset=143,1039,\
vcpupin115.vcpu=115,vcpupin115.cpuset=143,1039,\
vcpupin116.vcpu=116,vcpupin116.cpuset=144,1040,\
vcpupin117.vcpu=117,vcpupin117.cpuset=144,1040,\
vcpupin118.vcpu=118,vcpupin118.cpuset=145,1041,\
vcpupin119.vcpu=119,vcpupin119.cpuset=145,1041,\
vcpupin120.vcpu=120,vcpupin120.cpuset=146,1042,\
vcpupin121.vcpu=121,vcpupin121.cpuset=146,1042,\
vcpupin122.vcpu=122,vcpupin122.cpuset=147,1043,\
vcpupin123.vcpu=123,vcpupin123.cpuset=147,1043,\
vcpupin124.vcpu=124,vcpupin124.cpuset=148,1044,\
vcpupin125.vcpu=125,vcpupin125.cpuset=148,1044,\
vcpupin126.vcpu=126,vcpupin126.cpuset=149,1046,\
vcpupin127.vcpu=127,vcpupin127.cpuset=149,1046,\
vcpupin128.vcpu=128,vcpupin128.cpuset=150,1046,\
vcpupin129.vcpu=129,vcpupin129.cpuset=150,1046,\
vcpupin130.vcpu=130,vcpupin130.cpuset=151,1047,\
vcpupin131.vcpu=131,vcpupin131.cpuset=151,1047,\
vcpupin132.vcpu=132,vcpupin132.cpuset=169,1065,\
vcpupin133.vcpu=133,vcpupin133.cpuset=169,1065,\
vcpupin134.vcpu=134,vcpupin134.cpuset=170,1066,\
vcpupin135.vcpu=135,vcpupin135.cpuset=170,1066,\
vcpupin136.vcpu=136,vcpupin136.cpuset=171,1067,\
vcpupin137.vcpu=137,vcpupin137.cpuset=171,1067,\
vcpupin138.vcpu=138,vcpupin138.cpuset=172,1068,\
vcpupin139.vcpu=139,vcpupin139.cpuset=172,1068,\
vcpupin140.vcpu=140,vcpupin140.cpuset=173,1069,\
vcpupin141.vcpu=141,vcpupin141.cpuset=173,1069,\
vcpupin142.vcpu=142,vcpupin142.cpuset=174,1070,\
vcpupin143.vcpu=143,vcpupin143.cpuset=174,1070,\
vcpupin144.vcpu=144,vcpupin144.cpuset=175,1071,\
vcpupin145.vcpu=145,vcpupin145.cpuset=175,1071,\
vcpupin146.vcpu=146,vcpupin146.cpuset=176,1072,\
vcpupin147.vcpu=147,vcpupin147.cpuset=176,1072,\
vcpupin148.vcpu=148,vcpupin148.cpuset=177,1074,\
vcpupin149.vcpu=149,vcpupin149.cpuset=177,1074,\
vcpupin150.vcpu=150,vcpupin150.cpuset=178,1074,\
vcpupin151.vcpu=151,vcpupin151.cpuset=178,1074,\
vcpupin152.vcpu=152,vcpupin152.cpuset=179,1075,\
vcpupin153.vcpu=153,vcpupin153.cpuset=179,1075,\
vcpupin154.vcpu=154,vcpupin154.cpuset=197,1093,\
vcpupin155.vcpu=155,vcpupin155.cpuset=197,1093,\
vcpupin156.vcpu=156,vcpupin156.cpuset=198,1094,\
vcpupin157.vcpu=157,vcpupin157.cpuset=198,1094,\
vcpupin158.vcpu=158,vcpupin158.cpuset=199,1095,\
vcpupin159.vcpu=159,vcpupin159.cpuset=199,1095,\
vcpupin160.vcpu=160,vcpupin160.cpuset=200,1096,\
vcpupin161.vcpu=161,vcpupin161.cpuset=200,1096,\
vcpupin162.vcpu=162,vcpupin162.cpuset=201,1097,\
vcpupin163.vcpu=163,vcpupin163.cpuset=201,1097,\
vcpupin164.vcpu=164,vcpupin164.cpuset=202,1098,\
vcpupin165.vcpu=165,vcpupin165.cpuset=202,1098,\
vcpupin166.vcpu=166,vcpupin166.cpuset=203,1099,\
vcpupin167.vcpu=167,vcpupin167.cpuset=203,1099,\
vcpupin168.vcpu=168,vcpupin168.cpuset=204,1100,\
vcpupin169.vcpu=169,vcpupin169.cpuset=204,1100,\
vcpupin170.vcpu=170,vcpupin170.cpuset=205,1102,\
vcpupin171.vcpu=171,vcpupin171.cpuset=205,1102,\
vcpupin172.vcpu=172,vcpupin172.cpuset=206,1102,\
vcpupin173.vcpu=173,vcpupin173.cpuset=206,1102,\
vcpupin174.vcpu=174,vcpupin174.cpuset=207,1103,\
vcpupin175.vcpu=175,vcpupin175.cpuset=207,1103,\
vcpupin176.vcpu=176,vcpupin176.cpuset=225,1121,\
vcpupin177.vcpu=177,vcpupin177.cpuset=225,1121,\
vcpupin178.vcpu=178,vcpupin178.cpuset=226,1122,\
vcpupin179.vcpu=179,vcpupin179.cpuset=226,1122,\
vcpupin180.vcpu=180,vcpupin180.cpuset=227,1123,\
vcpupin181.vcpu=181,vcpupin181.cpuset=227,1123,\
vcpupin182.vcpu=182,vcpupin182.cpuset=228,1124,\
vcpupin183.vcpu=183,vcpupin183.cpuset=228,1124,\
vcpupin184.vcpu=184,vcpupin184.cpuset=229,1125,\
vcpupin185.vcpu=185,vcpupin185.cpuset=229,1125,\
vcpupin186.vcpu=186,vcpupin186.cpuset=230,1126,\
vcpupin187.vcpu=187,vcpupin187.cpuset=230,1126,\
vcpupin188.vcpu=188,vcpupin188.cpuset=231,1127,\
vcpupin189.vcpu=189,vcpupin189.cpuset=231,1127,\
vcpupin190.vcpu=190,vcpupin190.cpuset=232,1128,\
vcpupin191.vcpu=191,vcpupin191.cpuset=232,1128,\
vcpupin192.vcpu=192,vcpupin192.cpuset=233,1130,\
vcpupin193.vcpu=193,vcpupin193.cpuset=233,1130,\
vcpupin194.vcpu=194,vcpupin194.cpuset=234,1130,\
vcpupin195.vcpu=195,vcpupin195.cpuset=234,1130,\
vcpupin196.vcpu=196,vcpupin196.cpuset=235,1131,\
vcpupin197.vcpu=197,vcpupin197.cpuset=235,1131,\
vcpupin198.vcpu=198,vcpupin198.cpuset=253,1149,\
vcpupin199.vcpu=199,vcpupin199.cpuset=253,1149,\
vcpupin200.vcpu=200,vcpupin200.cpuset=254,1150,\
vcpupin201.vcpu=201,vcpupin201.cpuset=254,1150,\
vcpupin202.vcpu=202,vcpupin202.cpuset=255,1151,\
vcpupin203.vcpu=203,vcpupin203.cpuset=255,1151,\
vcpupin204.vcpu=204,vcpupin204.cpuset=256,1152,\
vcpupin205.vcpu=205,vcpupin205.cpuset=256,1152,\
vcpupin206.vcpu=206,vcpupin206.cpuset=257,1153,\
vcpupin207.vcpu=207,vcpupin207.cpuset=257,1153,\
vcpupin208.vcpu=208,vcpupin208.cpuset=258,1154,\
vcpupin209.vcpu=209,vcpupin209.cpuset=258,1154,\
vcpupin210.vcpu=210,vcpupin210.cpuset=259,1155,\
vcpupin211.vcpu=211,vcpupin211.cpuset=259,1155,\
vcpupin212.vcpu=212,vcpupin212.cpuset=260,1156,\
vcpupin213.vcpu=213,vcpupin213.cpuset=260,1156,\
vcpupin214.vcpu=214,vcpupin214.cpuset=261,1158,\
vcpupin215.vcpu=215,vcpupin215.cpuset=261,1158,\
vcpupin216.vcpu=216,vcpupin216.cpuset=262,1158,\
vcpupin217.vcpu=217,vcpupin217.cpuset=262,1158,\
vcpupin218.vcpu=218,vcpupin218.cpuset=263,1159,\
vcpupin219.vcpu=219,vcpupin219.cpuset=263,1159,\
vcpupin220.vcpu=220,vcpupin220.cpuset=281,1177,\
vcpupin221.vcpu=221,vcpupin221.cpuset=281,1177,\
vcpupin222.vcpu=222,vcpupin222.cpuset=282,1178,\
vcpupin223.vcpu=223,vcpupin223.cpuset=282,1178,\
vcpupin224.vcpu=224,vcpupin224.cpuset=283,1179,\
vcpupin225.vcpu=225,vcpupin225.cpuset=283,1179,\
vcpupin226.vcpu=226,vcpupin226.cpuset=284,1180,\
vcpupin227.vcpu=227,vcpupin227.cpuset=284,1180,\
vcpupin228.vcpu=228,vcpupin228.cpuset=285,1181,\
vcpupin229.vcpu=229,vcpupin229.cpuset=285,1181,\
vcpupin230.vcpu=230,vcpupin230.cpuset=286,1182,\
vcpupin231.vcpu=231,vcpupin231.cpuset=286,1182,\
vcpupin232.vcpu=232,vcpupin232.cpuset=287,1183,\
vcpupin233.vcpu=233,vcpupin233.cpuset=287,1183,\
vcpupin234.vcpu=234,vcpupin234.cpuset=288,1184,\
vcpupin235.vcpu=235,vcpupin235.cpuset=288,1184,\
vcpupin236.vcpu=236,vcpupin236.cpuset=289,1186,\
vcpupin237.vcpu=237,vcpupin237.cpuset=289,1186,\
vcpupin238.vcpu=238,vcpupin238.cpuset=290,1186,\
vcpupin239.vcpu=239,vcpupin239.cpuset=290,1186,\
vcpupin240.vcpu=240,vcpupin240.cpuset=291,1187,\
vcpupin241.vcpu=241,vcpupin241.cpuset=291,1187,\
vcpupin242.vcpu=242,vcpupin242.cpuset=309,1205,\
vcpupin243.vcpu=243,vcpupin243.cpuset=309,1205,\
vcpupin244.vcpu=244,vcpupin244.cpuset=310,1206,\
vcpupin245.vcpu=245,vcpupin245.cpuset=310,1206,\
vcpupin246.vcpu=246,vcpupin246.cpuset=311,1207,\
vcpupin247.vcpu=247,vcpupin247.cpuset=311,1207,\
vcpupin248.vcpu=248,vcpupin248.cpuset=312,1208,\
vcpupin249.vcpu=249,vcpupin249.cpuset=312,1208,\
vcpupin250.vcpu=250,vcpupin250.cpuset=313,1209,\
vcpupin251.vcpu=251,vcpupin251.cpuset=313,1209,\
vcpupin252.vcpu=252,vcpupin252.cpuset=314,1210,\
vcpupin253.vcpu=253,vcpupin253.cpuset=314,1210,\
vcpupin254.vcpu=254,vcpupin254.cpuset=315,1211,\
vcpupin255.vcpu=255,vcpupin255.cpuset=315,1211,\
vcpupin256.vcpu=256,vcpupin256.cpuset=316,1212,\
vcpupin257.vcpu=257,vcpupin257.cpuset=316,1212,\
vcpupin258.vcpu=258,vcpupin258.cpuset=317,1214,\
vcpupin259.vcpu=259,vcpupin259.cpuset=317,1214,\
vcpupin260.vcpu=260,vcpupin260.cpuset=318,1214,\
vcpupin261.vcpu=261,vcpupin261.cpuset=318,1214,\
vcpupin262.vcpu=262,vcpupin262.cpuset=319,1215,\
vcpupin263.vcpu=263,vcpupin263.cpuset=319,1215,\
vcpupin264.vcpu=264,vcpupin264.cpuset=337,1233,\
vcpupin265.vcpu=265,vcpupin265.cpuset=337,1233,\
vcpupin266.vcpu=266,vcpupin266.cpuset=338,1234,\
vcpupin267.vcpu=267,vcpupin267.cpuset=338,1234,\
vcpupin268.vcpu=268,vcpupin268.cpuset=339,1235,\
vcpupin269.vcpu=269,vcpupin269.cpuset=339,1235,\
vcpupin270.vcpu=270,vcpupin270.cpuset=340,1236,\
vcpupin271.vcpu=271,vcpupin271.cpuset=340,1236,\
vcpupin272.vcpu=272,vcpupin272.cpuset=341,1237,\
vcpupin273.vcpu=273,vcpupin273.cpuset=341,1237,\
vcpupin274.vcpu=274,vcpupin274.cpuset=342,1238,\
vcpupin275.vcpu=275,vcpupin275.cpuset=342,1238,\
vcpupin276.vcpu=276,vcpupin276.cpuset=343,1239,\
vcpupin277.vcpu=277,vcpupin277.cpuset=343,1239,\
vcpupin278.vcpu=278,vcpupin278.cpuset=344,1240,\
vcpupin279.vcpu=279,vcpupin279.cpuset=344,1240,\
vcpupin280.vcpu=280,vcpupin280.cpuset=345,1242,\
vcpupin281.vcpu=281,vcpupin281.cpuset=345,1242,\
vcpupin282.vcpu=282,vcpupin282.cpuset=346,1242,\
vcpupin283.vcpu=283,vcpupin283.cpuset=346,1242,\
vcpupin284.vcpu=284,vcpupin284.cpuset=347,1243,\
vcpupin285.vcpu=285,vcpupin285.cpuset=347,1243,\
vcpupin286.vcpu=286,vcpupin286.cpuset=365,1261,\
vcpupin287.vcpu=287,vcpupin287.cpuset=365,1261,\
vcpupin288.vcpu=288,vcpupin288.cpuset=366,1262,\
vcpupin289.vcpu=289,vcpupin289.cpuset=366,1262,\
vcpupin290.vcpu=290,vcpupin290.cpuset=367,1263,\
vcpupin291.vcpu=291,vcpupin291.cpuset=367,1263,\
vcpupin292.vcpu=292,vcpupin292.cpuset=368,1264,\
vcpupin293.vcpu=293,vcpupin293.cpuset=368,1264,\
vcpupin294.vcpu=294,vcpupin294.cpuset=369,1265,\
vcpupin295.vcpu=295,vcpupin295.cpuset=369,1265,\
vcpupin296.vcpu=296,vcpupin296.cpuset=370,1266,\
vcpupin297.vcpu=297,vcpupin297.cpuset=370,1266,\
vcpupin298.vcpu=298,vcpupin298.cpuset=371,1267,\
vcpupin299.vcpu=299,vcpupin299.cpuset=371,1267,\
vcpupin300.vcpu=300,vcpupin300.cpuset=372,1268,\
vcpupin301.vcpu=301,vcpupin301.cpuset=372,1268,\
vcpupin302.vcpu=302,vcpupin302.cpuset=373,1270,\
vcpupin303.vcpu=303,vcpupin303.cpuset=373,1270,\
vcpupin304.vcpu=304,vcpupin304.cpuset=374,1270,\
vcpupin305.vcpu=305,vcpupin305.cpuset=374,1270,\
vcpupin306.vcpu=306,vcpupin306.cpuset=375,1271,\
vcpupin307.vcpu=307,vcpupin307.cpuset=375,1271,\
vcpupin308.vcpu=308,vcpupin308.cpuset=393,1289,\
vcpupin309.vcpu=309,vcpupin309.cpuset=393,1289,\
vcpupin310.vcpu=310,vcpupin310.cpuset=394,1290,\
vcpupin311.vcpu=311,vcpupin311.cpuset=394,1290,\
vcpupin312.vcpu=312,vcpupin312.cpuset=395,1291,\
vcpupin313.vcpu=313,vcpupin313.cpuset=395,1291,\
vcpupin314.vcpu=314,vcpupin314.cpuset=396,1292,\
vcpupin315.vcpu=315,vcpupin315.cpuset=396,1292,\
vcpupin316.vcpu=316,vcpupin316.cpuset=397,1293,\
vcpupin317.vcpu=317,vcpupin317.cpuset=397,1293,\
vcpupin318.vcpu=318,vcpupin318.cpuset=398,1294,\
vcpupin319.vcpu=319,vcpupin319.cpuset=398,1294,\
vcpupin320.vcpu=320,vcpupin320.cpuset=399,1295,\
vcpupin321.vcpu=321,vcpupin321.cpuset=399,1295,\
vcpupin322.vcpu=322,vcpupin322.cpuset=400,1296,\
vcpupin323.vcpu=323,vcpupin323.cpuset=400,1296,\
vcpupin324.vcpu=324,vcpupin324.cpuset=401,1298,\
vcpupin325.vcpu=325,vcpupin325.cpuset=401,1298,\
vcpupin326.vcpu=326,vcpupin326.cpuset=402,1298,\
vcpupin327.vcpu=327,vcpupin327.cpuset=402,1298,\
vcpupin328.vcpu=328,vcpupin328.cpuset=403,1299,\
vcpupin329.vcpu=329,vcpupin329.cpuset=403,1299,\
vcpupin330.vcpu=330,vcpupin330.cpuset=421,1317,\
vcpupin331.vcpu=331,vcpupin331.cpuset=421,1317,\
vcpupin332.vcpu=332,vcpupin332.cpuset=422,1318,\
vcpupin333.vcpu=333,vcpupin333.cpuset=422,1318,\
vcpupin334.vcpu=334,vcpupin334.cpuset=423,1319,\
vcpupin335.vcpu=335,vcpupin335.cpuset=423,1319,\
vcpupin336.vcpu=336,vcpupin336.cpuset=424,1320,\
vcpupin337.vcpu=337,vcpupin337.cpuset=424,1320,\
vcpupin338.vcpu=338,vcpupin338.cpuset=425,1321,\
vcpupin339.vcpu=339,vcpupin339.cpuset=425,1321,\
vcpupin340.vcpu=340,vcpupin340.cpuset=426,1322,\
vcpupin341.vcpu=341,vcpupin341.cpuset=426,1322,\
vcpupin342.vcpu=342,vcpupin342.cpuset=427,1323,\
vcpupin343.vcpu=343,vcpupin343.cpuset=427,1323,\
vcpupin344.vcpu=344,vcpupin344.cpuset=428,1324,\
vcpupin345.vcpu=345,vcpupin345.cpuset=428,1324,\
vcpupin346.vcpu=346,vcpupin346.cpuset=429,1326,\
vcpupin347.vcpu=347,vcpupin347.cpuset=429,1326,\
vcpupin348.vcpu=348,vcpupin348.cpuset=430,1326,\
vcpupin349.vcpu=349,vcpupin349.cpuset=430,1326,\
vcpupin350.vcpu=350,vcpupin350.cpuset=431,1327,\
vcpupin351.vcpu=351,vcpupin351.cpuset=431,1327,\
vcpupin352.vcpu=352,vcpupin352.cpuset=449,1345,\
vcpupin353.vcpu=353,vcpupin353.cpuset=449,1345,\
vcpupin354.vcpu=354,vcpupin354.cpuset=450,1346,\
vcpupin355.vcpu=355,vcpupin355.cpuset=450,1346,\
vcpupin356.vcpu=356,vcpupin356.cpuset=451,1347,\
vcpupin357.vcpu=357,vcpupin357.cpuset=451,1347,\
vcpupin358.vcpu=358,vcpupin358.cpuset=452,1348,\
vcpupin359.vcpu=359,vcpupin359.cpuset=452,1348,\
vcpupin360.vcpu=360,vcpupin360.cpuset=453,1349,\
vcpupin361.vcpu=361,vcpupin361.cpuset=453,1349,\
vcpupin362.vcpu=362,vcpupin362.cpuset=454,1350,\
vcpupin363.vcpu=363,vcpupin363.cpuset=454,1350,\
vcpupin364.vcpu=364,vcpupin364.cpuset=455,1351,\
vcpupin365.vcpu=365,vcpupin365.cpuset=455,1351,\
vcpupin366.vcpu=366,vcpupin366.cpuset=456,1352,\
vcpupin367.vcpu=367,vcpupin367.cpuset=456,1352,\
vcpupin368.vcpu=368,vcpupin368.cpuset=457,1354,\
vcpupin369.vcpu=369,vcpupin369.cpuset=457,1354,\
vcpupin370.vcpu=370,vcpupin370.cpuset=458,1354,\
vcpupin371.vcpu=371,vcpupin371.cpuset=458,1354,\
vcpupin372.vcpu=372,vcpupin372.cpuset=459,1355,\
vcpupin373.vcpu=373,vcpupin373.cpuset=459,1355,\
vcpupin374.vcpu=374,vcpupin374.cpuset=477,1373,\
vcpupin375.vcpu=375,vcpupin375.cpuset=477,1373,\
vcpupin376.vcpu=376,vcpupin376.cpuset=478,1374,\
vcpupin377.vcpu=377,vcpupin377.cpuset=478,1374,\
vcpupin378.vcpu=378,vcpupin378.cpuset=479,1375,\
vcpupin379.vcpu=379,vcpupin379.cpuset=479,1375,\
vcpupin380.vcpu=380,vcpupin380.cpuset=480,1376,\
vcpupin381.vcpu=381,vcpupin381.cpuset=480,1376,\
vcpupin382.vcpu=382,vcpupin382.cpuset=481,1377,\
vcpupin383.vcpu=383,vcpupin383.cpuset=481,1377,\
vcpupin384.vcpu=384,vcpupin384.cpuset=482,1378,\
vcpupin385.vcpu=385,vcpupin385.cpuset=482,1378,\
vcpupin386.vcpu=386,vcpupin386.cpuset=483,1379,\
vcpupin387.vcpu=387,vcpupin387.cpuset=483,1379,\
vcpupin388.vcpu=388,vcpupin388.cpuset=484,1380,\
vcpupin389.vcpu=389,vcpupin389.cpuset=484,1380,\
vcpupin390.vcpu=390,vcpupin390.cpuset=485,1382,\
vcpupin391.vcpu=391,vcpupin391.cpuset=485,1382,\
vcpupin392.vcpu=392,vcpupin392.cpuset=486,1382,\
vcpupin393.vcpu=393,vcpupin393.cpuset=486,1382,\
vcpupin394.vcpu=394,vcpupin394.cpuset=487,1383,\
vcpupin395.vcpu=395,vcpupin395.cpuset=487,1383,\
vcpupin396.vcpu=396,vcpupin396.cpuset=505,1401,\
vcpupin397.vcpu=397,vcpupin397.cpuset=505,1401,\
vcpupin398.vcpu=398,vcpupin398.cpuset=506,1402,\
vcpupin399.vcpu=399,vcpupin399.cpuset=506,1402,\
vcpupin400.vcpu=400,vcpupin400.cpuset=507,1403,\
vcpupin401.vcpu=401,vcpupin401.cpuset=507,1403,\
vcpupin402.vcpu=402,vcpupin402.cpuset=508,1404,\
vcpupin403.vcpu=403,vcpupin403.cpuset=508,1404,\
vcpupin404.vcpu=404,vcpupin404.cpuset=509,1405,\
vcpupin405.vcpu=405,vcpupin405.cpuset=509,1405,\
vcpupin406.vcpu=406,vcpupin406.cpuset=510,1406,\
vcpupin407.vcpu=407,vcpupin407.cpuset=510,1406,\
vcpupin408.vcpu=408,vcpupin408.cpuset=511,1407,\
vcpupin409.vcpu=409,vcpupin409.cpuset=511,1407,\
vcpupin410.vcpu=410,vcpupin410.cpuset=512,1408,\
vcpupin411.vcpu=411,vcpupin411.cpuset=512,1408,\
vcpupin412.vcpu=412,vcpupin412.cpuset=513,1410,\
vcpupin413.vcpu=413,vcpupin413.cpuset=513,1410,\
vcpupin414.vcpu=414,vcpupin414.cpuset=514,1410,\
vcpupin415.vcpu=415,vcpupin415.cpuset=514,1410,\
vcpupin416.vcpu=416,vcpupin416.cpuset=515,1411,\
vcpupin417.vcpu=417,vcpupin417.cpuset=515,1411,\
vcpupin418.vcpu=418,vcpupin418.cpuset=533,1429,\
vcpupin419.vcpu=419,vcpupin419.cpuset=533,1429,\
vcpupin420.vcpu=420,vcpupin420.cpuset=534,1430,\
vcpupin421.vcpu=421,vcpupin421.cpuset=534,1430,\
vcpupin422.vcpu=422,vcpupin422.cpuset=535,1431,\
vcpupin423.vcpu=423,vcpupin423.cpuset=535,1431,\
vcpupin424.vcpu=424,vcpupin424.cpuset=536,1432,\
vcpupin425.vcpu=425,vcpupin425.cpuset=536,1432,\
vcpupin426.vcpu=426,vcpupin426.cpuset=537,1433,\
vcpupin427.vcpu=427,vcpupin427.cpuset=537,1433,\
vcpupin428.vcpu=428,vcpupin428.cpuset=538,1434,\
vcpupin429.vcpu=429,vcpupin429.cpuset=538,1434,\
vcpupin430.vcpu=430,vcpupin430.cpuset=539,1435,\
vcpupin431.vcpu=431,vcpupin431.cpuset=539,1435,\
vcpupin432.vcpu=432,vcpupin432.cpuset=540,1436,\
vcpupin433.vcpu=433,vcpupin433.cpuset=540,1436,\
vcpupin434.vcpu=434,vcpupin434.cpuset=541,1438,\
vcpupin435.vcpu=435,vcpupin435.cpuset=541,1438,\
vcpupin436.vcpu=436,vcpupin436.cpuset=542,1438,\
vcpupin437.vcpu=437,vcpupin437.cpuset=542,1438,\
vcpupin438.vcpu=438,vcpupin438.cpuset=543,1439,\
vcpupin439.vcpu=439,vcpupin439.cpuset=543,1439,\
vcpupin440.vcpu=440,vcpupin440.cpuset=561,1457,\
vcpupin441.vcpu=441,vcpupin441.cpuset=561,1457,\
vcpupin442.vcpu=442,vcpupin442.cpuset=562,1458,\
vcpupin443.vcpu=443,vcpupin443.cpuset=562,1458,\
vcpupin444.vcpu=444,vcpupin444.cpuset=563,1459,\
vcpupin445.vcpu=445,vcpupin445.cpuset=563,1459,\
vcpupin446.vcpu=446,vcpupin446.cpuset=564,1460,\
vcpupin447.vcpu=447,vcpupin447.cpuset=564,1460,\
vcpupin448.vcpu=448,vcpupin448.cpuset=565,1461,\
vcpupin449.vcpu=449,vcpupin449.cpuset=565,1461,\
vcpupin450.vcpu=450,vcpupin450.cpuset=566,1462,\
vcpupin451.vcpu=451,vcpupin451.cpuset=566,1462,\
vcpupin452.vcpu=452,vcpupin452.cpuset=567,1463,\
vcpupin453.vcpu=453,vcpupin453.cpuset=567,1463,\
vcpupin454.vcpu=454,vcpupin454.cpuset=568,1464,\
vcpupin455.vcpu=455,vcpupin455.cpuset=568,1464,\
vcpupin456.vcpu=456,vcpupin456.cpuset=569,1466,\
vcpupin457.vcpu=457,vcpupin457.cpuset=569,1466,\
vcpupin458.vcpu=458,vcpupin458.cpuset=570,1466,\
vcpupin459.vcpu=459,vcpupin459.cpuset=570,1466,\
vcpupin460.vcpu=460,vcpupin460.cpuset=571,1467,\
vcpupin461.vcpu=461,vcpupin461.cpuset=571,1467,\
vcpupin462.vcpu=462,vcpupin462.cpuset=589,1485,\
vcpupin463.vcpu=463,vcpupin463.cpuset=589,1485,\
vcpupin464.vcpu=464,vcpupin464.cpuset=590,1486,\
vcpupin465.vcpu=465,vcpupin465.cpuset=590,1486,\
vcpupin466.vcpu=466,vcpupin466.cpuset=591,1487,\
vcpupin467.vcpu=467,vcpupin467.cpuset=591,1487,\
vcpupin468.vcpu=468,vcpupin468.cpuset=592,1488,\
vcpupin469.vcpu=469,vcpupin469.cpuset=592,1488,\
vcpupin470.vcpu=470,vcpupin470.cpuset=593,1489,\
vcpupin471.vcpu=471,vcpupin471.cpuset=593,1489,\
vcpupin472.vcpu=472,vcpupin472.cpuset=594,1490,\
vcpupin473.vcpu=473,vcpupin473.cpuset=594,1490,\
vcpupin474.vcpu=474,vcpupin474.cpuset=595,1491,\
vcpupin475.vcpu=475,vcpupin475.cpuset=595,1491,\
vcpupin476.vcpu=476,vcpupin476.cpuset=596,1492,\
vcpupin477.vcpu=477,vcpupin477.cpuset=596,1492,\
vcpupin478.vcpu=478,vcpupin478.cpuset=597,1494,\
vcpupin479.vcpu=479,vcpupin479.cpuset=597,1494,\
vcpupin480.vcpu=480,vcpupin480.cpuset=598,1494,\
vcpupin481.vcpu=481,vcpupin481.cpuset=598,1494,\
vcpupin482.vcpu=482,vcpupin482.cpuset=599,1495,\
vcpupin483.vcpu=483,vcpupin483.cpuset=599,1495,\
vcpupin484.vcpu=484,vcpupin484.cpuset=617,1513,\
vcpupin485.vcpu=485,vcpupin485.cpuset=617,1513,\
vcpupin486.vcpu=486,vcpupin486.cpuset=618,1514,\
vcpupin487.vcpu=487,vcpupin487.cpuset=618,1514,\
vcpupin488.vcpu=488,vcpupin488.cpuset=619,1515,\
vcpupin489.vcpu=489,vcpupin489.cpuset=619,1515,\
vcpupin490.vcpu=490,vcpupin490.cpuset=620,1516,\
vcpupin491.vcpu=491,vcpupin491.cpuset=620,1516,\
vcpupin492.vcpu=492,vcpupin492.cpuset=621,1517,\
vcpupin493.vcpu=493,vcpupin493.cpuset=621,1517,\
vcpupin494.vcpu=494,vcpupin494.cpuset=622,1518,\
vcpupin495.vcpu=495,vcpupin495.cpuset=622,1518,\
vcpupin496.vcpu=496,vcpupin496.cpuset=623,1519,\
vcpupin497.vcpu=497,vcpupin497.cpuset=623,1519,\
vcpupin498.vcpu=498,vcpupin498.cpuset=624,1520,\
vcpupin499.vcpu=499,vcpupin499.cpuset=624,1520,\
vcpupin500.vcpu=500,vcpupin500.cpuset=625,1522,\
vcpupin501.vcpu=501,vcpupin501.cpuset=625,1522,\
vcpupin502.vcpu=502,vcpupin502.cpuset=626,1522,\
vcpupin503.vcpu=503,vcpupin503.cpuset=626,1522,\
vcpupin504.vcpu=504,vcpupin504.cpuset=627,1523,\
vcpupin505.vcpu=505,vcpupin505.cpuset=627,1523,\
vcpupin506.vcpu=506,vcpupin506.cpuset=645,1541,\
vcpupin507.vcpu=507,vcpupin507.cpuset=645,1541,\
vcpupin508.vcpu=508,vcpupin508.cpuset=646,1542,\
vcpupin509.vcpu=509,vcpupin509.cpuset=646,1542,\
vcpupin510.vcpu=510,vcpupin510.cpuset=647,1543,\
vcpupin511.vcpu=511,vcpupin511.cpuset=647,1543,\
vcpupin512.vcpu=512,vcpupin512.cpuset=648,1544,\
vcpupin513.vcpu=513,vcpupin513.cpuset=648,1544,\
vcpupin514.vcpu=514,vcpupin514.cpuset=649,1545,\
vcpupin515.vcpu=515,vcpupin515.cpuset=649,1545,\
vcpupin516.vcpu=516,vcpupin516.cpuset=650,1546,\
vcpupin517.vcpu=517,vcpupin517.cpuset=650,1546,\
vcpupin518.vcpu=518,vcpupin518.cpuset=651,1547,\
vcpupin519.vcpu=519,vcpupin519.cpuset=651,1547,\
vcpupin520.vcpu=520,vcpupin520.cpuset=652,1548,\
vcpupin521.vcpu=521,vcpupin521.cpuset=652,1548,\
vcpupin522.vcpu=522,vcpupin522.cpuset=653,1550,\
vcpupin523.vcpu=523,vcpupin523.cpuset=653,1550,\
vcpupin524.vcpu=524,vcpupin524.cpuset=654,1550,\
vcpupin525.vcpu=525,vcpupin525.cpuset=654,1550,\
vcpupin526.vcpu=526,vcpupin526.cpuset=655,1551,\
vcpupin527.vcpu=527,vcpupin527.cpuset=655,1551,\
vcpupin528.vcpu=528,vcpupin528.cpuset=673,1569,\
vcpupin529.vcpu=529,vcpupin529.cpuset=673,1569,\
vcpupin530.vcpu=530,vcpupin530.cpuset=674,1570,\
vcpupin531.vcpu=531,vcpupin531.cpuset=674,1570,\
vcpupin532.vcpu=532,vcpupin532.cpuset=675,1571,\
vcpupin533.vcpu=533,vcpupin533.cpuset=675,1571,\
vcpupin534.vcpu=534,vcpupin534.cpuset=676,1572,\
vcpupin535.vcpu=535,vcpupin535.cpuset=676,1572,\
vcpupin536.vcpu=536,vcpupin536.cpuset=677,1573,\
vcpupin537.vcpu=537,vcpupin537.cpuset=677,1573,\
vcpupin538.vcpu=538,vcpupin538.cpuset=678,1574,\
vcpupin539.vcpu=539,vcpupin539.cpuset=678,1574,\
vcpupin540.vcpu=540,vcpupin540.cpuset=679,1575,\
vcpupin541.vcpu=541,vcpupin541.cpuset=679,1575,\
vcpupin542.vcpu=542,vcpupin542.cpuset=680,1576,\
vcpupin543.vcpu=543,vcpupin543.cpuset=680,1576,\
vcpupin544.vcpu=544,vcpupin544.cpuset=681,1578,\
vcpupin545.vcpu=545,vcpupin545.cpuset=681,1578,\
vcpupin546.vcpu=546,vcpupin546.cpuset=682,1578,\
vcpupin547.vcpu=547,vcpupin547.cpuset=682,1578,\
vcpupin548.vcpu=548,vcpupin548.cpuset=683,1579,\
vcpupin549.vcpu=549,vcpupin549.cpuset=683,1579,\
vcpupin550.vcpu=550,vcpupin550.cpuset=701,1597,\
vcpupin551.vcpu=551,vcpupin551.cpuset=701,1597,\
vcpupin552.vcpu=552,vcpupin552.cpuset=702,1598,\
vcpupin553.vcpu=553,vcpupin553.cpuset=702,1598,\
vcpupin554.vcpu=554,vcpupin554.cpuset=703,1599,\
vcpupin555.vcpu=555,vcpupin555.cpuset=703,1599,\
vcpupin556.vcpu=556,vcpupin556.cpuset=704,1600,\
vcpupin557.vcpu=557,vcpupin557.cpuset=704,1600,\
vcpupin558.vcpu=558,vcpupin558.cpuset=705,1601,\
vcpupin559.vcpu=559,vcpupin559.cpuset=705,1601,\
vcpupin560.vcpu=560,vcpupin560.cpuset=706,1602,\
vcpupin561.vcpu=561,vcpupin561.cpuset=706,1602,\
vcpupin562.vcpu=562,vcpupin562.cpuset=707,1603,\
vcpupin563.vcpu=563,vcpupin563.cpuset=707,1603,\
vcpupin564.vcpu=564,vcpupin564.cpuset=708,1604,\
vcpupin565.vcpu=565,vcpupin565.cpuset=708,1604,\
vcpupin566.vcpu=566,vcpupin566.cpuset=709,1606,\
vcpupin567.vcpu=567,vcpupin567.cpuset=709,1606,\
vcpupin568.vcpu=568,vcpupin568.cpuset=710,1606,\
vcpupin569.vcpu=569,vcpupin569.cpuset=710,1606,\
vcpupin570.vcpu=570,vcpupin570.cpuset=711,1607,\
vcpupin571.vcpu=571,vcpupin571.cpuset=711,1607,\
vcpupin572.vcpu=572,vcpupin572.cpuset=729,1625,\
vcpupin573.vcpu=573,vcpupin573.cpuset=729,1625,\
vcpupin574.vcpu=574,vcpupin574.cpuset=730,1626,\
vcpupin575.vcpu=575,vcpupin575.cpuset=730,1626,\
vcpupin576.vcpu=576,vcpupin576.cpuset=731,1627,\
vcpupin577.vcpu=577,vcpupin577.cpuset=731,1627,\
vcpupin578.vcpu=578,vcpupin578.cpuset=732,1628,\
vcpupin579.vcpu=579,vcpupin579.cpuset=732,1628,\
vcpupin580.vcpu=580,vcpupin580.cpuset=733,1629,\
vcpupin581.vcpu=581,vcpupin581.cpuset=733,1629,\
vcpupin582.vcpu=582,vcpupin582.cpuset=734,1630,\
vcpupin583.vcpu=583,vcpupin583.cpuset=734,1630,\
vcpupin584.vcpu=584,vcpupin584.cpuset=735,1631,\
vcpupin585.vcpu=585,vcpupin585.cpuset=735,1631,\
vcpupin586.vcpu=586,vcpupin586.cpuset=736,1632,\
vcpupin587.vcpu=587,vcpupin587.cpuset=736,1632,\
vcpupin588.vcpu=588,vcpupin588.cpuset=737,1634,\
vcpupin589.vcpu=589,vcpupin589.cpuset=737,1634,\
vcpupin590.vcpu=590,vcpupin590.cpuset=738,1634,\
vcpupin591.vcpu=591,vcpupin591.cpuset=738,1634,\
vcpupin592.vcpu=592,vcpupin592.cpuset=739,1635,\
vcpupin593.vcpu=593,vcpupin593.cpuset=739,1635,\
vcpupin594.vcpu=594,vcpupin594.cpuset=757,1653,\
vcpupin595.vcpu=595,vcpupin595.cpuset=757,1653,\
vcpupin596.vcpu=596,vcpupin596.cpuset=758,1654,\
vcpupin597.vcpu=597,vcpupin597.cpuset=758,1654,\
vcpupin598.vcpu=598,vcpupin598.cpuset=759,1655,\
vcpupin599.vcpu=599,vcpupin599.cpuset=759,1655,\
vcpupin600.vcpu=600,vcpupin600.cpuset=760,1656,\
vcpupin601.vcpu=601,vcpupin601.cpuset=760,1656,\
vcpupin602.vcpu=602,vcpupin602.cpuset=761,1657,\
vcpupin603.vcpu=603,vcpupin603.cpuset=761,1657,\
vcpupin604.vcpu=604,vcpupin604.cpuset=762,1658,\
vcpupin605.vcpu=605,vcpupin605.cpuset=762,1658,\
vcpupin606.vcpu=606,vcpupin606.cpuset=763,1659,\
vcpupin607.vcpu=607,vcpupin607.cpuset=763,1659,\
vcpupin608.vcpu=608,vcpupin608.cpuset=764,1660,\
vcpupin609.vcpu=609,vcpupin609.cpuset=764,1660,\
vcpupin610.vcpu=610,vcpupin610.cpuset=765,1662,\
vcpupin611.vcpu=611,vcpupin611.cpuset=765,1662,\
vcpupin612.vcpu=612,vcpupin612.cpuset=766,1662,\
vcpupin613.vcpu=613,vcpupin613.cpuset=766,1662,\
vcpupin614.vcpu=614,vcpupin614.cpuset=767,1663,\
vcpupin615.vcpu=615,vcpupin615.cpuset=767,1663,\
vcpupin616.vcpu=616,vcpupin616.cpuset=785,1681,\
vcpupin617.vcpu=617,vcpupin617.cpuset=785,1681,\
vcpupin618.vcpu=618,vcpupin618.cpuset=786,1682,\
vcpupin619.vcpu=619,vcpupin619.cpuset=786,1682,\
vcpupin620.vcpu=620,vcpupin620.cpuset=787,1683,\
vcpupin621.vcpu=621,vcpupin621.cpuset=787,1683,\
vcpupin622.vcpu=622,vcpupin622.cpuset=788,1684,\
vcpupin623.vcpu=623,vcpupin623.cpuset=788,1684,\
vcpupin624.vcpu=624,vcpupin624.cpuset=789,1685,\
vcpupin625.vcpu=625,vcpupin625.cpuset=789,1685,\
vcpupin626.vcpu=626,vcpupin626.cpuset=790,1686,\
vcpupin627.vcpu=627,vcpupin627.cpuset=790,1686,\
vcpupin628.vcpu=628,vcpupin628.cpuset=791,1687,\
vcpupin629.vcpu=629,vcpupin629.cpuset=791,1687,\
vcpupin630.vcpu=630,vcpupin630.cpuset=792,1688,\
vcpupin631.vcpu=631,vcpupin631.cpuset=792,1688,\
vcpupin632.vcpu=632,vcpupin632.cpuset=793,1690,\
vcpupin633.vcpu=633,vcpupin633.cpuset=793,1690,\
vcpupin634.vcpu=634,vcpupin634.cpuset=794,1690,\
vcpupin635.vcpu=635,vcpupin635.cpuset=794,1690,\
vcpupin636.vcpu=636,vcpupin636.cpuset=795,1691,\
vcpupin637.vcpu=637,vcpupin637.cpuset=795,1691,\
vcpupin638.vcpu=638,vcpupin638.cpuset=813,1709,\
vcpupin639.vcpu=639,vcpupin639.cpuset=813,1709,\
vcpupin640.vcpu=640,vcpupin640.cpuset=814,1710,\
vcpupin641.vcpu=641,vcpupin641.cpuset=814,1710,\
vcpupin642.vcpu=642,vcpupin642.cpuset=815,1711,\
vcpupin643.vcpu=643,vcpupin643.cpuset=815,1711,\
vcpupin644.vcpu=644,vcpupin644.cpuset=816,1712,\
vcpupin645.vcpu=645,vcpupin645.cpuset=816,1712,\
vcpupin646.vcpu=646,vcpupin646.cpuset=817,1713,\
vcpupin647.vcpu=647,vcpupin647.cpuset=817,1713,\
vcpupin648.vcpu=648,vcpupin648.cpuset=818,1714,\
vcpupin649.vcpu=649,vcpupin649.cpuset=818,1714,\
vcpupin650.vcpu=650,vcpupin650.cpuset=819,1715,\
vcpupin651.vcpu=651,vcpupin651.cpuset=819,1715,\
vcpupin652.vcpu=652,vcpupin652.cpuset=820,1716,\
vcpupin653.vcpu=653,vcpupin653.cpuset=820,1716,\
vcpupin654.vcpu=654,vcpupin654.cpuset=821,1718,\
vcpupin655.vcpu=655,vcpupin655.cpuset=821,1718,\
vcpupin656.vcpu=656,vcpupin656.cpuset=822,1718,\
vcpupin657.vcpu=657,vcpupin657.cpuset=822,1718,\
vcpupin658.vcpu=658,vcpupin658.cpuset=823,1719,\
vcpupin659.vcpu=659,vcpupin659.cpuset=823,1719,\
vcpupin660.vcpu=660,vcpupin660.cpuset=841,1737,\
vcpupin661.vcpu=661,vcpupin661.cpuset=841,1737,\
vcpupin662.vcpu=662,vcpupin662.cpuset=842,1738,\
vcpupin663.vcpu=663,vcpupin663.cpuset=842,1738,\
vcpupin664.vcpu=664,vcpupin664.cpuset=843,1739,\
vcpupin665.vcpu=665,vcpupin665.cpuset=843,1739,\
vcpupin666.vcpu=666,vcpupin666.cpuset=844,1740,\
vcpupin667.vcpu=667,vcpupin667.cpuset=844,1740,\
vcpupin668.vcpu=668,vcpupin668.cpuset=845,1741,\
vcpupin669.vcpu=669,vcpupin669.cpuset=845,1741,\
vcpupin670.vcpu=670,vcpupin670.cpuset=846,1742,\
vcpupin671.vcpu=671,vcpupin671.cpuset=846,1742,\
vcpupin672.vcpu=672,vcpupin672.cpuset=847,1743,\
vcpupin673.vcpu=673,vcpupin673.cpuset=847,1743,\
vcpupin674.vcpu=674,vcpupin674.cpuset=848,1744,\
vcpupin675.vcpu=675,vcpupin675.cpuset=848,1744,\
vcpupin676.vcpu=676,vcpupin676.cpuset=849,1746,\
vcpupin677.vcpu=677,vcpupin677.cpuset=849,1746,\
vcpupin678.vcpu=678,vcpupin678.cpuset=850,1746,\
vcpupin679.vcpu=679,vcpupin679.cpuset=850,1746,\
vcpupin680.vcpu=680,vcpupin680.cpuset=851,1747,\
vcpupin681.vcpu=681,vcpupin681.cpuset=851,1747,\
vcpupin682.vcpu=682,vcpupin682.cpuset=869,1765,\
vcpupin683.vcpu=683,vcpupin683.cpuset=869,1765,\
vcpupin684.vcpu=684,vcpupin684.cpuset=870,1766,\
vcpupin685.vcpu=685,vcpupin685.cpuset=870,1766,\
vcpupin686.vcpu=686,vcpupin686.cpuset=871,1767,\
vcpupin687.vcpu=687,vcpupin687.cpuset=871,1767,\
vcpupin688.vcpu=688,vcpupin688.cpuset=872,1768,\
vcpupin689.vcpu=689,vcpupin689.cpuset=872,1768,\
vcpupin690.vcpu=690,vcpupin690.cpuset=873,1769,\
vcpupin691.vcpu=691,vcpupin691.cpuset=873,1769,\
vcpupin692.vcpu=692,vcpupin692.cpuset=874,1770,\
vcpupin693.vcpu=693,vcpupin693.cpuset=874,1770,\
vcpupin694.vcpu=694,vcpupin694.cpuset=875,1771,\
vcpupin695.vcpu=695,vcpupin695.cpuset=875,1771,\
vcpupin696.vcpu=696,vcpupin696.cpuset=876,1772,\
vcpupin697.vcpu=697,vcpupin697.cpuset=876,1772,\
vcpupin698.vcpu=698,vcpupin698.cpuset=877,1774,\
vcpupin699.vcpu=699,vcpupin699.cpuset=877,1774,\
vcpupin700.vcpu=700,vcpupin700.cpuset=878,1774,\
vcpupin701.vcpu=701,vcpupin701.cpuset=878,1774,\
vcpupin702.vcpu=702,vcpupin702.cpuset=879,1775,\
vcpupin703.vcpu=703,vcpupin703.cpuset=879,1775

Comment 10 Igor Mammedov 2022-03-16 10:43:29 UTC
this fixes, virt-install only by letting specify iommu option manually,
however virt-manager GUS is still not able to create VM with more than 255 cpus.

(testing with virt-manager-3.2.0-4)

FYI:
also with QEMU starting from 6.0 + RHEL8.4+ host kernel it may be possible
to create a valid VM config without interrupt remapping
(although iommu is still required for Windows and for Linux guest it depends on
guest kernel version (since RHEL8.4 guest, it can work without iommu present))

Comment 12 errata-xmlrpc 2022-05-10 13:54:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt-manager bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1862


Note You need to log in before you can comment on or make changes to this bug.