Description of problem: Use iothreadadd to add an iothread ID to a running guest, and then print domstats --iothread to check. The iothread id is different from the one set in iothreadadd. This is a regression issue. Version-Release number of selected component (if applicable): libvirt-5.9.0-4.module+el8.2.0+4836+a8e32ad7.x86_64 qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64 How reproducible: 100% Steps to Reproduce: 1. Start a guest and add an iothread with ID 2 # virsh iothreadadd vm2 2 --live # virsh dumpxml vm2 <domain type='kvm' id='4'> <name>vm2</name> ... <vcpu placement='static'>1</vcpu> <iothreads>1</iothreads> <iothreadids> <iothread id='2'/> </iothreadids> 2. Check in domstats # virsh domstats vm2 --iothread Domain: 'vm2' iothread.count=1 iothread.0.poll-max-ns=32768 <=== should be iothread.2.poll-max-ns=32768 iothread.0.poll-grow=0 iothread.0.poll-shrink=0 3. Output from libvirt-5.6.0-7.module+el8.2.0+4673+ff4b3b61.x86_64 # virsh domstats vm2 --iothread Domain: 'vm2' iothread.count=1 iothread.2.poll-max-ns=32768 iothread.2.poll-grow=0 iothread.2.poll-shrink=0 Actual results: See above Expected results: iothreadid in domstats should be consistent with the one in guest xml. Additional info: snippet in libvirtd log: 2019-11-28 09:00:54.815+0000: 25912: debug : virDomainAddIOThread:7741 : dom=0x7ff1e400a620, (VM: name=vm2, uuid=3da05f15-6445-4249-923b-79b2042c5c8f), iothread_id=2, flags=0x1 ... 2019-11-28 09:00:54.815+0000: 25912: info : qemuMonitorSend:996 : QEMU_MONITOR_SEND_MSG: mon=0x7ff19408c3a0 msg={"execute":"object-add","arguments":{"qom-type":"iothread","id":"iothread2"},"id":"libvirt-24"} fd=-1 2019-11-28 09:00:54.815+0000: 25908: info : qemuMonitorIOWrite:453 : QEMU_MONITOR_IO_WRITE: mon=0x7ff19408c3a0 buf={"execute":"object-add","arguments":{"qom-type":"iothread","id":"iothread2"},"id":"libvirt-24"} len=97 ret=97 errno=0 2019-11-28 09:00:54.816+0000: 25908: debug : qemuMonitorJSONIOProcessLine:221 : Line [{"return": {}, "id": "libvirt-24"}] 2019-11-28 09:00:54.816+0000: 25908: info : qemuMonitorJSONIOProcessLine:241 : QEMU_MONITOR_RECV_REPLY: mon=0x7ff19408c3a0 reply={"return": {}, "id": "libvirt-24"} 2019-11-28 09:00:54.816+0000: 25912: debug : qemuMonitorGetIOThreads:4109 : iothreads=0x7ff1efffe6e8 2019-11-28 09:00:54.816+0000: 25912: debug : qemuMonitorGetIOThreads:4111 : mon:0x7ff19408c3a0 vm:0x7ff19c202d80 fd:22 2019-11-28 09:00:54.816+0000: 25912: info : qemuMonitorSend:996 : QEMU_MONITOR_SEND_MSG: mon=0x7ff19408c3a0 msg={"execute":"query-iothreads","id":"libvirt-25"} fd=-1 2019-11-28 09:00:54.816+0000: 25908: info : qemuMonitorIOWrite:453 : QEMU_MONITOR_IO_WRITE: mon=0x7ff19408c3a0 buf={"execute":"query-iothreads","id":"libvirt-25"} len=49 ret=49 errno=0 2019-11-28 09:00:54.816+0000: 25908: debug : qemuMonitorJSONIOProcessLine:221 : Line [{"return": [{"poll-shrink": 0, "thread-id": 3385, "poll-grow": 0, "poll-max-ns": 32768, "id": "iothread2"}], "id": "libvirt-25"}] 2019-11-28 09:00:54.816+0000: 25908: info : qemuMonitorJSONIOProcessLine:241 : QEMU_MONITOR_RECV_REPLY: mon=0x7ff19408c3a0 reply={"return": [{"poll-shrink": 0, "thread-id": 3385, "poll-grow": 0, "poll-max-ns": 32768, "id": "iothread2"}], "id": "libvirt-25"} 2019-11-28 09:00:54.816+0000: 25912: debug : qemuDomainObjExitMonitorInternal:8748 : Exited monitor (mon=0x7ff19408c3a0 vm=0x7ff19c202d80 name=vm2)
(In reply to Dan Zheng from comment #0) > Description of problem: > Use iothreadadd to add an iothread ID to a running guest, and then print > domstats --iothread to check. The iothread id is different from the one set > in iothreadadd. This is a regression issue. Could you please specify in which version the output was different? > Version-Release number of selected component (if applicable): > libvirt-5.9.0-4.module+el8.2.0+4836+a8e32ad7.x86_64 > qemu-kvm-4.2.0-1.module+el8.2.0+4793+b09dd2fb.x86_64 > > > How reproducible: > > 100% > > Steps to Reproduce: > 1. > Start a guest and add an iothread with ID 2 > # virsh iothreadadd vm2 2 --live > # virsh dumpxml vm2 > <domain type='kvm' id='4'> > <name>vm2</name> > ... > <vcpu placement='static'>1</vcpu> > <iothreads>1</iothreads> > <iothreadids> > <iothread id='2'/> > </iothreadids> > > 2. Check in domstats > # virsh domstats vm2 --iothread > > Domain: 'vm2' > iothread.count=1 > iothread.0.poll-max-ns=32768 <=== should be iothread.2.poll-max-ns=32768 > iothread.0.poll-grow=0 > iothread.0.poll-shrink=0 No. This output is correct. The number here is used as an array index and thus must start from zero. That is not the iothread id. The iothread ID should be a separate field such as 'iothread.0.id=2'.
Okay, so I found where it was changed: commit 2ccb5335dc41fc0dfedb2ed0ec387decb7fb5299 Author: Peter Krempa <pkrempa> Date: Thu Sep 19 10:36:28 2019 +0200 qemu: Use virTypedParamList in the bulk stats gathering functions The bulk stats functions are specific as they pass around the list into many sub-functions and also a substantial amount of the entries uses formatted names for indexing purposes. This makes them ideal to be converted to the new virTypedParamList helpers. Unfortunately given how the functions are used this requires a big-bang rewrite of all of the calls to add entries to the parameter list. Given that a substantial simplification is achieved as well as a pretty significant change to the original code is required some macros which were used only sporadically were replaced by inline calls rather than tweaking the macros first and deleting them later. v5.7.0-242-g2ccb5335dc The change was not intentional as per the commit message above. I still think that the current state is as the API was intended if the 'id' will be added as a separate field.
Thinking about it a bit more I chose to fix it to the state as it was before since that state was extensively documented despite deviating from the design of other stat types. https://www.redhat.com/archives/libvir-list/2019-November/msg01442.html
Fixed upstream: commit 21986f5047ba0e4d11268e2fcd9756ddabc78705 (HEAD -> master, origin/master, origin/HEAD) Author: Peter Krempa <pkrempa> Date: Fri Nov 29 09:11:15 2019 +0100 qemu: Fix indexes in statistics of iothreads In commit 2ccb5335dc4 I've refactored how we fill the typed parameters for domain statistics. The commit introduced a regression in the formating of stats for IOthreads by using the array index to label the entries as it's common for all other types of statistics rather than the iothread IDs used for iothreads. Since only the design of iothread deviates from the common approach used in all other statistic types this was not caught. v5.10.0-rc2-2-g21986f5047
Verified this bug on libvirt-5.10.0-1.module+el8.2.0+5135+ed3b2489.x86_64. Version: libvirt-5.10.0-1.module+el8.2.0+5135+ed3b2489.x86_64 qemu-kvm-4.2.0-2.module+el8.2.0+5135+ed3b2489.x86_64 kernel-4.18.0-160.el8.x86_64 Steps: 1. Prepare a running VM and configure iothread for the VM # virsh domstate test820 running # virsh iothreadadd test820 1 # virsh iothreadadd test820 2 # virsh iothreadadd test820 3 # virsh iothreadinfo test820 IOThread ID CPU Affinity ----------------------------- 3 0-31 1 0-31 2 0-31 # virsh iothreadpin test820 1 3 # virsh iothreadpin test820 3 9-17 # virsh iothreadinfo test820 IOThread ID CPU Affinity ----------------------------- 3 9-17 1 3 2 0-31 2. Check iothread info through "virsh domstats" # virsh domstats test820 --iothread Domain: 'test820' iothread.count=3 iothread.3.poll-max-ns=32768 iothread.3.poll-grow=0 iothread.3.poll-shrink=0 iothread.1.poll-max-ns=32768 iothread.1.poll-grow=0 iothread.1.poll-shrink=0 iothread.2.poll-max-ns=32768 iothread.2.poll-grow=0 iothread.2.poll-shrink=0 3. Delete some iothread and check the iothread info again # virsh iothreaddel test820 1 # virsh iothreaddel test820 2 # virsh domstats test820 --iothread Domain: 'test820' iothread.count=1 iothread.3.poll-max-ns=32768 iothread.3.poll-grow=0 iothread.3.poll-shrink=0 All the result is as expected above.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2017