RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1970277 - Attaching a virtio disk ends with an IOThreads-related error
Summary: Attaching a virtio disk ends with an IOThreads-related error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: CentOS Stream
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: beta
: ---
Assignee: Peter Krempa
QA Contact: Han Han
URL:
Whiteboard:
: 1972480 (view as bug list)
Depends On:
Blocks: 1974096
TreeView+ depends on / blocked
 
Reported: 2021-06-10 08:26 UTC by Marcin Sobczyk
Modified: 2021-12-07 22:01 UTC (History)
12 users (show)

Fixed In Version: libvirt-7.5.0-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-12-07 21:57:54 UTC
Type: Bug
Target Upstream Version: 7.5.0
Embargoed:

Attachments (Terms of Use)
git bisect script (1.01 KB, application/x-shellscript)
2021-06-11 12:52 UTC, Han Han 		no flags Details
View All

Description Marcin Sobczyk 2021-06-10 08:26:27 UTC 
Description of problem:


Version-Release number of selected component (if applicable):
libvirt-7.4.0-1.el8s.x86_64


How reproducible:
Always.

Steps to Reproduce:
Attach a disk to a VM, example disk XML:

<disk device="disk" snapshot="no" type="file">
    <source file="/rhev/data-center/mnt/192.168.202.2:_exports_nfs_share1/225cef17-e432-4b57-9607-9fb2073dda9c/images/f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e/b7b81c1e-53ce-44ba-9151-af2291784593">
        <seclabel model="dac" relabel="no" type="none" />
    </source>
    <target bus="virtio" dev="vdb" />
    <serial>f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e</serial>
    <driver cache="none" error_policy="stop" io="threads" iothread="1" name="qemu" type="qcow2" />
    <alias name="ua-f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e" />
</disk>

Actual results:

'virDomainAttachDevice' call fails, relevant log fragment:

2021-06-10 08:16:17.361+0000: 12670: debug : virDomainAttachDevice:8325 : dom=0x7fac94005640, (VM: name=vm0, uuid=25f38807-7053-4f88-a23a-99673abac2b9), xml=<?xml version='1.0' encoding='utf-8'?>
<disk device="disk" snapshot="no" type="file">
    <source file="/rhev/data-center/mnt/192.168.202.2:_exports_nfs_share1/225cef17-e432-4b57-9607-9fb2073dda9c/images/f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e/b7b81c1e-53ce-44ba-9151-af2291784593">
        <seclabel model="dac" relabel="no" type="none" />
    </source>
    <target bus="virtio" dev="vdb" />
    <serial>f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e</serial>
    <driver cache="none" error_policy="stop" io="threads" iothread="1" name="qemu" type="qcow2" />
    <alias name="ua-f8c4371b-8335-4fb3-8ad4-2cea4ef35c3e" />
</disk>

2021-06-10 08:16:17.361+0000: 12670: debug : qemuDomainObjBeginJobInternal:845 : Starting job: job=modify agentJob=none asyncJob=none (vm=0x7facd80175e0 name=vm0, current job=none agentJob=none async=none)
2021-06-10 08:16:17.361+0000: 12670: debug : qemuDomainObjBeginJobInternal:892 : Started job: modify (async=none vm=0x7facd80175e0 name=vm0)
2021-06-10 08:16:17.362+0000: 12670: error : qemuValidateDomainDeviceDefDiskIOThreads:2418 : unsupported configuration: IOThreads only available for virtio pci and virtio ccw disk

Expected results:
The disk attachment succeeds.

Additional info:
Worked in libvirt 7.0.
Comment 1 Han Han 2021-06-11 08:20:12 UTC 
Reproduced on v7.4.0-32-ge76ec0fe65 qemu-6.0.0-1.fc35.x86_64
when attach a disk with disk xml:
<disk device="disk" snapshot="no" type="file">
    <source file="/tmp/vdb">
    </source>
    <target bus="virtio" dev="vdb" />
    <driver io="threads" iothread="1" name="qemu" type="raw" />
</disk>
Comment 2 Han Han 2021-06-11 12:52:14 UTC 
Created attachment 1790215 [details]
git bisect script

I write a script for git-bisect and then I found out the first bad commit:

3df66f27a606c5c0f27958edf7bfaaf7dcc71264 is the first bad commit
commit 3df66f27a606c5c0f27958edf7bfaaf7dcc71264
Author: Peter Krempa <pkrempa>
Date:   Fri Apr 30 16:19:15 2021 +0200

    qemu: Move iothread and s390 address validation for disk devices into the validator
    
    The "machine-loadparm-multiple-disks-nets-s390" case now requires the
    QEMU_CAPS_CCW feature to pass validation.
    
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>
    Reviewed-by: Pavel Hrdina <phrdina>

 src/qemu/qemu_command.c  | 50 ------------------------------------------------
 src/qemu/qemu_validate.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++
 tests/qemuxml2xmltest.c  |  2 +-
 3 files changed, 50 insertions(+), 51 deletions(-)
bisect run success
Comment 3 Sandro Bonazzola 2021-06-14 08:27:45 UTC 
This is affecting oVirt 4.4.6 release based on CentOS 8 Stream.
Comment 4 Peter Krempa 2021-06-14 08:38:03 UTC 
(In reply to Sandro Bonazzola from comment #3)
> This is affecting oVirt 4.4.6 release based on CentOS 8 Stream.

Note that this is filed as a rhel-9 bug.

Regardless, it will be fixed upstream in the next libvirt release. Patch is already posted for review.
Comment 5 Nir Soffer 2021-06-14 13:26:21 UTC 
Peter, do we have an estimate when the fix will be available in Centos Stream?

For next oVirt release, we have a patch requiring livirt < 7.4.0:
https://gerrit.ovirt.org/c/vdsm/+/115193

because there several issues with this libvirt version. We need to know
when fixed version will be released to update our dependencies.
Comment 6 Peter Krempa 2021-06-14 13:28:55 UTC 
AFAIK centos stream is/will be updated regularly to the latest release version, so I expect it to be updated few days after the next upstream release of libvirt.
Comment 7 Han Han 2021-06-16 03:34:51 UTC 
Tested on libvirt v7.4.0-181-g9a51edebf8 qemu-6.0.0-5.fc35.x86_64 as comment1. PASSED
Comment 8 Han Han 2021-06-16 03:36:07 UTC 
*** Bug 1972480 has been marked as a duplicate of this bug. ***
Comment 9 Peter Krempa 2021-06-16 06:45:15 UTC 
Fixed upstream:

commit c29bb0fbb623af7994925bccad04060c5e27b64b
Author: Peter Krempa <pkrempa>
Date:   Thu Jun 10 16:43:36 2021 +0200

    qemu: validate: Don't check bus type in qemuValidateDomainDeviceDefDiskIOThreads
    
    IOThreads are supported with all 3 currently supported buses which can
    have virtio devices (PCI, CCW, MMIO) , so there's no need for this check.
    
    Additionally this check was buggy in the current location as on e.g.
    hotplug cases the address may not yet be assigned for the disk and thus
    a bogus error would be printed.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1970277
    Signed-off-by: Peter Krempa <pkrempa>
    Reviewed-by: Ján Tomko <jtomko>

v7.4.0-161-gc29bb0fbb6
Comment 10 Han Han 2021-06-17 02:42:24 UTC 
Covered by iothread_negative_test_disk_attach_delete_without_detach
Comment 14 Han Han 2021-07-26 07:05:45 UTC 
rhel.cpu.iothread.negative_test.disk_attach.delete_without_detach  PASSED on libvirt-7.5.0-1.el9.x86_64
Note You need to log in before you can comment on or make changes to this bug.