Bug 1433183
Summary: | libvirtd crashed when querying persistent blkdeviotune config with groupname set | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Fangge Jin <fjin> | ||||||
Component: | libvirt | Assignee: | Peter Krempa <pkrempa> | ||||||
Status: | CLOSED ERRATA | QA Contact: | jiyan <jiyan> | ||||||
Severity: | high | Docs Contact: | |||||||
Priority: | medium | ||||||||
Version: | 7.4 | CC: | dyuan, fjin, lmen, pkrempa, rbalakri, xuzhang, yafu, zpeng | ||||||
Target Milestone: | rc | ||||||||
Target Release: | --- | ||||||||
Hardware: | x86_64 | ||||||||
OS: | Linux | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | libvirt-3.2.0-6.el7 | Doc Type: | If docs needed, set a value | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2017-08-01 17:24:15 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Please also attach the backtrace of all threads if libvirtd crashes. I managed to reproduce it. The problem is that the code steals the pointer from the definition rather than copying it. The RPC layer then frees it. Fixed upstream: commit 4b57f765027c6e157a41a7445ec23359655be272 Author: Peter Krempa <pkrempa> Date: Fri Mar 17 08:43:27 2017 +0100 qemu: Don't steal pointers from 'persistentDef' in qemuDomainGetBlockIoTune While the code path that queries the monitor allocates a separate copy of the 'group_name' string the path querying the config would not copy it. The call to virTypedParameterAssign would then steal the pointer (without clearing it) and the RPC layer freed it. Any subsequent call resulted into a crash. Hi Peter I tested with latest build libvirt-3.2.0-6.virtcov.el7.x86_64, libvirtd crash when set --group-name and other blkdeviotune parameters together by virsh command: # virsh blkdeviotune vm1 vda --total-bytes-sec 2000 --group-name fffff error: Disconnected from qemu:///system due to I/O error error: Unable to change block I/O throttle error: End of file while reading data: Input/output error As another colleague have tried before, this problem also exists with build libvirt-3.2.0-5.virtcov.el7.x86_64 please post the backtrace Created attachment 1282140 [details] backtrace for comment 8 Indeed, there was another bug in the setter code path. Would you mind filing a bug for that too? The specific way to reproduce this is to set something other than group name along with it. The typed parameter containing the group name needs to be second or further in the list. I file a new Bug 1455510 for this problem Version-Release number of selected component: libvirt-3.2.0-7.el7.x86_64 qemu-kvm-rhev-2.9.0-7.el7.x86_64 Kernel-3.10.0-675.el7.x86_64 Steps to verify: 1.Prepare guest.xml with blkdeviotune group_name, prepare guest1.xml with blkdeviotune group_name and # cat guest.xml |grep "<disk" -A 8 <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/RHEL-7.4-x86_64-latest.qcow2'/> <target dev='vdb' bus='virtio'/> <iotune> <group_name>group1</group_name> </iotune> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> # cat guest1.xml |grep "<disk" -A 9 <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/RHEL-7.4-x86_64-latest.qcow2'/> <target dev='vdb' bus='virtio'/> <iotune> <group_name>group1</group_name> <total_bytes_sec>10000000</total_bytes_sec> </iotune> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> Note:reffer to the bug ,https://bugzilla.redhat.com/show_bug.cgi?id=1433180,"Don't allow setting 'group_name' alone", so the guest1.xml needs to be verified. 2.define a vm though the guest.xml and query blkdeviotune # virsh define guest.xml Domain guest defined from guest.xml # virsh start guest error: Failed to start domain guest error: unsupported configuration: group_name can be configured only together with settings # virsh blkdeviotune guest vdb total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : group1 total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 3.define a vm though the guest1.xml and query blkdeviotune # virsh define guest1.xml Domain guest1 defined from guest1.xml # virsh blkdeviotune guest vdb total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : group1 total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 |
Created attachment 1263943 [details] libvirtd log Description of problem: Prepare a guest with blkdeviotune groupname, query blkdeviotune when guest is shutoff or query blkdeviotune with --config when guest is running, libvirtd will crash. Version-Release number of selected component: libvirt-3.1.0-2.el7.x86_64 How reproducible: 100% Steps to Reproduce: 0. Prepare a guest xml with blkdeviotune group_name: # cat /tmp/guest.xml ... <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/libvirt/images/image-builder.qcow2'/> <target dev='vda' bus='virtio'/> <iotune> <group_name>group1</group_name> </iotune> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> ... 1. # virsh define /tmp/guest.xml 2. # while true; do virsh blkdeviotune guest vda ; sleep 2; done total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : group1 total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 error: Disconnected from qemu:///system due to I/O error error: Unable to get block I/O throttle parameters error: End of file while reading data: Input/output error total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : group1 total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 error: Disconnected from qemu:///system due to I/O error error: Unable to get block I/O throttle parameters error: End of file while reading data: Input/output error Actual results: libvirtd crashed when querying blkdeviotune Expected results: libvirtd doesn't crash Additional info: