Bug 770520
| Summary: | blkiotune set weight on total and virtio device together will cause libvirtd hang | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | weizhang <weizhan> |
| Component: | libvirt | Assignee: | Peter Krempa <pkrempa> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.3 | CC: | acathrow, ajia, dallan, dyuan, eblake, mzhan, pkrempa, rwu, veillard, zhpeng, zpeng |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-0.9.10-1.el6 | Doc Type: | Bug Fix |
| Doc Text: |
No Documentation needed
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2012-06-20 06:40:22 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The programming will enter into a infinite loop due to 'i != ndevices' is true forever in this case:
static int qemuDomainSetBlkioParameters(virDomainPtr dom,
virTypedParameterPtr params,
int nparams,
unsigned int flags)
{
......
for (i = 0; i < nparams; i++) {
int rc;
virTypedParameterPtr param = ¶ms[i];
......
for (i = 0; i < ndevices; i++) {
rc = virCgroupSetBlkioDeviceWeight(group,
devices[i].path,
devices[i].weight);
if (rc < 0) {
virReportSystemError(-rc,
_("Unable to set io device weight "
"for path %s"),
devices[i].path);
break;
}
}
if (i != ndevices) {
ret = -1;
continue;
}
......
commit 1a3f6608aa4c945bb3f392c25ff06b13f1dc5f30
Author: Eric Blake <eblake>
Date: Wed Dec 28 06:53:27 2011 -0700
qemu: fix inf-loop in blkio parameters
https://bugzilla.redhat.com/show_bug.cgi?id=770520
We had two nested loops both trying to use 'i' as the iteration
variable, which can result in an infinite loop when the inner
loop interferes with the outer loop. Introduced in commit 93ab585.
* src/qemu/qemu_driver.c (qemuDomainSetBlkioParameters): Don't
reuse iteration variable across two loops.
I test it with: libvirt-0.9.9-0rc1.el6.x86_64 qemu-kvm-0.12.1.2-2.209.el6_2.2.x86_64 Steps: 1. virsh blkiotune guest --device-weights /dev/sda,100 --weight 500 Results: it hangs forever. Check /var/log/libvirt/libvirtd.log: 2012-01-06 09:21:37.693+0000: 2294: warning : virKeepAliveTimer:182 : No response from client 0x234d270 after 5 keepalive messages in 30 seconds 2012-01-06 09:23:50.529+0000: 2294: error : virNetSocketReadWire:996 : End of file while reading data: Input/output error (*When press Ctrl+C to terminate Step 1*) Sorry, the PKGs are libvirt-0.9.9-0rc1.el6.x86_64 qemu-kvm-0.12.1.2-2.213.el6.x86_64 The issue still exists on libvirt-0.9.9-1.el6.x86_64, the root reason is a copy-paste error on previous patch, and I have committed a patch to fix the issue: https://www.redhat.com/archives/libvir-list/2012-January/msg00309.html Upstream fix for the typo in the previous patch:
commit d8d9b0e05844802d9f659f35e1c8a4653f32d5f2
Author: Alex Jia <ajia>
Date: Tue Jan 10 17:55:01 2012 +0800
qemu: fix a typo on qemuDomainSetBlkioParameters
It should be a copy-paste error, the result is programming will result in an
infinite loop again due to without iterating 'j' variable.
* src/qemu/qemu_driver.c: fix a typo on qemuDomainSetBlkioParameters.
verify with:
libvirt-0.9.10-1.el6.x86_64
qemu-kvm-0.12.1.2-2.229.el6.x86_64
step:
1.start the guest
#virsh start v1-clone
2.run command
#virsh blkiotune v1-clone --device-weights /dev/sda,100 --weight 500
no hang,the command finished w/o error
2. virsh dumpxml v1-clone
.........
<blkiotune>
<device>
<path>/dev/sda</path>
<weight>100</weight>
</device>
</blkiotune>
.........
3. #virsh blkiotune v1-clone
weight : 500
device_weight : /dev/sda,100
verification passed.
move to verified.
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
No Documentation needed
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-0748.html |
Description of problem: when do blkiotune with --weight and --device-weights on virtio device, libvirtd will hang, and on libvirtd.log there will be lots of error logs like "qemuDomainSetBlkioParameters:6131 : Unable to set io device weight for path /dev/vda: No such file or directory" and occupy a large number of disk size Version-Release number of selected component (if applicable): kernel-2.6.32-223.el6.x86_64 libvirt-0.9.8-1.el6.x86_64 qemu-kvm-0.12.1.2-2.213.el6.x86_64 How reproducible: 100% Steps to Reproduce: 1. start a guest with virtio disk # virsh dumpxml guest <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/kvm-rhel6u2-x86_64-new.img'/> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </disk> 2. Do command # virsh blkiotune guest --device-weights /dev/vda,100 --weight 500 3. Check /var/log/libvirt/libvirtd.log Actual results: step 2 will hang and there are lots of error info continue writing on libvirtd.log Expected results: It may works well Additional info: