Bug 1336564
| Summary: | Extend <iotune> for QEMU group I/O throttling | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Stefan Hajnoczi <stefanha> |
| Component: | libvirt | Assignee: | John Ferlan <jferlan> |
| Status: | CLOSED ERRATA | QA Contact: | yisun |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.2 | CC: | dyuan, jdenemar, jferlan, lmen, ngu, pzhang, rbalakri, stefanha, xuzhang, yisun |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-3.1.0-1.el7 | Doc Type: | Enhancement |
| Doc Text: |
Feature: Add support for QEMU group I/O throttling
Reason: This prevents end-users from circumventing a hosting provider's throttling policy by splitting 1 large drive into N small drives and getting N times the normal throttling quota.
Result: A new domain <disk> <iotune> subelement <group_name> will define the group name to be used from libvirt.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-01 17:09:12 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Stefan Hajnoczi
2016-05-16 21:48:29 UTC
Note the feature is not present in RHEL 7.2. It's a new feature that became available in upstream QEMU 2.4. Stefan, What should we do for burst_length? Should <iotune> also be exended? Move to consideration for 7.4 (In reply to Gu Nini from comment #2) > What should we do for burst_length? Should <iotune> also be exended? Yes, I have created a new BZ for burst_length: bz#1349898 Looks like I forget to update this when I posted patches... In any case, here's the most recent v2 posting: http://www.redhat.com/archives/libvir-list/2016-November/msg01022.html Changes are now pushed: $ git describe fbec5b949ba61cdf67dab2c70a385d3062306b83 v2.5.0-21-gfbec5b9 $ Few more commits for this are v3.0.0-75-ge9d75343d4cf^..v3.0.0-78-g862bea96d9d9 and v3.0.0-100-geae7cfd42d44^..v3.0.0-101-gbb5d6379a009:
commit bb5d6379a009b2806936f341523f80c9fabbcd4c
Author: Martin Kletzander <mkletzan>
Date: Tue Jan 24 15:50:17 2017 +0100
qemu: Don't lose group_name
commit eae7cfd42d442ccb1b4d086a1ed792c27e4cdc54
Author: Martin Kletzander <mkletzan>
Date: Mon Jan 30 20:37:48 2017 +0100
conf: Add virDomainDiskSetBlockIOTune
commit 862bea96d9d9c75ba4cb6173fe484eb3d52d3540
Author: Martin Kletzander <mkletzan>
Date: Wed Jan 25 09:38:09 2017 +0100
virsh: Use consistent naming for blkdeviotune options
commit a20e8bcad5c17380b52cc3c2469660eb63a78304
Author: Martin Kletzander <mkletzan>
Date: Wed Jan 25 09:35:47 2017 +0100
virsh: Actually make blkdeviotune --group_name work
commit 87ee705183241a42ffd36d9f5d3934cacf91c343
Author: Martin Kletzander <mkletzan>
Date: Thu Jan 26 15:13:27 2017 +0100
qemu: Miscellaneous Block I/O tune cleanups
commit e9d75343d4cf552575a3b305fa00a36ee71e9309
Author: Martin Kletzander <mkletzan>
Date: Tue Jan 24 15:50:00 2017 +0100
qemu: Only set group_name when actually requested
hit a issue seems related to this bug Version: libvirt-3.0.0-1.el7.x86_64 Steps to Reproduce 1. having a running vm # virsh list Id Name State ---------------------------------------------------- 3 avocado-vt-vm2 running 2. VM having a virtual disk - vda # virsh domblklist avocado-vt-vm2 Target Source ------------------------------------------------ vda /var/lib/libvirt/images/RHEL-7.2-x86_64-latest.qcow2 3. use blkdeviotune to set --read-bytes-sec to vm's vda # virsh blkdeviotune avocado-vt-vm2 --live --device vda --read-bytes-sec 4096 error: Unable to change block I/O throttle error: internal error: argument key 'group' must not have null value expected result: blkdeviotune should work, group is not a mandatory parameter as virsh manual says. actual result: failed See comment 6 - Martin found a few issues which would be fixed in libvirt 3.1. I'll put this back in POST so that Jiri would mark it when he builds 3.1 The new parameters will trigger a libvirtd crash when setting together with other parameters steps: 1. prepare a vm with blk devs ## virsh domblklist r Target Source ------------------------------------------------ vda /var/lib/libvirt/images/r.qcow2 vdb /var/lib/libvirt/images/luks.disk 2. let's use vdb as test object ## virsh blkdeviotune r vdb total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 3. set group name and some other parameters together, such as --total-bytes-sec ## virsh blkdeviotune r vdb --group-name goodname --total-bytes-sec 100 error: Disconnected from qemu:///system due to I/O error error: Unable to change block I/O throttle error: End of file while reading data: Input/output error <==== libvirtd crashed~~ ( if setting the parameters separately, everything'll be fine - `virsh blkdeviotune r vdb --group-name goodname; virsh blkdeviotune r vdb --total-bytes-sec 100`) I put the gdb backtrace in pastebin: http://pastebin.test.redhat.com/489354 http://pastebin.test.redhat.com/489354 I think comment 10 is fixed by bz 1455510. It's not clear from the comment which libvirt version you used to test... Can this be retried with: Version-Release number of selected component: libvirt-3.2.0-7.el7.x86_64 (In reply to John Ferlan from comment #11) > I think comment 10 is fixed by bz 1455510. > > It's not clear from the comment which libvirt version you used to test... > > Can this be retried with: > > Version-Release number of selected component: > libvirt-3.2.0-7.el7.x86_64 Due to the time I added comment 10, it should be 3.2.0-6, I didn't notice there was another bug tracking this... Should be same problem, so move this back to ON_QA and will test it soon. thx Verified on libvirt-3.2.0-7.el7.x86_64 1. do not explicitly setting group name, but setting other parameters ## virsh start r Domain r started ## virsh blkdeviotune r vda total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 ## virsh blkdeviotune r vda --total-bytes-sec 100 ## virsh blkdeviotune r vda total_bytes_sec: 100 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 10 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name :*** drive-virtio-disk0 *** total_bytes_sec_max_length: 1 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 <=== according to https://bugzilla.redhat.com/show_bug.cgi?id=1455834#c1 this is expected. 2. explicitly setting group name ## virsh blkdeviotune r vda --group-name goodname ## virsh blkdeviotune r vda total_bytes_sec: 100 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 10 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : *** goodname *** total_bytes_sec_max_length: 1 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 ## virsh dumpxml r | grep goodname -a2 <iotune> <total_bytes_sec>100</total_bytes_sec> <group_name>goodname</group_name> </iotune> ## virsh dumpxml r --inactive| grep goodname -a2 3. start a vm with existing group name setting ## virsh blkdeviotune r vda --group-name goodname --config ## virsh destroy r Domain r destroyed ## virsh dumpxml r --inactive| grep goodname -a2 <iotune> <group_name>goodname</group_name> </iotune> ## virsh start r error: Failed to start domain r error: unsupported configuration: group_name can be configured only together with settings <==== clear error message, acceptable 4. start a vm with existing group name and read_bytes_sec setting ## virsh dumpxml r --inactive | grep iotune -a3 ... <iotune> <read_bytes_sec>200</read_bytes_sec> <group_name>goodname</group_name> </iotune> ## virsh blkdeviotune r vda total_bytes_sec: 0 read_bytes_sec : 200 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 20 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : *** goodname *** total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 1 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 5. check --config option works fine for --group-name ## virsh blkdeviotune r vda --group-name goodname --read-iops-sec 100 --config ## virsh dumpxml r | grep iotune -a3 <=== nothing as expected ## virsh dumpxml r --inactive | grep iotune -a3 ... <iotune> <read_iops_sec>100</read_iops_sec> <group_name>goodname</group_name> </iotune> ... ## virsh blkdeviotune r vda total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 0 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 0 write_iops_sec_max: 0 size_iops_sec : 0 group_name : total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 0 write_iops_sec_max_length: 0 ## virsh destroy r; virsh start r Domain r destroyed Domain r started ## virsh blkdeviotune r vda total_bytes_sec: 0 read_bytes_sec : 0 write_bytes_sec: 0 total_iops_sec : 0 read_iops_sec : 100 write_iops_sec : 0 total_bytes_sec_max: 0 read_bytes_sec_max: 0 write_bytes_sec_max: 0 total_iops_sec_max: 0 read_iops_sec_max: 10 write_iops_sec_max: 0 size_iops_sec : 0 group_name : *** goodname *** total_bytes_sec_max_length: 0 read_bytes_sec_max_length: 0 write_bytes_sec_max_length: 0 total_iops_sec_max_length: 0 read_iops_sec_max_length: 1 write_iops_sec_max_length: 0 (some params combination tests were carried out in https://bugzilla.redhat.com/show_bug.cgi?id=1455510#c4) Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 |