Bug 1270442

Summary: the behaviour of virsh command "blkdeviotune" has obscure logics but not mentioned in manual or libvirt.org doc.
Product: Red Hat Enterprise Linux 7 Reporter: yisun
Component: doc-Virtualization_Deployment_and_Administration_GuideAssignee: Jiri Herrmann <jherrman>
Status: CLOSED WONTFIX QA Contact:
Severity: low Docs Contact:
Priority: medium    
Version: 7.2CC: dyuan, jsuchane, mzhan, rbalakri, rhel-docs, xuzhang, yafu
Target Milestone: rcKeywords: Documentation
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-03 18:21:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description yisun 2015-10-10 07:01:41 UTC
Description of problem:
the behaviour of virsh command "blkdeviotune" has obscure logic but not mentioned in manual or libvirt.org doc. 

Version-Release number of selected component (if applicable):
libvirt-1.2.17-13.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
(in additional info, there is the original mail thread talking about this)

1.set total-bytes-sec / read-bytes-sec / write-bytes-sec will affect total-bytes-sec-max /read-bytes-sec-max /write-bytes-sec-max /total-iops-sec-max / read-iops-sec-max / write-iops-sec-max, and the affected value not displayed in domain's xml.

2. When set the xxxx_max value, it's not displayed in output of "virsh blkdeviotune <domain>", but it's in domain's xml


3. The relationship between [total-bytes-sec / read-bytes-sec / write-bytes-sec / total-iops-sec / read-iops-sec / write-iops-sec] and [total-bytes-sec-max /read-bytes-sec-max /total-iops-sec-max / total-iops-sec-max / read-iops-sec-max / write-iops-sec-max] is ambiguous. IMO, first group is a kind of speed limitation, so what's the meaning of the second group? They are limitations to limitations?

Actual results:
Above 3 questions are not described in libvirt manual or libvirt.org 

Expected results:
we should describe the logics clearly in our documents so user can have a better understanding than guessing the logics. 

Additional info:
============== original mail thread ==============
On 14.05.2015 07:54, Yi Sun wrote:
> Hi Michal & Daniel,
> I am testing the new options for blkdeviotune in virsh command and feel confused. Here are my questions, could you please spend a little time to deal with it, thanks in advance.
>
> 1. set total-bytes-sec / read-bytes-sec / write-bytes-sec will affect total-bytes-sec-max /read-bytes-sec-max /write-bytes-sec-max /total-iops-sec-max / read-iops-sec-max / write-iops-sec-max
> 1.1 But why the rate is 10:1?
> [root@localhost ~]# virsh blkdeviotune vm hdb
> total_bytes_sec: 0
> read_bytes_sec : 0
> write_bytes_sec: 0
> total_iops_sec : 0
> read_iops_sec  : 0
> write_iops_sec : 0
> total_bytes_sec_max: 0
> read_bytes_sec_max: 0
> write_bytes_sec_max: 0
> total_iops_sec_max: 0
> read_iops_sec_max: 0
> write_iops_sec_max: 0
> size_iops_sec  : 0
>
> [root@localhost ~]# virsh blkdeviotune vm hdb --total-bytes-sec 1000
>
> [root@localhost ~]# virsh blkdeviotune vm hdb
> total_bytes_sec: 1000
> read_bytes_sec : 0
> write_bytes_sec: 0
> total_iops_sec : 0
> read_iops_sec  : 0
> write_iops_sec : 0
> total_bytes_sec_max: 100    <== set total_bytes_sec: 1000 will cause a total_bytes_sec_max: 100, why?
> read_bytes_sec_max: 0
> write_bytes_sec_max: 0
> total_iops_sec_max: 0
> read_iops_sec_max: 0
> write_iops_sec_max: 0
> size_iops_sec  : 0

This is a result of a qemu behaviour. The corresponding qemu code can is
in throttle_fix_bucket() in util/throttle.c:

http://git.qemu.org/?p=qemu.git;a=blob;f=util/throttle.c;h=f976ac7de585da8683811b1275001cf73c71c636;hb=HEAD#l294

So setting an average, but not setting the max (i.e. setting
total-bytes-sec but not total-bytes-sec-max) will result in qemu setting
max to the 1/10 of avg. This is a result of the throttling algorithm
used in qemu. It just needs both.

> 1.2. Why the xxxx_max is not set in domain's xml file?
> [root@localhost ~]# virsh dumpxml vm | grep total_bytes -a2
>       <target dev='hdb' bus='ide'/>
>       <iotune>
>         <total_bytes_sec>1000</total_bytes_sec>
>       </iotune>
>       <alias name='ide0-0-1'/>

I can see why this causes confusion, but I think we should not pollute
domain XML with qemu internal implementation. They may change the way
how default for max is computed and domain XML will be inaccurate at
that point.

>
> 2. When set the xxxx_max value, why it's not displayed in followed up "virsh blkdeviotune <domain>", but it's in domain's xml
> [root@localhost ~]# virsh blkdeviotune vm hdb --read-bytes-sec-max 1000
>
> [root@localhost ~]# virsh blkdeviotune vm hdb
> total_bytes_sec: 0
> read_bytes_sec : 0
> write_bytes_sec: 0
> total_iops_sec : 0
> read_iops_sec  : 0
> write_iops_sec : 0
> total_bytes_sec_max: 0
> read_bytes_sec_max: 0  <==== nothing changed here
> write_bytes_sec_max: 0
> total_iops_sec_max: 0
> read_iops_sec_max: 0
> write_iops_sec_max: 0
> size_iops_sec  : 0
> # virsh dumpxml vm | grep read_bytes
>         <read_bytes_sec_max>1000</read_bytes_sec_max>   <=== it shows up here
>

Again, this is result of qemu implementation.

> 3. what is the relationship between [total-bytes-sec / read-bytes-sec / write-bytes-sec / total-iops-sec / read-iops-sec / write-iops-sec] and [total-bytes-sec-max /read-bytes-sec-max /total-iops-sec-max / total-iops-sec-max / read-iops-sec-max / write-iops-sec-max]? IMO, first group is a kind of speed limitation, so what's the meaning of the second group? They are limitations to limitations?
>

They are basically bursts. The throttle algorithm used in qemu is the
leaky bucket. It uses two variables: avg and max. So while the first set
refers to the avg, the other set refers to the max.

Michal
========================================

Comment 1 Jaroslav Suchanek 2015-11-26 15:05:06 UTC
Laura, can you please help us with this one? Thanks.

Comment 4 Jaroslav Suchanek 2016-04-18 11:33:54 UTC
Jiri,

this basically relies on your decision. If you think that the disk I/O throttling section should include more precise description then I think comment 0 includes all necessary information. From my POV the details are not necessary.

For your information, manual page contains this info:

If no limit is specified, it will query current I/O limits setting.  Otherwise, alter the limits with these flags: --total-bytes-sec specifies total throughput limit in bytes per second. --read-bytes-sec specifies read throughput limit in bytes per second.  --write-bytes-sec specifies write throughput limit in bytes per second.  --total-iops-sec specifies total I/O operations limit per second.  --read-iops-sec specifies read I/O operations limit per second.  --write-iops-sec specifies write I/O operations limit per second.  --total-bytes-sec-max specifies maximum total throughput limit in bytes per second. --read-bytes-sec-max specifies maximum read throughput limit in bytes per second.  --write-bytes-sec-max specifies maximum write throughput limit in bytes per second.  --total-iops-sec-max specifies maximum total I/O operations limit per second. --read-iops-sec-max specifies maximum read I/O operations limit per second.  --write-iops-sec-max specifies maximum write I/O operations limit per second.  --size-iops-sec specifies size I/O operations limit per second.

Comment 5 Jiri Herrmann 2016-06-03 18:21:07 UTC
Thank you for the intel everyone!

Nevertheless, I'd like to prevent more bloat to the documentation, and from Michal's response in comment 0, this information does not seem to help the users achieve anything specific, but would rather just add duplicated reference material. Also, as mentioned by Jarda in comment 4, there is additional information on the man page, to which the guide does refer:

http://jenkinscat.gsslab.pnq.redhat.com:8080/job/doc-Red_Hat_Enterprise_Linux-7-Virtualization_Deployment_and_Administration_Guide%20%28html-single%29/lastStableBuild/artifact/tmp/en-US/html-single/index.html#sect-Managing_guest_virtual_machines_with_virsh-Disk_IO_throttling

Therefore, I'll be closing this BZ. Yi, if you feel like the blkdeviotune behaviours you discussed should be documented nonetheless, feel free to reopen this and let me know.