Bug 1414630

Summary: The throttling option value of FOO_max lower than FOO.eg :bps_max/iops_max cannot be lower than bps/iops
Product: Red Hat Enterprise Linux 7 Reporter: Yongxue Hong <yhong>
Component: qemu-kvm-rhevAssignee: Stefan Hajnoczi <stefanha>
Status: CLOSED NOTABUG QA Contact: Gu Nini <ngu>
Severity: low Docs Contact:
Priority: medium    
Version: 7.4CC: chayang, coli, hhuang, juzhang, michen, ngu, qzhang, virt-maint, xuhan, xuma
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-17 13:02:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yongxue Hong 2017-01-19 04:49:19 UTC
Description of problem:
The throttling value of FOO is defined in qemu command,but show FOO_max lower than FOO after input info block.

Version-Release number of selected component (if applicable):
Host kernel:3.10.0-543.el7.ppc64le 
qemu: qemu-kvm-rhev-2.8.0-2.el7 
slof: SLOF-20160223-6.gitdbbfda4.el7 
Guest kernel:3.10.0-543.el7.ppc64

How reproducible:
100%

Steps to Reproduce:
1.create two disk image
eg:
qemu-img  create -f raw RHEL7-8380-20G.raw 20G
qemu-img  create -f raw RHEL7-8380-2-16G.raw 16G
2.boot a guest with bps,but not set the value of bps_max.
/usr/libexec/qemu-kvm \
-name RHEL7-8380 \
-M pseries-rhel7.4.0 \
-m 64G \
-smp 4 \
-boot menu=on,order=c \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=06 \
-drive file=/home/hyx/os/RHEL-7.3-20161019.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,id=image0 \
-device scsi-cd,id=scsi-cd0,drive=image0,channel=0,scsi-id=0,lun=0,bootindex=1 \
-drive file=/home/hyx/image/RHEL7-8380-2-16G.raw,if=none,media=disk,format=raw,cache=none,bps=102400,iops=100,id=image2 \
-device scsi-hd,id=scsi-hd2,drive=image2,channel=0,scsi-id=0,lun=1 \
-drive file=/home/hyx/image/RHEL7-8380-20G.raw,if=none,media=disk,format=raw,id=image1 \
-device scsi-hd,id=scsi-hd1,drive=image1,channel=0,scsi-id=0,lun=2,bootindex=0 \
-netdev tap,id=hostnet0,script=/etc/qemu-ifup \
-device spapr-vlan,netdev=hostnet0,id=virtio-net-pci0,mac=70:e2:84:14:0e:03 \
-monitor stdio \
-serial unix:./sock0,server,nowait \
-qmp tcp:0:6666,server,nowait \
-device usb-tablet \
-vnc :0 
3. info block
(qemu) info block
image0 (#block166): /home/hyx/os/RHEL-7.3-20161019.0-Server-ppc64-dvd1.iso (raw, read-only)
    Removable device: locked, tray closed
    Cache mode:       writeback

image2 (#block316): /home/hyx/image/RHEL7-8380-2-16G.raw (raw)
    Cache mode:       writeback, direct
    I/O throttling:   bps=102400 bps_rd=0 bps_wr=0 bps_max=10240 bps_rd_max=0 bps_wr_max=0 iops=100 iops_rd=0 iops_wr=0 iops_max=10 iops_rd_max=0 iops_wr_max=0 iops_size=0 group=image2

image1 (#block562): /home/hyx/image/RHEL7-8380-20G.raw (raw)
    Cache mode:       writeback
4.set the value of bps_max same as the above info block show.
/usr/libexec/qemu-kvm \
-name RHEL7-8380 \
-M pseries-rhel7.4.0 \
-m 64G \
-smp 4 \
-boot menu=on,order=c \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=06 \
-drive file=/home/hyx/os/RHEL-7.3-20161019.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,id=image0 \
-device scsi-cd,id=scsi-cd0,drive=image0,channel=0,scsi-id=0,lun=0,bootindex=1 \
-drive file=/home/hyx/image/RHEL7-8380-2-16G.raw,if=none,media=disk,format=raw,cache=none,bps=102400,bps_max=10240,iops=100,id=image2 \
-device scsi-hd,id=scsi-hd2,drive=image2,channel=0,scsi-id=0,lun=1 \
-drive file=/home/hyx/image/RHEL7-8380-20G.raw,if=none,media=disk,format=raw,id=image1 \
-device scsi-hd,id=scsi-hd1,drive=image1,channel=0,scsi-id=0,lun=2,bootindex=0 \
-netdev tap,id=hostnet0,script=/etc/qemu-ifup \
-device spapr-vlan,netdev=hostnet0,id=virtio-net-pci0,mac=70:e2:84:14:0e:03 \
-monitor stdio \
-serial unix:./sock0,server,nowait \
-qmp tcp:0:6666,server,nowait \
-device usb-tablet \
-vnc :0 
5. boot guest

Actual results:
qemu-kvm:-drive file=/home/hyx/image/RHEL7-8380-2-16G.raw,if=none,media=disk,format=raw,cache=none,bps=102400,bps_max=10240,iops=100,id=image2: bps_max/iops_max cannot be lower than bps/iops.
the bps_max is lower than bps.

Expected results:
The value of bps_max should be equal to bps, if the max value is not defined in cli.

Additional info:

Comment 2 Stefan Hajnoczi 2017-02-17 13:02:38 UTC
The default value of bps_max is bps / 10 unless bps_max= was given on the command-line.

The following code explains why in util/throttle.c:throttle_fix_bucket():

/* The following is done to cope with the Linux CFQ block scheduler
 * which regroup reads and writes by block of 100ms in the guest.
 * When they are two process one making reads and one making writes cfq
 * make a pattern looking like the following:
 * WWWWWWWWWWWRRRRRRRRRRRRRRWWWWWWWWWWWWWwRRRRRRRRRRRRRRRRR
 * Having a max burst value of 100ms of the average will help smooth the
 * throttling
 */
min = bkt->avg / 10;
if (bkt->avg && !bkt->max) {
    bkt->max = min;
}

I'm not sure if this default burst value is effective, but this behavior was intentionally put into the code by its author.

I guess the aim is to guarantee that QEMU will process one full 100 ms CFQ time slice, even if the limit value exactly matches the rate of I/O generated by the guest.

Comment 3 Stefan Hajnoczi 2017-02-17 13:05:23 UTC
By the way, if there's a place where you'd like to see this documented just let me know.  The QEMU man page doesn't document bps_max, etc.

Comment 4 Gu Nini 2017-02-20 05:52:05 UTC
(In reply to Stefan Hajnoczi from comment #3)
> By the way, if there's a place where you'd like to see this documented just
> let me know.  The QEMU man page doesn't document bps_max, etc.

Stefan, why not document it in the QEMU man page?

Comment 5 Stefan Hajnoczi 2017-02-20 17:08:24 UTC
(In reply to Gu Nini from comment #4)
> (In reply to Stefan Hajnoczi from comment #3)
> > By the way, if there's a place where you'd like to see this documented just
> > let me know.  The QEMU man page doesn't document bps_max, etc.
> 
> Stefan, why not document it in the QEMU man page?

I have sent a QEMU patch to document all the -drive throttling options:
https://lists.nongnu.org/archive/html/qemu-devel/2017-02/msg04433.html