Bug 1151957

Summary: Throughput limitaion is not enforced using blkdeviotune
Product: Red Hat Enterprise Linux 7 Reporter: Elad <ebenahar>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED WORKSFORME QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 7.0CC: acanan, dfediuck, dyuan, ebenahar, gchaplik, rbalakri, rgolan, xuzhang, yanyang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-03 05:22:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 906938    
Attachments:
Description Flags
/var/log/messages and /var/log/vdsm/vdsm.log none

Description Elad 2014-10-13 07:50:36 UTC
Created attachment 946266 [details]
/var/log/messages and /var/log/vdsm/vdsm.log

Description of problem:
I tried to limit the writing to a virtual disk on a rhev hypervisor using virsh.
The limitation doesn't seem to be enforced.

Version-Release number of selected component (if applicable):
RHEL7 
libvirt-daemon-1.2.8-4.el7.x86_64
vdsm-4.16.6-1.el7.x86_64
qemu-kvm-rhev-1.5.3-60.el7_0.10.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Start a VM via RHEVM on a RHEV hypervisor (VDSM installed on RHEL7). Install RHEL OS on the VM
2. Attach a second disk to the VM. Crate a FS on the disk and mount it
3. Perform a base-line test. In the guest, write out a large file and see the IO rate: 
# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072
4. Limit the write troughput:
 # virsh blkdeviotune 1 vda --write_bytes_sec $(expr 1024 \* 1024 \* 10)
 # virsh blkdeviotune 1 vda
5. Now test the throughput, execute the same 'dd' test from step3:
# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072



Actual results:
Without the limitation and with it, the troughput reamins the same, even though it is supposed to be limited:

Before limitation configured (step3):

[root@localhost ~]# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072
122707+0 records in                            
122706+0 records out       
1005207552 bytes (1.0 GB) copied, 77.1332 s, 13.0 MB/s

Limitation is configured (step5):

# virsh blkdeviotune 1 vdc --write_bytes_sec $(expr 1024 \* 1024 \* 1)

virsh # blkdeviotune 1 vdc
Please enter your authentication name: elad
Please enter your password: 
total_bytes_sec: 0
read_bytes_sec : 0
write_bytes_sec: 1048576
total_iops_sec : 0
read_iops_sec  : 0
write_iops_sec : 0


[root@localhost ~]# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072
122707+0 records in
122706+0 records out
1005207552 bytes (1.0 GB) copied, 77.0214 s, 13.1 MB/s


Expected results:
Limitation should be enforced

Additional info:
/var/log/messages and /var/log/vdsm/vdsm.log
(Can't find more logs of libvirt under /var/log/libvirt/)

Comment 1 Doron Fediuck 2014-10-26 07:49:24 UTC
Elad, is this relevant for rhel 6.6 as well?

Comment 2 Ján Tomko 2014-10-27 16:44:42 UTC
I can't reliably reproduce this, but the one time it happened dd of=/dev/vdb was unable to open the device with oflag=direct. Copying to a file on a filesystem created on the hotplugged disk was reported to happen at about 500 MB/s in the guest, but it didn't go over the 1 MB/s limit in the host. It seems to me libvirt did everything right there.

Are you able to use oflag=direct on the hotplugged disk?
What is the XML of the attached disk?

Comment 3 Elad 2014-10-28 10:39:11 UTC
(In reply to Doron Fediuck from comment #1)
> Elad, is this relevant for rhel 6.6 as well?

It seems that using RHEL6.6, the blkdeviotune lomitation is enforced:
Before setting the limitation, wrote to the device:

[root@localhost elad]# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072
131072+0 records in
131072+0 records out
1073741824 bytes (1.1 GB) copied, 114.7 s, 9.4 MB/s


I set the limitation: 
virsh blkdeviotune vm_1 vdb --write_bytes_sec $(expr 1024 \* 1024 \* 1)


Now limitation for writting to the device is enforced:

 [root@localhost elad]# dd if=/dev/zero of=/elad/1 oflag=direct bs=8K count=131072
^C85275+0 records in
85275+0 records out
698572800 bytes (699 MB) copied, 693.119 s, 1.0 MB/s



I'm using a RHEL6.6 host:
libvirt-0.10.2-46.el6_6.1.x86_64
qemu-kvm-rhev-0.12.1.2-2.448.el6.x86_64
vdsm-4.16.7.1-1.el6ev.x86_64

Comment 4 Doron Fediuck 2014-10-28 16:36:35 UTC
So the problem is only in rhel 7?
Can you provide the xml dump as asked for in comment 2?

Comment 5 Elad 2014-10-29 14:20:39 UTC
Couldn't reprodeuce the issue using RHEL7 with libvirt-daemon-1.2.8-5.el7_0.x86_64


The write limitation is enforced.

Comment 6 Aharon Canan 2014-11-02 13:35:08 UTC
Doron, 

As this issue doesn't reproduce, we thought maybe it is related to libvirt version ...

anyway we can close as worksforme unless you find something while investigating  

Up to you.

Comment 7 Doron Fediuck 2014-11-02 21:16:28 UTC
(In reply to Aharon Canan from comment #6)
> Doron, 
> 
> As this issue doesn't reproduce, we thought maybe it is related to libvirt
> version ...
> 
> anyway we can close as worksforme unless you find something while
> investigating  
> 
> Up to you.

Go ahead and close.
We'll re-open if needed.