Bug 889941

Summary: [virtio-win][scsi] virtio_scsi driver performs bad during large buffers transfer
Product: Red Hat Enterprise Linux 6 Reporter: Xiaomei Gao <xigao>
Component: virtio-winAssignee: Vadim Rozenfeld <vrozenfe>
virtio-win sub component: virtio-win-prewhql QA Contact: Virtualization Bugs <virt-bugs>
Status: CLOSED WONTFIX Docs Contact:
Severity: unspecified    
Priority: unspecified CC: ailan, jhradile, juzhang, knoel, lijin, michen, rbalakri, rpacheco, virt-bugs, vrozenfe, wquan, xigao, yama
Version: 6.4   
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-06 11:15:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
iometer profile none

Comment 2 Xiaomei Gao 2012-12-24 06:14:33 UTC
Created attachment 668319 [details]
iometer profile

Comment 9 Xiaomei Gao 2013-08-29 03:45:47 UTC
hi,vadim

  Which build includes the fixed patch? We will verify the issue.

Thanks
xiaomei

Comment 10 Vadim Rozenfeld 2013-08-29 09:57:20 UTC
Hi Xiaomei,
please try the latest build (68): http://download.devel.redhat.com/brewroot/work/tasks/7431/6217431/virtio-win-prewhql-0.1.zip

Thank you,
Vadim.

Comment 11 Xiaomei Gao 2013-09-04 02:28:16 UTC
(In reply to Vadim Rozenfeld from comment #10)
> Hi Xiaomei,
> please try the latest build (68):
> http://download.devel.redhat.com/brewroot/work/tasks/7431/6217431/virtio-win-
> prewhql-0.1.zip

  Do IO test on virtio_scsi driver disk and guest BSOD. BSOD code is the same as bug 1001616 and bug 996951 .  

Best Regards
Xiaomei

Comment 12 Vadim Rozenfeld 2013-09-08 05:30:40 UTC
please re-check with build 69 available at https://brewweb.devel.redhat.com/buildinfo?buildID=293655

Thanks
Vadim.

Comment 13 Xiaomei Gao 2013-10-10 09:06:46 UTC
(In reply to Vadim Rozenfeld from comment #12)
> please re-check with build 69 available at
> https://brewweb.devel.redhat.com/buildinfo?buildID=293655

Verify this bug on virtio-win-prewhql-0.1-72, but it is blocked by Bug 1017529.

Thanks
Xiaomei

Comment 14 Vadim Rozenfeld 2013-10-10 11:15:25 UTC
Hi Xiaomei,
Bug 1017529 seems to be the serous bug,
but I believe shouldn't hit this problem 
on a system running with one virtio-scsi
device only. I will much appreciate if 
you can give it a try.

Thanks,
Vadim.

Comment 15 Xiaomei Gao 2013-10-11 01:53:20 UTC
(In reply to Vadim Rozenfeld from comment #14)
> Hi Xiaomei,
> Bug 1017529 seems to be the serous bug,
> but I believe shouldn't hit this problem 
> on a system running with one virtio-scsi
> device only. I will much appreciate if 
> you can give it a try.

Hi, Vadim

I tried it for many times with both iometer and fio benchmark for one virtio-scsi
device, unfortunately, guest always hit BSOD since the test beginning and we can't get results.

Thanks
Xiaomei

Comment 16 Vadim Rozenfeld 2013-10-11 05:45:05 UTC
Hi Xiaomei,
Could you please upload the relevant crash dump file?
It will be very helpful.

Thanks,
Vadim.

Comment 29 Vadim Rozenfeld 2016-10-27 05:27:32 UTC
Can we retest it again with the latest drivers?

Thanks,
Vadim.

Comment 30 Yanhui Ma 2016-10-28 02:35:58 UTC
(In reply to Vadim Rozenfeld from comment #29)
> Can we retest it again with the latest drivers?
> 
Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1

host:
qemu-kvm-0.12.1.2-2.491.el6_8.3
kernel-2.6.32-642.11.1.el6

> Thanks,
> Vadim.

Comment 31 Yanhui Ma 2016-11-01 07:02:03 UTC
(In reply to Yanhui Ma from comment #30)
> (In reply to Vadim Rozenfeld from comment #29)
> > Can we retest it again with the latest drivers?
> > 
> Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> 
> host:
> qemu-kvm-0.12.1.2-2.491.el6_8.3
> kernel-2.6.32-642.11.1.el6
> 

Here are results of win2012r2.x86_64 with above component version:

http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.virtio_net.Win2012r2.x86_64.fio_win.html

No regression is found.

> > Thanks,
> > Vadim.

Comment 32 Amnon Ilan 2016-11-01 19:27:15 UTC
(In reply to Yanhui Ma from comment #31)
> (In reply to Yanhui Ma from comment #30)
> > (In reply to Vadim Rozenfeld from comment #29)
> > > Can we retest it again with the latest drivers?
> > > 
> > Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> > 
> > host:
> > qemu-kvm-0.12.1.2-2.491.el6_8.3
> > kernel-2.6.32-642.11.1.el6
> > 
> 
> Here are results of win2012r2.x86_64 with above component version:
> 
> http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.
> virtio_net.Win2012r2.x86_64.fio_win.html
> 
> No regression is found.

Can you elaborate on that? regression compared to what?

Comment 33 Yanhui Ma 2016-11-02 02:48:35 UTC
(In reply to Amnon Ilan from comment #32)
> (In reply to Yanhui Ma from comment #31)
> > (In reply to Yanhui Ma from comment #30)
> > > (In reply to Vadim Rozenfeld from comment #29)
> > > > Can we retest it again with the latest drivers?
> > > > 
> > > Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> > > 
> > > host:
> > > qemu-kvm-0.12.1.2-2.491.el6_8.3
> > > kernel-2.6.32-642.11.1.el6
> > > 
> > 
> > Here are results of win2012r2.x86_64 with above component version:
> > 
> > http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.
> > virtio_net.Win2012r2.x86_64.fio_win.html
> > 
> > No regression is found.
> 
> Can you elaborate on that? regression compared to what?

It is comparison between ide and virtio_scsi.
As requested by Vadim in comment 29, I retest performance between ide and virtio_scsi with virtio-win-1.7.5-0.el6_8.1. From results, we can see throughput of virtio_scsi is almost the same as ide. There is no obvious and consistent degradation with iodepth=64 as described by comment 24.

Comment 34 Vadim Rozenfeld 2016-11-02 05:11:20 UTC
(In reply to Yanhui Ma from comment #33)
> (In reply to Amnon Ilan from comment #32)
> > (In reply to Yanhui Ma from comment #31)
> > > (In reply to Yanhui Ma from comment #30)
> > > > (In reply to Vadim Rozenfeld from comment #29)
> > > > > Can we retest it again with the latest drivers?
> > > > > 
> > > > Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> > > > 
> > > > host:
> > > > qemu-kvm-0.12.1.2-2.491.el6_8.3
> > > > kernel-2.6.32-642.11.1.el6
> > > > 
> > > 
> > > Here are results of win2012r2.x86_64 with above component version:
> > > 
> > > http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.
> > > virtio_net.Win2012r2.x86_64.fio_win.html
> > > 
> > > No regression is found.
> > 
> > Can you elaborate on that? regression compared to what?
> 
> It is comparison between ide and virtio_scsi.
> As requested by Vadim in comment 29, I retest performance between ide and
> virtio_scsi with virtio-win-1.7.5-0.el6_8.1. From results, we can see
> throughput of virtio_scsi is almost the same as ide. There is no obvious and

And it looks very strange, we would expect way better performance from virtio_scsi on small blocks and the same performance as ide on large bolcks.


> consistent degradation with iodepth=64 as described by comment 24.

Comment 35 Yanhui Ma 2016-11-02 05:34:52 UTC
(In reply to Vadim Rozenfeld from comment #34)
> (In reply to Yanhui Ma from comment #33)
> > (In reply to Amnon Ilan from comment #32)
> > > (In reply to Yanhui Ma from comment #31)
> > > > (In reply to Yanhui Ma from comment #30)
> > > > > (In reply to Vadim Rozenfeld from comment #29)
> > > > > > Can we retest it again with the latest drivers?
> > > > > > 
> > > > > Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> > > > > 
> > > > > host:
> > > > > qemu-kvm-0.12.1.2-2.491.el6_8.3
> > > > > kernel-2.6.32-642.11.1.el6
> > > > > 
> > > > 
> > > > Here are results of win2012r2.x86_64 with above component version:
> > > > 
> > > > http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.
> > > > virtio_net.Win2012r2.x86_64.fio_win.html
> > > > 
> > > > No regression is found.
> > > 
> > > Can you elaborate on that? regression compared to what?
> > 
> > It is comparison between ide and virtio_scsi.
> > As requested by Vadim in comment 29, I retest performance between ide and
> > virtio_scsi with virtio-win-1.7.5-0.el6_8.1. From results, we can see
> > throughput of virtio_scsi is almost the same as ide. There is no obvious and
> 
> And it looks very strange, we would expect way better performance from
> virtio_scsi on small blocks and the same performance as ide on large bolcks.
> 
Hi Vadim,

Compared with steps in comment 0, my steps are a little different, I don't format disk into NTFS in windows guest, and I run fio in raw disk, not NTFS.
Does it matter?



> 
> > consistent degradation with iodepth=64 as described by comment 24.

Comment 36 Vadim Rozenfeld 2016-11-02 07:33:41 UTC
(In reply to Yanhui Ma from comment #35)
> (In reply to Vadim Rozenfeld from comment #34)
> > (In reply to Yanhui Ma from comment #33)
> > > (In reply to Amnon Ilan from comment #32)
> > > > (In reply to Yanhui Ma from comment #31)
> > > > > (In reply to Yanhui Ma from comment #30)
> > > > > > (In reply to Vadim Rozenfeld from comment #29)
> > > > > > > Can we retest it again with the latest drivers?
> > > > > > > 
> > > > > > Ok, I will retest it in RHEL6.8.z host with virtio-win-1.7.5-0.el6_8.1
> > > > > > 
> > > > > > host:
> > > > > > qemu-kvm-0.12.1.2-2.491.el6_8.3
> > > > > > kernel-2.6.32-642.11.1.el6
> > > > > > 
> > > > > 
> > > > > Here are results of win2012r2.x86_64 with above component version:
> > > > > 
> > > > > http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/raw.*.smp2.
> > > > > virtio_net.Win2012r2.x86_64.fio_win.html
> > > > > 
> > > > > No regression is found.
> > > > 
> > > > Can you elaborate on that? regression compared to what?
> > > 
> > > It is comparison between ide and virtio_scsi.
> > > As requested by Vadim in comment 29, I retest performance between ide and
> > > virtio_scsi with virtio-win-1.7.5-0.el6_8.1. From results, we can see
> > > throughput of virtio_scsi is almost the same as ide. There is no obvious and
> > 
> > And it looks very strange, we would expect way better performance from
> > virtio_scsi on small blocks and the same performance as ide on large bolcks.
> > 
> Hi Vadim,
> 
> Compared with steps in comment 0, my steps are a little different, I don't
> format disk into NTFS in windows guest, and I run fio in raw disk, not NTFS.
> Does it matter?

No, it shouldn't. Raw partition should be even more preferable, because it doesn't introduce any file system specific "features". 
I will try to run the same tests on my setup to see what can lead to such strange results.

Best regards,
Vadim.

> 
> 
> 
> > 
> > > consistent degradation with iodepth=64 as described by comment 24.

Comment 37 Vadim Rozenfeld 2016-11-03 23:31:35 UTC
Will it be possible to retest the same configurations, with "hv_time" flag  enabled on rhel7?
Thanks,
Vadim.

Comment 38 Yanhui Ma 2016-11-07 02:11:00 UTC
(In reply to Vadim Rozenfeld from comment #37)
> Will it be possible to retest the same configurations, with "hv_time" flag 
> enabled on rhel7?
ok, I will test win2012r2_x86_64 with "hv_time" flag on rhel7 host when the machine is available.
> Thanks,
> Vadim.

Comment 39 Yanhui Ma 2016-11-22 09:52:37 UTC
(In reply to Vadim Rozenfeld from comment #37)
> Will it be possible to retest the same configurations, with "hv_time" flag 
> enabled on rhel7?


I retested win2012r2_x86_64 with "hv_time" flag enabled on rhel7.3 host with virtio-win-1.9.0-3.el7.

host:
qemu-kvm-rhev-2.6.0-22.el7.x86_64
kernel-3.10.0-514.el7.x86_64

Here are results comparison between ide and virtio_scsi.
http://kvm-perf.englab.nay.redhat.com/results/request/bug889941/hv_time_on/raw.*.smp2.virtio_net.Win2012r2.x86_64.fio_win.html

Throughput is still almost the same between ide and virtio_scsi.
> Thanks,
> Vadim.

Comment 41 Jan Kurik 2017-12-06 11:15:40 UTC
Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available.

The official life cycle policy can be reviewed here:

http://redhat.com/rhel/lifecycle

This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL:

https://access.redhat.com/