Bug 1449638
Summary: | Poor write speed performance of fio test on distributed-disperse volume | ||
---|---|---|---|
Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Prashant Dhange <pdhange> |
Component: | disperse | Assignee: | Ashish Pandey <aspandey> |
Status: | CLOSED ERRATA | QA Contact: | Nag Pavan Chilakam <nchilaka> |
Severity: | high | Docs Contact: | |
Priority: | high | ||
Version: | rhgs-3.2 | CC: | amukherj, aspandey, pdhange, pkarampu, pmulay, psony, rhinduja, rhs-bugs, sheggodu, storage-qa-internal, ubansal |
Target Milestone: | --- | Keywords: | ZStream |
Target Release: | RHGS 3.4.0 | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | rebase | ||
Fixed In Version: | glusterfs-3.12.2-1 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2018-09-04 06:32:21 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1417153, 1472361, 1503135 |
Comment 41
Ashish Pandey
2017-07-26 10:14:33 UTC
on-qa validation: based on all above comments this is a problem with fio writing in 1Byte chunks As this should be fixed in latest FIO, I am comparing the writes b/w fio.2.2.8(where the issue was seen) as with latest fio also tested with 10MB file below is the result [root@dhcp35-72 dispersevol1]# rpm -qa|grep fio fio-2.2.8-2.el7.x86_64 [root@dhcp35-72 dispersevol1]# time fio --randrepeat=1 --ioengine=libaio --direct=0 --gtod_reduce=1 --name=test --filename=fiotest.fio --iodepth=16 --size=10M --readwrite=write test: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=16 fio-2.2.8 Starting 1 process test: Laying out IO file(s) (1 file(s) / 10MB) Jobs: 1 (f=1): [W(1)] [100.0% done] [0KB/1144KB/0KB /s] [0/286/0 iops] [eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=11112: Tue May 29 21:35:52 2018 write: io=10240KB, bw=1160.4KB/s, iops=290, runt= 8825msec cpu : usr=0.27%, sys=1.10%, ctx=2566, majf=0, minf=23 IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=99.4%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=2560/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): WRITE: io=10240KB, aggrb=1160KB/s, minb=1160KB/s, maxb=1160KB/s, mint=8825msec, maxt=8825msec real 0m19.064s user 0m0.136s sys 0m0.718s ########## with latest fio [root@dhcp35-72 fio]# time fio --randrepeat=1 --ioengine=libaio --direct=0 --gtod_reduce=1 --name=test --filename=fiotest.fio --iodepth=16 --size=10M --readwrite=write test: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=16 fio-3.1 Starting 1 process test: Laying out IO file (1 file / 10MiB) fio: native_fallocate call failed: Operation not supported Jobs: 1 (f=1): [W(1)][100.0%][r=0KiB/s,w=976KiB/s][r=0,w=244 IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=11132: Tue May 29 21:41:55 2018 write: IOPS=270, BW=1080KiB/s (1106kB/s)(10.0MiB/9479msec) bw ( KiB/s): min= 888, max= 1208, per=99.39%, avg=1073.44, stdev=92.80, samples=18 iops : min= 222, max= 302, avg=268.33, stdev=23.19, samples=18 cpu : usr=0.19%, sys=1.52%, ctx=2566, majf=0, minf=20 IO depths : 1=0.1%, 2=0.1%, 4=0.2%, 8=0.3%, 16=99.4%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.1%, 32=0.0%, 64=0.0%, >=64=0.0% issued rwt: total=0,2560,0, short=0,0,0, dropped=0,0,0 latency : target=0, window=0, percentile=100.00%, depth=16 Run status group 0 (all jobs): WRITE: bw=1080KiB/s (1106kB/s), 1080KiB/s-1080KiB/s (1106kB/s-1106kB/s), io=10.0MiB (10.5MB), run=9479-9479msec real 0m10.252s user 0m0.118s sys 0m0.665s [root@dhcp35-72 fio]# date Tue May 29 21:41:57 IST 2018 [root@dhcp35-72 fio]# rpm -qa|grep fio fio-3.1-2.el7.x86_64 [root@dhcp35-72 fio]# du -sh fiotest.fio 10M fiotest.fio [root@dhcp35-72 fio]# as the performance has improved a lot , moving to verified test on gnfs as reported by customer Srijita brought this bug to my notice where this bug is actually marked as Verified Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607 |