Bug 762633 (GLUSTER-901)

Summary:	windows iozone causes posix writev to fail
Product:	[Community] GlusterFS	Reporter:	Krishna Srinivas <krishna>
Component:	posix	Assignee:	Vijay Bellur <vbellur>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	medium	Docs Contact:
Priority:	urgent
Version:	mainline	CC:	chida, gluster-bugs, shehjart, vijay
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Windows
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	RTP	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:

Description Krishna Srinivas 2010-05-06 23:19:44 UTC

i was running iozone from windows:
iozone -i 0 -i 1 -s 24g -r 128k -t 2 -F 1 2

iozone complained "fd 4 error"

client log had:
[2010-05-06 22:02:40] W [fuse-bridge.c:1916:fuse_writev_cbk] glusterfs-fuse: 15079609: WRITE => -1 (Invalid argument)
[2010-05-06 22:05:47] W [fuse-bridge.c:1916:fuse_writev_cbk] glusterfs-fuse: 15731866: WRITE => -1 (Invalid argument)
(END)


server log had:

[2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:841
[2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:832
[2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:833
[2010-05-06 20:33:00] E [posix.c:2609:posix_writev] posix3: lseek(8589934591) on fd=0x7f89e404bb50 failed: Invalid argument
[2010-05-06 20:34:19] E [posix.c:2609:posix_writev] posix3: lseek(12884901887) on fd=0x7f89e404bb50 failed: Invalid argument
[2010-05-06 20:35:58] E [posix.c:2609:posix_writev] posix3: lseek(17179869183) on fd=0x7f89e404bb50 failed: Invalid argument
[2010-05-06 20:37:49] E [posix.c:2609:posix_writev] posix3: lseek(21474836479) on fd=0x7f89e404bb50 failed: Invalid argument
[2010-05-06 20:39:43] E [posix.c:2609:posix_writev] posix3: lseek(25769803775) on fd=0x7f89e404bb50 failed: Invalid argument
[2010-05-06 21:21:07] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:21:14] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 21:22:25] E [posix.c:2609:posix_writev] posix2: lseek(8589934591) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:22:48] E [posix.c:2609:posix_writev] posix2: lseek(8589934591) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 21:23:44] E [posix.c:2609:posix_writev] posix2: lseek(12884901887) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:24:21] E [posix.c:2609:posix_writev] posix2: lseek(12884901887) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 21:25:04] E [posix.c:2609:posix_writev] posix2: lseek(17179869183) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:25:55] E [posix.c:2609:posix_writev] posix2: lseek(17179869183) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 21:26:24] E [posix.c:2609:posix_writev] posix2: lseek(21474836479) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:27:34] E [posix.c:2609:posix_writev] posix2: lseek(21474836479) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 21:27:45] E [posix.c:2609:posix_writev] posix2: lseek(25769803775) on fd=0x7f89e4176680 failed: Invalid argument
[2010-05-06 21:32:26] E [posix.c:2609:posix_writev] posix2: lseek(25769803775) on fd=0x7f89e40a8e30 failed: Invalid argument
[2010-05-06 22:14:03] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e40d2440 failed: Invalid argument
[2010-05-06 22:17:10] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e416b5e0 failed: Invalid argument


Notice that lseek values are 4G-1 8G-1 12G-1 16G-1 20G-1 24G-1

Comment 1 Shehjar Tikoo 2010-05-07 01:55:36 UTC

I see you're using mainline. Can you try a git pull and use the latest head from the mainline. A recent patch has removed the use of lseek from posix_writev completely.

That patch is:
http://git.gluster.com/?p=glusterfs.git;a=commitdiff;h=b0ed997cda2195e9178cbaa96d26976aa6dd2acf

Comment 2 Anand Avati 2010-05-08 10:37:52 UTC

PATCH: http://patches.gluster.com/patch/3241 in release-3.0 (storage/posix: Use off_t for return value from lseek())

Comment 3 Krishna Srinivas 2010-05-10 02:17:03 UTC

(In reply to comment #1)
> I see you're using mainline. Can you try a git pull and use the latest head
> from the mainline. A recent patch has removed the use of lseek from
> posix_writev completely.
> 
> That patch is:
> http://git.gluster.com/?p=glusterfs.git;a=commitdiff;h=b0ed997cda2195e9178cbaa96d26976aa6dd2acf

Avati's patch above will take care of existing customers on 3.0

But you have mentioned that recent iothreads patch can cause same file to be written by different threads, how do you take care of writes not to overtake each other? or fsync not to overtake write?