Hide Forgot
i was running iozone from windows: iozone -i 0 -i 1 -s 24g -r 128k -t 2 -F 1 2 iozone complained "fd 4 error" client log had: [2010-05-06 22:02:40] W [fuse-bridge.c:1916:fuse_writev_cbk] glusterfs-fuse: 15079609: WRITE => -1 (Invalid argument) [2010-05-06 22:05:47] W [fuse-bridge.c:1916:fuse_writev_cbk] glusterfs-fuse: 15731866: WRITE => -1 (Invalid argument) (END) server log had: [2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:841 [2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:832 [2010-05-06 19:30:00] N [server-protocol.c:5852:mop_setvolume] server-ib-verbs: accepted client from 192.168.102.53:833 [2010-05-06 20:33:00] E [posix.c:2609:posix_writev] posix3: lseek(8589934591) on fd=0x7f89e404bb50 failed: Invalid argument [2010-05-06 20:34:19] E [posix.c:2609:posix_writev] posix3: lseek(12884901887) on fd=0x7f89e404bb50 failed: Invalid argument [2010-05-06 20:35:58] E [posix.c:2609:posix_writev] posix3: lseek(17179869183) on fd=0x7f89e404bb50 failed: Invalid argument [2010-05-06 20:37:49] E [posix.c:2609:posix_writev] posix3: lseek(21474836479) on fd=0x7f89e404bb50 failed: Invalid argument [2010-05-06 20:39:43] E [posix.c:2609:posix_writev] posix3: lseek(25769803775) on fd=0x7f89e404bb50 failed: Invalid argument [2010-05-06 21:21:07] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:21:14] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 21:22:25] E [posix.c:2609:posix_writev] posix2: lseek(8589934591) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:22:48] E [posix.c:2609:posix_writev] posix2: lseek(8589934591) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 21:23:44] E [posix.c:2609:posix_writev] posix2: lseek(12884901887) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:24:21] E [posix.c:2609:posix_writev] posix2: lseek(12884901887) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 21:25:04] E [posix.c:2609:posix_writev] posix2: lseek(17179869183) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:25:55] E [posix.c:2609:posix_writev] posix2: lseek(17179869183) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 21:26:24] E [posix.c:2609:posix_writev] posix2: lseek(21474836479) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:27:34] E [posix.c:2609:posix_writev] posix2: lseek(21474836479) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 21:27:45] E [posix.c:2609:posix_writev] posix2: lseek(25769803775) on fd=0x7f89e4176680 failed: Invalid argument [2010-05-06 21:32:26] E [posix.c:2609:posix_writev] posix2: lseek(25769803775) on fd=0x7f89e40a8e30 failed: Invalid argument [2010-05-06 22:14:03] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e40d2440 failed: Invalid argument [2010-05-06 22:17:10] E [posix.c:2609:posix_writev] posix2: lseek(4294967295) on fd=0x7f89e416b5e0 failed: Invalid argument Notice that lseek values are 4G-1 8G-1 12G-1 16G-1 20G-1 24G-1
I see you're using mainline. Can you try a git pull and use the latest head from the mainline. A recent patch has removed the use of lseek from posix_writev completely. That patch is: http://git.gluster.com/?p=glusterfs.git;a=commitdiff;h=b0ed997cda2195e9178cbaa96d26976aa6dd2acf
PATCH: http://patches.gluster.com/patch/3241 in release-3.0 (storage/posix: Use off_t for return value from lseek())
(In reply to comment #1) > I see you're using mainline. Can you try a git pull and use the latest head > from the mainline. A recent patch has removed the use of lseek from > posix_writev completely. > > That patch is: > http://git.gluster.com/?p=glusterfs.git;a=commitdiff;h=b0ed997cda2195e9178cbaa96d26976aa6dd2acf Avati's patch above will take care of existing customers on 3.0 But you have mentioned that recent iothreads patch can cause same file to be written by different threads, how do you take care of writes not to overtake each other? or fsync not to overtake write?