Bug 242116

+++ This bug was initially created as a clone of Bug #198859 +++

Description of problem:
See http://bugzilla.kernel.org/show_bug.cgi?id=6831

Version-Release number of selected component (if applicable):
See http://bugzilla.kernel.org/show_bug.cgi?id=6831

How reproducible:
See http://bugzilla.kernel.org/show_bug.cgi?id=6831

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

-- Additional comment from jmoyer on 2006-08-09 13:52 EST --
I'll work with Zach on this.

-- Additional comment from jmoyer on 2007-01-12 18:04 EST --
I put together a kernel that includes Zach's patch set.  Could you please test
with this kernel to ensure that the bug is fixed for you?  I will test this on
Monday, but for now my RHEL 4 test box is toast.

-- Additional comment from wijata on 2007-01-15 05:27 EST --
Since there is a testcase, I'll pass over. Sorry have other stuff todo.

-- Additional comment from jmoyer on 2007-01-15 10:14 EST --
No problem, I'll take care of it.

Thanks!

-- Additional comment from jmoyer on 2007-01-16 13:01 EST --
The kernel which includes Zach's patch set (found at
http://people.redhat.com/jmoyer/dio/) fixes this problem in my test environment.

-- Additional comment from wijata on 2007-02-26 08:24 EST --
I have some bad news. It looks like the race was fixed, but another bug was
introduced.
First i encountered following scenario:
aio read issued to the file - OK
exactly same aio read issued to same file - returned less data than the first.
of course between those two reads there were some appends to the file.

Decided to create testcase, success. Moreover, upon the testcase failure
bug210281 was hit as well.

The test is ugly, and need iterating, but after 5 hours it fired at mine
machine. I hope the test is valid(need verification), but hitting bug210281
suggests it's valid.

compile(without DO_SYNCH), run as: while ./aio_backread; do true; done

-- Additional comment from wijata on 2007-02-26 08:28 EST --
Created an attachment (id=148791)
testcase for shortread

Tested on Tyan(something) 2*opteron275 with sata disk WD3200JS-00P attached to
sata_sil(onboard)

-- Additional comment from wijata on 2007-02-26 08:42 EST --
The kernel version: Linux server 2.6.9-42.27.EL.dio.2smp #1 SMP Mon Jan 15
15:05:46 EST 2007 i686 athlon i386 GNU/Linux

-- Additional comment from wijata on 2007-02-26 08:56 EST --
> compile(without DO_SYNCH)
in fact compiling with the DO_SYNCH (gcc aio_backread.c -DDO_SYNCH -lpthread
-laio) made the bug rendering faster(but no bug210281). No idea why...
Still trying on official RH kernel...

-- Additional comment from jmoyer on 2007-02-26 14:33 EST --
Thanks for your continued help on this problem;  it is greatly appreciated.  I
have some comments on the code for you, and some test results.

I'm not sure you should use the same buffer for concurrent reads and writes.  It
looks like you meant to initialize the write iocbs with buf.  Is that right?

The DO_SYNCH code looks incorrect to me.  You should guard access to
submittedSize with a mutex/conditional pair.

I was able to reproduce the short read twice so far.  In both cases, I got the
following output:
  short read: expected at least 1024 bytes, receiced 0

I think the expected size of 1024 can only happen after the very first set of
writes completes, right?  Is this consistent with the failures you've witnessed?

-- Additional comment from jmoyer on 2007-02-26 14:58 EST --
Sure enough, I modified the source code to only issue a dozen reads and then to
cancel the write thread and exit the program.  I can now reproduce the short
read in a matter of seconds.

-- Additional comment from wijata on 2007-02-27 03:09 EST --
> looks like you meant to initialize the write iocbs with buf.  Is that right?
Right, my mistake.

> submittedSize with a mutex/conditional pair
possibly(or atomic asm op), but: int writes are atomic anyway on x86, and the
bug appears without synch as well.

> I can now reproduce the short read in a matter of seconds.
glad You could improve the testcase.

BTW: official RH kernel 2.6.9-42.0.3.ELsmp with the synch was working well for
20hours, so I guess the patch introduced something malicious.

-- Additional comment from pm-rhel on 2007-04-27 15:24 EST --
This request was evaluated by Red Hat Kernel Team for inclusion in a Red
Hat Enterprise Linux maintenance release, and has moved to bugzilla 
status POST.