Bug 397141 - linux readahead bug is back (causing mediacheck to fail)
linux readahead bug is back (causing mediacheck to fail)
Status: CLOSED DUPLICATE of bug 186512
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
10
i386 Linux
low Severity medium
: ---
: ---
Assigned To: Alan Cox
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2007-11-23 13:04 EST by Andre Robatino
Modified: 2008-11-28 06:09 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-11-28 06:09:52 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Andre Robatino 2007-11-23 13:04:12 EST
Description of problem:
  My father bought a F8 DVD, and did the mediacheck, which failed.  Then he used
the rawread script at

http://www.troubleshooters.com/linux/coasterless.htm

to read off the ISO image to a file (this uses the isoinfo command to determine
how big the ISO is, then runs a dd command to attempt reading off exactly that
much).  What it read off was correct, except for being truncated by about 40 KB.
 The end of the rawread output was this (it repeated many more times):

3424731136 bytes (3.4 GB) copied, 345.048 s, 9.9 MB/s
dd: reading `/dev/dvd': Input/output error
1672232+0 records in
1672232+0 records out
3424731136 bytes (3.4 GB) copied, 350.265 s, 9.8 MB/s
1672232+0 records in
1672232+0 records out
3424731136 bytes (3.4 GB) copied, 350.266 s, 9.8 MB/s
60294a5a6efea17c1b13b50a8d83da6b2cc44c2f  -

  By using wget -c, he was able to resume downloading the last 40 KB of the file
and get a correct sha1sum for the result.  He is currently running a fully
updated 32-bit F7, so the bug is affecting that kernel.  Then I started testing
my stack of old, un-zero-padded, Fedora CDs, using the rawread script, but on my
fully updated 32-bit F8.  Although every disc I tested checked good about 6
months ago, using checkisomd5 from anaconda-runtime, just after F7 came out, now
the first 3 of 3 discs all test bad (I stopped testing there).  Running rawread
on the first FC4 CD, which is unpadded, I was able to read off only 665305088
out of 665434112 bytes, but the part I could read off was correct (again, using
wget -c to finish the file).  Unlike my father, my rawread never stopped, but
just kept repeating the same attempt to read the same part of the file:

665305088 bytes (665 MB) copied, 364.556 s, 1.8 MB/s
dd: reading `/dev/cdrom': Input/output error
324856+0 records in
324856+0 records out
665305088 bytes (665 MB) copied, 364.752 s, 1.8 MB/s
324856+0 records in
324856+0 records out
665305088 bytes (665 MB) copied, 364.753 s, 1.8 MB/s

etc., etc., so I finally Ctrl-C'd it.  In dmesg, there were lots of the
following messages:

end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
printk: 26 messages suppressed.
Buffer I/O error on device sr0, logical block 324920
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680
end_request: I/O error, dev sr0, sector 1299680

Version-Release number of selected component (if applicable):
kernel-2.6.23.1-49.fc8

How reproducible:
always
Comment 1 Andre Robatino 2007-11-23 13:15:29 EST
  Forgot to include the important info that both I and my father are using
identical Sony DW-G120A DVD RW drives (firmware version MYR5, no firmware update
available) purchased less than a year ago.

Comment 2 Andre Robatino 2007-11-23 15:13:04 EST
  Additionally, we were using these same drives 6 months ago, when F7 came out.
 So in my case, I was checking those old unpadded discs with the same drive, so
the fact that they all passed back then, with this drive, but not now, indicates
that there really is an OS regression, not just a failure to take this
particular drive into account.
Comment 3 Andre Robatino 2007-11-23 21:35:15 EST
  I took my unpadded but otherwise good FC4 disc 1, and did a mediacheck on it
after booting from each of several recent install DVDs.

Fedora 7 DVD:                                passes
Fedura Unity 20070912 7 DVD:  passes
Fedora 7.92 (F8t3) DVD:              fails
Fedora 8 DVD:                                fails

So it appears the problem came back sometime in September.  I don't happen to
have a F8 test 1 or 2 DVD, but could download and test it if really necessary.
Comment 4 Andre Robatino 2007-11-24 03:16:11 EST
  One more data point:

Fedora Unity 20071030 7 DVD:     fails

FU7 20070912 shipped with kernel-2.6.22.5-76.fc7, and FU7 20071030 shipped with
kernel-2.6.23.1-10.fc7.  Looking through fedora-package-announce for September
and October, the only F7 kernel updates between these two were

Sep. 25: kernel-2.6.22.7-85.fc7
Sep. 28: kernel-2.6.22.9-91.fc7

so the first F7 kernel to bring back the readahead bug was one of
kernel-2.6.22.7-85.fc7, kernel-2.6.22.9-91.fc7, or kernel-2.6.23.1-10.fc7.
Comment 5 Andre Robatino 2007-11-24 15:42:49 EST
  I downloaded and installed the kernels for F7.90 and F7.91 (F8 test 1 and 2,
resp.).  The former has almost been wiped from the face of the earth, but I was
able to find a copy at ftp.cse.yzu.edu.tw.  That shouldn't happen - all the old
versions should be available.  Anyway, after booting from the F7.90
kernel-2.6.23-0.164.rc5.fc8, and then checking my unpadded disc with
checkisomd5, the mediacheck fails.  I couldn't check F7.91 since the kernel
panicked while booting.  But it seems that the readahead bug had already
returned with F8t1, which was released on August 7.
Comment 6 Andre Robatino 2007-11-24 15:44:17 EST
  Sorry, I should have said "latter", not "former".  The
download.fedora.redhat.com server, and almost all of the mirrors, only have 7.90
and 7.92.
Comment 7 Andre Robatino 2007-11-24 18:01:09 EST
  I had my father, who is running a fully updated F7, install the
anaconda-runtime package, and then use checkisomd5 to check his unpadded
purchased F8 DVD, which he could not read near the end due to the readahead bug
affecting the kernel he's currently running.  Oddly, the mediacheck passed.  I
have no explanation for this.
Comment 8 Andre Robatino 2007-12-03 19:51:48 EST
  Still broken in kernel-2.6.23.8-63.fc8.
Comment 9 Andre Robatino 2007-12-20 13:55:13 EST
  Still broken in kernel-2.6.23.9-85.fc8.  This time, though, when trying to use
rawread to read my unpadded FC4 disc 1, it was able to read 665419776 out of
665434112 bytes, whereas the two previous F8 kernels both failed at 665305088
bytes.  Has the relevant kernel code changed?
Comment 10 Andre Robatino 2008-01-24 18:02:03 EST
  Still broken in kernel-2.6.23.14-107.fc8.  The number of bytes it can read
from the unpadded FC4 disc 1 has gone back to 665305088.
Comment 11 Andre Robatino 2008-02-01 20:14:22 EST
Could the priority on this be raised?  There appears to be no interest currently
in fixing this.  Considering that it WAS fixed when F7 came out, it shouldn't be
too hard for someone familiar with the relevant code to discover what went
wrong, assuming it was around 2.6.23.  And mediacheck failures won't be taken
seriously until well after this is fixed, since the bug has existed for almost
the entire time mediacheck has, and people have been trained to ignore failures
and install anyway.  
Comment 12 Alan Cox 2008-02-04 11:37:20 EST
I have no other reports, no way to duplicate it and no plausible code changes to
explain it so at the moment the priority for this bug is very low.
Comment 13 Andre Robatino 2008-02-04 21:08:29 EST
  Following the advice in the following post

http://www.mail-archive.com/cdwrite@other.debian.org/msg11444.html

I ran the command

[root@localhost ~]# blockdev --setra 0 /dev/dvd

to change the default readahead from 256 to 0, and then ran the rawread command
again on the same disc.  This time, it read 665432064 bytes instead of 665305088
(the read is also much slower), but still failed near the end.  Unfortunately,
most of the discussion on that thread is over my head, so I don't know if this
is helpful or not.  I know the bug is reproducible with a Sony DRU-120C DVD
drive and an unpadded Fedora CD burned in TAO mode, so I would be willing to buy
one of these drives and have it shipped to someone qualified to diagnose the
bug, if a listing turns up on Ebay (I bought mine in 12/2006 and I don't think
they are sold anymore).
Comment 14 Andre Robatino 2008-02-06 16:38:30 EST
I downloaded Fedora-9-Alpha-i386-DVD.iso and burned this ISO, unpadded, to both
a DVD+R and a DVD+RW using the command

growisofs -dvd-compat -speed=1 -Z /dev/dvd=Fedora-9-Alpha-i386-DVD.iso

and then checked each using the command

rawread /dev/dvd | sha1sum

while running F8.  To my surprise, both discs checked properly.  Then I
downloaded Fedora-9-Alpha-i386-rescuecd.iso and burned this ISO, unpadded, to a
CD-RW using the command

wodim -v dev=/dev/cdrom blank=fast driveropts=burnfree
Fedora-9-Alpha-i386-rescuecd.iso

and to a CD-R using the command

wodim -v dev=/dev/cdrom driveropts=burnfree Fedora-9-Alpha-i386-rescuecd.iso

and checked each using the command

rawread /dev/cdrom | sha1sum

In both cases, the read failed shortly before the end.  With the CD-RW, it read
117452800 out of 117508096 bytes.  With the CD-R, it read 117424128 bytes.  In
both cases, writing the partial reads to a file, using wget -c to complete them,
and sha1sum to check them, shows that the partial reads are correct up to the
cutoff point.

So to reproduce this bug, one should burn an unpadded ISO to a CD using one of
the above commands, then check it using rawread with a >= 2.6.23 kernel and a CD
or DVD drive known to have been vulnerable to the readahead bug before F7.  The
bug doesn't seem to affect DVDs.
Comment 15 Andre Robatino 2008-02-06 16:50:35 EST
Correction - I know it affects DVDs, or at least did until recently, since both
unpadded F8 DVDs my father bought online failed mediacheck, but it seems to be
much harder to reproduce, plus a rescue CD image is much smaller so makes it
much easier to burn the disc and reproduce the bug.
Comment 16 Andre Robatino 2008-02-06 19:52:26 EST
  I tried reproducing my father's experience by burning the unpadded F8 DVD ISO
to a DVD+RW, booting from it, and running mediacheck.  It passes for me - which
means that there must be something different about the mass-pressed discs which
is what he probably got.  I tried to do the same with Fedora 9 Alpha, but
unfortunately due to bug #431138, it's impossible to see whether mediacheck
passed or failed.  So currently I can only reproduce the problem with CDs, and
most people never install using these anymore, which may account for the lack of
reports.
Comment 17 Andre Robatino 2008-02-07 05:22:07 EST
Still broken with kernel-2.6.23.14-115.fc8.  With the unpadded FC4 disc, can
read 665419776 out of 665434112 bytes again.  With the Fedora 9 Alpha rescuecd
CD-RW, can read 117489664 out of 117508096 bytes, and with the CD-R, 117424128
bytes, same as the previous kernel.
Comment 18 Alan Cox 2008-02-07 15:44:49 EST
The fact mass produced disks work is consistent with the bug report. Mass
produced disks have a defined end, CD-R's don't.
Comment 19 Andre Robatino 2008-02-07 16:17:00 EST
I think you misunderstood - the online-purchased F8 DVD my father bought is the
one that failed mediacheck, and also had the failed read near the end.  The
unpadded DVD+R and DVD+RW discs I burned for myself work properly.  I don't
doubt that if I obtained an identical F8 DVD from the same vendor, it would fail
for me as well, since I'm using exactly the same DVD drive as my father.
Comment 20 Andre Robatino 2008-02-09 14:49:37 EST
  I just had my father read the F8 DVD he purchased while running the latest F8
kernel, and the problem still exists for him.  The rawread command fails after
reading 3424710656 out of 3424749568 bytes which is 38912 bytes short of the
end.  If anyone else can reproduce the problem with CDs, then either they could
purchase the same DVD from the same vendor, or since my father has 2 copies of
the same DVD (the vendor sent a second one which failed in the same way), I
could have him send one of the copies to the person's address.
Comment 21 Andre Robatino 2008-02-11 20:12:23 EST
Still broken in kernel-2.6.23.15-137.fc8.

FC4 i386 Disc 1 CD-R: 665419776/665434112 bytes
Fedora 9 Alpha i386 rescuecd CD-RW: 117424128/117508096 bytes (changed)
Fedora 9 Alpha i386 rescuecd CD-R: 117452800/117508096 bytes (changed)
Comment 22 Andre Robatino 2008-03-07 00:15:17 EST
Still broken in kernel-2.6.24.3-12.fc8.

FC4 i386 Disc 1 CD-R: 665305088/665434112 bytes
Fedora 9 Alpha i386 rescuecd CD-RW: 117424128/117508096 bytes
Fedora 9 Alpha i386 rescuecd CD-R: 117424128/117508096 bytes
Comment 23 Andre Robatino 2008-03-10 07:54:06 EDT
I noticed that Fedora 9 Alpha was released with CD as well as DVD ISOs.  Will
this also be true for F9 final?  Since this bug manifests more easily with CDs,
it would result in a lot more reports of mediacheck failures, making it more
likely that this would get fixed.  (I never understood why providing CD images
was so difficult, and yes I know about Fedora Unity and jigdo, but those don't
help people who don't already have a working Fedora box.)
Comment 24 Andre Robatino 2008-03-16 23:23:00 EDT
Still broken in kernel-2.6.24.3-34.fc8.

FC4 i386 Disc 1 CD-R: 665427968/665434112 bytes
Fedora 9 Alpha i386 rescuecd CD-RW: 117424128/117508096 bytes
Fedora 9 Alpha i386 rescuecd CD-R: 117424128/117508096 bytes
Comment 25 Andre Robatino 2008-03-26 05:11:50 EDT
Burned the 6 F9-Beta CD images to CD-RWs in default TAO mode without padding,
using wodim.  Testing them using the original F7 mediacheck, the discs all
failed.  Previously I had only tested F7 using CD-Rs.  So apparently this bug
was not fixed even in F7.  I then tried reburning one disc using DAO mode
without padding, and another using TAO mode with padding.  Both passed.  So
apparently either DAO mode or padding is enough to allow the discs to be read
properly.  I then used rawread to read each of these two discs while running F8,
and both discs could be read properly.
Comment 26 Tom Horsley 2008-04-27 16:11:12 EDT
Perhaps bug 444344 is another instance of this
Comment 27 Andre Robatino 2008-04-27 16:27:32 EDT
Using the rawread script mentioned above would be the way to tell - if it is,
the read would fail about 100K or less from the end.  (BTW, this method can be
used instead of mediacheck to check burned Fedora discs, without having to
reboot.  Just pipe through sha1sum and compare to the expected value.)
Comment 28 Michael Schwendt 2008-04-27 17:43:54 EDT
For comparison, bug 440343 is not due to I/O errors.
dmesg is free of such errors. "readcd" can read the burnt
image just fine.
Comment 29 Andre Robatino 2008-05-01 13:36:46 EDT
I also find that I cannot use readcd to properly read off the ISO file from a
DVD, even a properly padded one which rawread has no trouble with.  What I get
contains extra data after the end of the actual ISO.  See comments 12 and 13 of
bug #440343.  Could this be related to the readahead bug?
Comment 30 Andre Robatino 2008-06-07 19:27:42 EDT
Still broken in F9 with kernel-2.6.25.4-30.fc9.i686.  Changing version to 9.
Comment 31 Andre Robatino 2008-11-28 00:13:45 EST
Still broken in F10 with kernel-2.6.27.5-117.fc10.i686.  Changing Version to 10.  (Should this be made a duplicate of bug #186512?)
Comment 32 Alan Cox 2008-11-28 06:09:52 EST
I think you are right

*** This bug has been marked as a duplicate of bug 186512 ***

Note You need to log in before you can comment on or make changes to this bug.