Bug 388911
Summary: | Data corruption in large files (guessing 1.5 - 2 Gigs as starting to cause issue) | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | leaf6s |
Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> |
Status: | CLOSED NOTABUG | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | medium | Docs Contact: | |
Priority: | low | ||
Version: | 8 | CC: | esandeen |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2008-01-04 22:05:46 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Attachments: |
Description
leaf6s
2007-11-17 21:23:08 UTC
In your md5sum example, where you copy & see different md5sums... maybe a dumb question from me, but is torrent still running & updating these files? No it's not! :0) It's after I had given up, so the torrent download had been terminated. Just checking. Is this something you can still reproduce? Can you reproduce it w/o torrent? Can you try "fuser" on the files in question to be sure they're not being touched? Is their size changing? The fact that you said you see this on both ext3 and xfs makes me suspicious that something else is going on here. Yes I still have these problems if I simply copy the existing files. What output do you wish to see from fuser (i.e which arguments should I pass to it)? Without any arguments, it returns nothing. Since my last attempt, I changed the files to be read-only and used ls -l to get the following: -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso I have been performing the sha1sum etc, sine then and they have not changed size nor timestamps. Just to double check, I did the following: [Fedora-8-dvd-x86_64]# cp -pr Fedora-8-x86_64-DVD.iso2 Fedora-8-x86_64-DVD.iso23 [Fedora-8-dvd-x86_64]# diff Fedora-8-x86_64-DVD.iso2 Fedora-8-x86_64-DVD.iso23 Binary files Fedora-8-x86_64-DVD.iso2 and Fedora-8-x86_64-DVD.iso23 differ [Fedora-8-dvd-x86_64]# ls -l Fedora-8-x86_64-DVD.iso* -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso2 -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso23 Very weird. Does that persist after an umount/remount of the filesystem (or a reboot if it's root?) Yes it does...unfortunately. (In reply to comment #5) > Just to double check, I did the following: > [Fedora-8-dvd-x86_64]# cp -pr Fedora-8-x86_64-DVD.iso2 Fedora-8-x86_64-DVD.iso23 > [Fedora-8-dvd-x86_64]# diff Fedora-8-x86_64-DVD.iso2 Fedora-8-x86_64-DVD.iso23 > Binary files Fedora-8-x86_64-DVD.iso2 and Fedora-8-x86_64-DVD.iso23 differ > [Fedora-8-dvd-x86_64]# ls -l Fedora-8-x86_64-DVD.iso* > -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso > -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso2 > -r--r--r-- 1 root root 3876407296 2007-11-13 18:26 Fedora-8-x86_64-DVD.iso23 > Can you boot kernel 2.6.23.1-42 and see if that still happens? I rebooted using the suggested kernel, but the problem still persists. A recent update, provided an updated kernel, so I dd the following: [videos]# uname -r 2.6.23.9-85.fc8 [videos]#ll -h total 6.8G -rw-rw-r-- 1 pug pug 247M 2007-12-09 23:06 video_294.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:07 video_294.avi.sha -rw-rw-r-- 1 pug pug 270M 2007-12-09 23:07 video_295.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:07 video_295.avi.sha -rw-rw-r-- 1 pug pug 307M 2007-12-09 23:07 video_296.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:07 video_296.avi.sha -rw-rw-r-- 1 pug pug 300M 2007-12-09 23:07 video_297.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:07 video_297.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:07 video_298.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:07 video_298.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:08 video_299.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:08 video_299.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:08 video_300.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:08 video_300.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:08 video_301.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:08 video_301.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:08 video_302.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:08 video_302.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:08 video_303.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:08 video_303.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:09 video_304.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:09 video_304.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:09 video_305.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:09 video_305.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:09 video_306.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:09 video_306.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:09 video_307.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:09 video_307.avi.sha -rw-rw-r-- 1 pug pug 258M 2007-12-09 23:09 video_308.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:09 video_308.avi.sha -rw-rw-r-- 1 pug pug 269M 2007-12-09 23:10 video_309.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:10 video_309.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:10 video_310.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:10 video_310.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:10 video_311.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:10 video_311.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:10 video_312.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:10 video_312.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:10 video_313.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:11 video_313.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:11 video_314.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:11 video_314.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:11 video_315.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:11 video_315.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:11 video_316.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:11 video_316.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:11 video_317.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:11 video_317.avi.sha -rw-rw-r-- 1 pug pug 276M 2007-12-09 23:12 video_318.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:12 video_318.avi.sha -rw-rw-r-- 1 pug pug 275M 2007-12-09 23:12 video_319.avi -rw-rw-r-- 1 pug pug 85 2007-12-09 23:12 video_319.avi.sha [videos]# sha1sum -c *sha video_294.avi: OK video_295.avi: OK video_296.avi: OK video_297.avi: OK video_298.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_299.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_300.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_301.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_302.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_303.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_304.avi: OK video_305.avi: OK video_306.avi: OK video_307.avi: OK video_308.avi: OK video_309.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_310.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_311.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_312.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_313.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_314.avi: OK video_315.avi: OK video_316.avi: OK video_317.avi: OK video_318.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_319.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match [videos]# sha1sum -c *sha video_294.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_295.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_296.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_297.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_298.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_299.avi: OK video_300.avi: OK video_301.avi: OK video_302.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_303.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_304.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_305.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_306.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_307.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_308.avi: OK video_309.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_310.avi: OK video_311.avi: OK video_312.avi: OK video_313.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_314.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_315.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_316.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_317.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_318.avi: OK video_319.avi: OK [videos]# sha1sum -c *sha video_294.avi: OK video_295.avi: OK video_296.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_297.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_298.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_299.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_300.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_301.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_302.avi: OK video_303.avi: OK video_304.avi: OK video_305.avi: OK video_306.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_307.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_308.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_309.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_310.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_311.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_312.avi: OK video_313.avi: OK video_314.avi: OK video_315.avi: OK video_316.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_317.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_318.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match video_319.avi: FAILED sha1sum: WARNING: 1 of 1 computed checksum did NOT match As you can see, the results from file verification are still not trustworthy and fluctuate from run to run... Can we try to get an idea of how these files differ.... using "cmp -l" of good file / bad file, or hexdump -C (probably only of the first few MB or so, or up to the first handful of bad data) I realize good file / bad file might be problematic; perhaps a compare (cmp) between your local problematic filesystem and an nfs-mounted "good" image? Or, you could dd if=Fedora-8-x86_64-DVD.iso bs=1M count=8 | hexdump -C > hexdump.txt, and we could compare that to known good... (assuming we find the corruption in the first 8M...) Basically I'd like to see the nature of the corruptions - bitflips, null regions, or whatnot. Since you say each access of the file (subsequent md5sums) changes, maybe even comparing subsequent hexdumps of the same file on your system would give us a clue about what differs. Created attachment 290307 [details]
Comparison of Fedora-8-x86_64-DVD.iso & a duplicate of it on my fs
Created attachment 290308 [details]
2nd comparison of Fedora-8-x86_64-DVD.iso & a duplicate of it on my fs
Created attachment 290309 [details]
3rd comparison of Fedora-8-x86_64-DVD.iso & a duplicate of it on my fs
Eric, That's a difficult one, because no media in this system (not even DVDs) give any consisten results. What I have done is: [Fedora-8-dvd-x86_64]# cp Fedora-8-x86_64-DVD.iso /tmp [Fedora-8-dvd-x86_64]# cmp -l Fedora-8-x86_64-DVD.iso /tmp/Fedora-8-x86_64-DVD.iso > cmpData1.txt [Fedora-8-dvd-x86_64]# cmp -l Fedora-8-x86_64-DVD.iso /tmp/Fedora-8-x86_64-DVD.iso > cmpData2.txt [Fedora-8-dvd-x86_64]# cmp -l Fedora-8-x86_64-DVD.iso /tmp/Fedora-8-x86_64-DVD.iso > cmpData3.txt [Fedora-8-dvd-x86_64]# ll -h * txt -rw-r--r-- 1 root root 89K 2007-12-23 18:12 cmpData1.txt -rw-r--r-- 1 root root 13K 2007-12-23 18:27 cmpData2.txt -rw-r--r-- 1 root root 165K 2007-12-23 18:41 cmpData3.txt As you can see the resulting files vary in size although no commands were issued to update or amend the files in any way. Please find in the previous 3 entries to this bug. My deepest apologies, but this is the closest that I can get to what I believe you are asking for. I can also add that having attempt to performed: [Fedora-8-dvd-x86_64]# dd if=Fedora-8-x86_64-DVD.iso bs=1M count=8 | hexdump -C > hexdump1.txt 8+0 records in 8+0 records out 8388608 bytes (8.4 MB) copied, 3.83822 s, 2.2 MB/s [Fedora-8-dvd-x86_64]# dd if=Fedora-8-x86_64-DVD.iso bs=1M count=8 | hexdump -C > hexdump2.txt 8+0 records in 8+0 records out 8388608 bytes (8.4 MB) copied, 3.75646 s, 2.2 MB/s [Fedora-8-dvd-x86_64]# diff hexdump1.txt hexdump2.txt [Fedora-8-dvd-x86_64]# I then increased the count to 30 and then to 90, with the same result. Frustrated I then tried: [Fedora-8-dvd-x86_64]# dd if=Fedora-8-x86_64-DVD.iso bs=1M of=/tmp/Fedora-8-x86_64-DVD.iso 3696+1 records in 3696+1 records out 3876407296 bytes (3.9 GB) copied, 148.562 s, 26.1 MB/s [Fedora-8-dvd-x86_64]# diff Fedora-8-x86_64-DVD.iso /tmp/Fedora-8-x86_64-DVD.iso Binary files Fedora-8-x86_64-DVD.iso and /tmp/Fedora-8-x86_64-DVD.iso differ I don't know if the above will be of any help, but just in case... > ...no media in this system (not even DVDs) give any consisten results. Are you saying that even files on read-only media, such as a mounted DVD, also gives you these different check errors? FWIW, from comment #12, every one of your errors that I checked was a single-bit error. Have you run memtest on this box? -Eric What I'm getting at, is this looks like a hardware error to me, either bad memory, bad cables, bad controller, or whatnot... Tho I'm a little surprised that the machine doesn't crash & burn... >Are you saying that even files on read-only media, such as a mounted DVD, also >gives you these different check errors? Yes they do, which is why I been having to install Fedora via the net. as I can never get the DVD to be completely downloaded... >FWIW, from comment #12, every one of your errors that I checked was a single-bit >error. Have you run memtest on this box? As part of the install process, one can run memtest which I did and that ran for over 7 hours without detecting any errors. I must confess that I don't feeling that my AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ is performing anywhere near what I was expecting or hoping for, but I don't have anything to compare it with. Do you know of any program which could be used to test the processor functionality for possible defects? I already spent a lot of money upgrading my system this year, so I cannot afford to replace everything again! Is the power supply OK? (In reply to comment #20) > Is the power supply OK? Err... I presume so, it's a good quality power supply bought in early Jan 2007 I think that I have finaly found the problem! Whilst cleaning my CPU cooler yesterday, I decided to test each of my memory modules - by only having one of them inserted into my MB during any given test. Although none of them had any errors detected by memtest86+, the sha1sum always failed when either of my CMX1024-3200C2PT memory modules was installed. It appears that I was unlucky and have a matched pair of memory modules, both of which either do not 'like' my motherboard or are faulty... The matched pair CMX512-3200C2PT memory modules gave consistent results when sha1sum was run, against the scenarios at the beginning of this case and have continued to do so after permanently removing the CMX1024-3200C2PT matched pair modules and several reboots. Thanks to all for the help, assistance, ideas and patients afforded. |