Red Hat Bugzilla – Bug 481765
grep-mem-exhausted.patch introduces regressions
Last modified: 2012-11-20 01:05:33 EST
[Originally reported by Simon Vallet as bug 440258.] grep-2.5.3-2 may fail on large files: $ grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file $ $ tail /env/cns/tmp/test_file > /env/cns/tmp/test_file.tail $ grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file.tail ../lecures_NM_eland_result.seq-15290226 TGGGGGAGGGTTTTTTGTTTTATAAAGATGATC [...] $ 1. wget http:[...]/test_file (warning: 1gb) 2. grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file The "test_file" contains some text at the beginnig and at the end, but most of it is occupied by a huge block of nul bytes. Actual results: No output Expected results: The last line of this file contains the pattern (see above) Additional info: Re-building without patch #9 (grep-mem-exhausted.patch) solves the problem.
Created attachment 330093 [details] The patch to be removed. This patch, meant to fix bug 198165, causes the problem.
With grep-mem-exhausted.patch, a match might be missed if it is on a line longer than 200 MB and if the matching substring itself is almost 200 MB long. That behavior was intentional. But this bug shows that the patch can also cause missing matches in the remainder of the file. I think that the problem is caused by the fact that the variables `save' and `residue' in function grep() do not reflect the truncation in the fillbuf() function. For the record: Moving the truncation to the function grep() and adjusting several variables so that the invariants are kept might be a way to fix the mem-exhaused patch so that it does what it was intended for. But, IMHO, it is better to drop the patch, see https://bugzilla.redhat.com/show_bug.cgi?id=198165#c3 Fixed in grep-2.5.3-3.
Oops, removing the patch introduces another problem: $ dd if=/dev/zero of=sparse seek=700000000 bs=1 count=1 $ (ulimit -v 100000 ; grep foo sparse ; echo $?) grep: memory exhausted 1 But the exit code should be 2, not 1!
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle. Changing version to '11'. More information and reason for this action is here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Should be fixed in grep-2.6.1 in rawhide. I am going to rebuild it for F11 soon.
I was unable the grab the original file, so I tested with similar file (about 1GB without newlines). It pushed my machine to knees, but it works ;)
grep-2.6.1-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/grep-2.6.1-1.fc11
grep-2.6.1-1.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update grep'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/grep-2.6.1-1.fc11
grep-2.6.3-1.fc11 has been submitted as an update for Fedora 11. http://admin.fedoraproject.org/updates/grep-2.6.3-1.fc11
grep-2.6.3-1.fc11 has been pushed to the Fedora 11 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update grep'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/grep-2.6.3-1.fc11
This message is a reminder that Fedora 11 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 11. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '11'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 11's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 11 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
grep-2.6.3-1.fc11 has been pushed to the Fedora 11 stable repository. If problems still persist, please make note of it in this bug report.