Bug 481765 - grep-mem-exhausted.patch introduces regressions
grep-mem-exhausted.patch introduces regressions
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: grep (Show other bugs)
11
x86_64 Linux
low Severity medium
: ---
: ---
Assigned To: Jaroslav Škarvada
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-01-27 09:53 EST by Stepan Kasal
Modified: 2012-11-20 01:05 EST (History)
4 users (show)

See Also:
Fixed In Version: grep-2.6.3-1.fc11
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 440258
: 483073 (view as bug list)
Environment:
Last Closed: 2010-06-15 12:07:04 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
The patch to be removed. (923 bytes, patch)
2009-01-27 09:58 EST, Stepan Kasal
no flags Details | Diff

  None (edit)
Description Stepan Kasal 2009-01-27 09:53:58 EST
[Originally reported by Simon Vallet as bug 440258.]

grep-2.5.3-2 may fail on large files:

$ grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file
$

$ tail /env/cns/tmp/test_file > /env/cns/tmp/test_file.tail
$ grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file.tail 
../lecures_NM_eland_result.seq-15290226 TGGGGGAGGGTTTTTTGTTTTATAAAGATGATC [...]
$

1. wget http:[...]/test_file (warning: 1gb)
2. grep '12884102-12884161_12884190-12884243;9,5;28' /env/cns/tmp/test_file

The "test_file" contains some text at the beginnig and at the end, but most of it is occupied by a huge block of nul bytes.

Actual results:
No output

Expected results:
The last line of this file contains the pattern (see above)

Additional info:
Re-building without patch #9 (grep-mem-exhausted.patch) solves the problem.
Comment 1 Stepan Kasal 2009-01-27 09:58:38 EST
Created attachment 330093 [details]
The patch to be removed.

This patch, meant to fix bug 198165, causes the problem.
Comment 2 Stepan Kasal 2009-01-27 10:19:03 EST
With grep-mem-exhausted.patch, a match might be missed if it is on a line longer than 200 MB and if the matching substring itself is almost 200 MB long.  That behavior was intentional.

But this bug shows that the patch can also cause missing matches in the remainder of the file.  I think that the problem is caused by the fact that the variables `save' and `residue' in function grep() do not reflect the truncation in the fillbuf() function.  For the record: Moving the truncation to the function grep() and adjusting several variables so that the invariants are kept might be a way to fix the mem-exhaused patch so that it does what it was intended for.

But, IMHO, it is better to drop the patch, see
https://bugzilla.redhat.com/show_bug.cgi?id=198165#c3

Fixed in grep-2.5.3-3.
Comment 3 Stepan Kasal 2009-02-03 10:36:07 EST
Oops, removing the patch introduces another problem:

$ dd if=/dev/zero of=sparse seek=700000000 bs=1 count=1
$ (ulimit -v 100000 ; grep foo sparse ; echo $?)
grep: memory exhausted
1

But the exit code should be 2, not 1!
Comment 4 Bug Zapper 2009-06-09 06:54:08 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 5 Jaroslav Škarvada 2010-03-30 11:13:59 EDT
Should be fixed in grep-2.6.1 in rawhide. I am going to rebuild it for F11 soon.
Comment 6 Jaroslav Škarvada 2010-03-31 04:12:14 EDT
I was unable the grab the original file, so I tested with similar file (about 1GB without newlines). It pushed my machine to knees, but it works ;)
Comment 7 Fedora Update System 2010-03-31 10:08:35 EDT
grep-2.6.1-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/grep-2.6.1-1.fc11
Comment 8 Fedora Update System 2010-04-03 00:44:03 EDT
grep-2.6.1-1.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update grep'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/grep-2.6.1-1.fc11
Comment 9 Fedora Update System 2010-04-07 18:17:37 EDT
grep-2.6.3-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/grep-2.6.3-1.fc11
Comment 10 Fedora Update System 2010-04-10 06:29:56 EDT
grep-2.6.3-1.fc11 has been pushed to the Fedora 11 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update grep'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/grep-2.6.3-1.fc11
Comment 11 Bug Zapper 2010-04-27 08:48:32 EDT
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 12 Fedora Update System 2010-06-15 12:06:44 EDT
grep-2.6.3-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.