Bug 43319

Summary: zgrep prints extraneous blank lines
Product: [Retired] Red Hat Linux Reporter: Jay Berkenbilt <ejb>
Component: gzipAssignee: Trond Eivind Glomsrxd <teg>
Status: CLOSED RAWHIDE QA Contact: David Lawrence <dkl>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: karsten
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2001-06-04 03:39:07 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
perl version of zgrep that behaves better wrt sigpipe none

Description Jay Berkenbilt 2001-06-03 04:02:43 UTC
Description of Problem:
zgrep prints extraneous blank lines

How Reproducible:
always reproducible

Steps to Reproduce:
1. do a zgrep -- observe the extra blank line.  For example: zgrep a 	 
/dev/null generates a blank line.  This is especially noticeable when
zgrepping through large numbers of files. It essentially renders the
command useless.  Sometimes I zgrep through hundreds of gzipped rmail files
only to get a line of text surrounded by hundreds of blank lines.

Actual Results:
One blank line is output per file on the commandline

Expected Results:
This should not happen.

Additional Information:
This happens because there is what appears to me to be an extraneous call
to "echo" at the end of the loop that actually does the work.  This is line
67 of /usr/bin/zgrep in RedHat 7.1.  Removing that line solves the problem.

This echo is added by gzip-1.3-zgreppipe.patch which seems to be have been
created in response to bug #24104 which I unfortunately do not have
permission to look at.  I don't see any reason why it is necessary.  zgrep
something * | less works fine without the echo.

Maybe Trond Eivind Glomsrxd <teg> who, from the ChangeLog, seems
to be responsible for this, knows why the echo is there.

Gratuitious Remarks:
If only I had noticed this during Fisher or Wolverine....

Comment 1 Trond Eivind Glomsrxd 2001-06-03 23:33:27 UTC
The bug is now readable... basically, we need an echo there - if not, the trap
doesn't work.

Comment 2 Jay Berkenbilt 2001-06-04 01:22:30 UTC
Hmm.  It seems somehow "suboptimal" to ruin the command in order to make this
feature work.  Would an echo -n enable the trap to work?  Maybe I'll send in a
better fix.  Thanks for making the other bug readable.

Comment 3 Trond Eivind Glomsrxd 2001-06-04 01:29:04 UTC
"echo -n" doesn't work - and being able to use less on the output is a rather
important feature.

Comment 4 Jay Berkenbilt 2001-06-04 03:25:13 UTC
Okay, I understand now.  The issue isn't that you can't pipe to less without
this; it's that the behavior is broken when you quit less.  This is because each
invocation of gzip | grep gets a sigpipe but the sigpipe doesn't get passed back
to the parent process because it's not actually writing anything to stdout,
which is the pipe that is now closed.  Adding the echo to the loop causes the
zgrep program itself to generate some output so that it can actually get the
PIPE signal itself.

The problem is that adding the extra blank line *changes the output* of zgrep so
that it is wrong.  A better fix would be for the output of the grep commands to
be written to stdout by zgrep itself so that zgrep would get the PIPE signal
when less (or whatever you pipe it to) exits.  This means that if you quit less,
zgrep won't actually get a PIPE signal until you get some output, but that makes
it no different from any other command.  (Try grepping for lkfjsodiuwer in some
iso image and pipe that to less.  It just sits there when exit less until the
grep finishes or you hit CTRL-C.  Surely no one would suggest that grep should
periodically just generate some output just to make sure it gets a sigpipe if
the thing you're piping it to exits!)

I've done a more-or-less literal translation of zgrep from bourne shell to perl
except that my perl version runs gzip | grep as a pipe and prints everything it
outputs to stdout.  This way, any output of zgrep | less will cause a sigpipe
which will cause zgrep to exit.  This puts zgrep on the same footing as any
other program and has the compelling advantage of not altering the output of
zgrep.  Also, in perl, you don't need to pipe things through sed all the time,
so even if perl itself takes longer to invoke than bash, the perl version of
zgrep should actually be just fine from a performance angle, and certainly you
can count on any RedHat system having perl.

I hope you'll consider my zgrep replacement. I've tested all the features to
make sure that they work properly.  For example, the invocation of the gzip
commands are such that either filenames or patterns can have embedded spaces. 
Also, zgrep recognizes -h and -l and handles them.  It also handles the case of
a single filename and of no filename given specially.  My rewritten zgrep
behaves identically to /usr/bin/zgrep in all these cases (except that it doesn't
output a bunch of extra blank lines).  My zgrep also remembers to unbuffer its
output which makes it more useful when piping to things.

I've attached my new zgrep to this bug report.

Comment 5 Jay Berkenbilt 2001-06-04 03:26:06 UTC
Created attachment 20199 [details]
perl version of zgrep that behaves better wrt sigpipe

Comment 6 Jay Berkenbilt 2001-06-04 03:39:02 UTC
A less severe fix would be to detect when the most recently run child command
has itself exit because of a sigpipe.  If you're willing to "know" that SIGPIPE
= 13 (and that $? = 128 + signal when a child process is killed with a signal)
then you can simply replace the offending "echo" command with

  test "$r" -eq 141 && exit $res

Try it.  It works.  (I really wish I had thought of this BEFORE spending 45
minutes rewriting zgrep in perl and posting it.  Oh well.)

Comment 7 Trond Eivind Glomsrxd 2001-06-04 14:40:57 UTC
Fixed in gzip-1.3-13 - thanks for your patch, it was very helpful. You should
probably submit it to the gzip maintainer as well.

Comment 8 Jay Berkenbilt 2001-06-06 01:46:30 UTC
I have now sent a patch to bug-gzip.  Note that the whole trap business
actually becomes unnecessary since zgrep itself isn't generating any output. 
The following patch is sufficient when applied to a clean gzip-1.3.tar.gz as
downloaded from alpha.gnu.org on 6/6/01 (as found from the source rpm).  Of
course the trap stuff is harmless in case zgrep should ever itself generate
output, but it's also probably unnecessary since the default action on SIGPIPE
would be to exit.

--- /home/ejb/tmp/zgrep.in~	Tue Jun  5 21:44:20 2001
+++ /home/ejb/tmp/zgrep.in	Tue Jun  5 21:44:28 2001
@@ -64,5 +64,6 @@
     r=$?
   fi
   test "$r" -ne 0 && res="$r"
+  test "$r" -eq 141 && exit $res
 done
 exit $res


Comment 9 Jay Berkenbilt 2001-06-06 01:47:26 UTC
For completeness, my message to bug-gzip also apologized for the hardcoding of
141 and suggested an autoconf test may be in order, but I didn't actually
provide one.

Comment 10 Trond Eivind Glomsrxd 2001-08-27 16:23:06 UTC
*** Bug 52594 has been marked as a duplicate of this bug. ***

Comment 11 Trond Eivind Glomsrxd 2001-09-24 22:17:33 UTC
*** Bug 53975 has been marked as a duplicate of this bug. ***

Comment 12 Need Real Name 2001-10-04 12:26:43 UTC
*** Bug 54327 has been marked as a duplicate of this bug. ***