Description of Problem:
zgrep prints extraneous blank lines
Steps to Reproduce:
1. do a zgrep -- observe the extra blank line. For example: zgrep a
/dev/null generates a blank line. This is especially noticeable when
zgrepping through large numbers of files. It essentially renders the
command useless. Sometimes I zgrep through hundreds of gzipped rmail files
only to get a line of text surrounded by hundreds of blank lines.
One blank line is output per file on the commandline
This should not happen.
This happens because there is what appears to me to be an extraneous call
to "echo" at the end of the loop that actually does the work. This is line
67 of /usr/bin/zgrep in RedHat 7.1. Removing that line solves the problem.
This echo is added by gzip-1.3-zgreppipe.patch which seems to be have been
created in response to bug #24104 which I unfortunately do not have
permission to look at. I don't see any reason why it is necessary. zgrep
something * | less works fine without the echo.
Maybe Trond Eivind Glomsrxd <email@example.com> who, from the ChangeLog, seems
to be responsible for this, knows why the echo is there.
If only I had noticed this during Fisher or Wolverine....
The bug is now readable... basically, we need an echo there - if not, the trap
Hmm. It seems somehow "suboptimal" to ruin the command in order to make this
feature work. Would an echo -n enable the trap to work? Maybe I'll send in a
better fix. Thanks for making the other bug readable.
"echo -n" doesn't work - and being able to use less on the output is a rather
Okay, I understand now. The issue isn't that you can't pipe to less without
this; it's that the behavior is broken when you quit less. This is because each
invocation of gzip | grep gets a sigpipe but the sigpipe doesn't get passed back
to the parent process because it's not actually writing anything to stdout,
which is the pipe that is now closed. Adding the echo to the loop causes the
zgrep program itself to generate some output so that it can actually get the
PIPE signal itself.
The problem is that adding the extra blank line *changes the output* of zgrep so
that it is wrong. A better fix would be for the output of the grep commands to
be written to stdout by zgrep itself so that zgrep would get the PIPE signal
when less (or whatever you pipe it to) exits. This means that if you quit less,
zgrep won't actually get a PIPE signal until you get some output, but that makes
it no different from any other command. (Try grepping for lkfjsodiuwer in some
iso image and pipe that to less. It just sits there when exit less until the
grep finishes or you hit CTRL-C. Surely no one would suggest that grep should
periodically just generate some output just to make sure it gets a sigpipe if
the thing you're piping it to exits!)
I've done a more-or-less literal translation of zgrep from bourne shell to perl
except that my perl version runs gzip | grep as a pipe and prints everything it
outputs to stdout. This way, any output of zgrep | less will cause a sigpipe
which will cause zgrep to exit. This puts zgrep on the same footing as any
other program and has the compelling advantage of not altering the output of
zgrep. Also, in perl, you don't need to pipe things through sed all the time,
so even if perl itself takes longer to invoke than bash, the perl version of
zgrep should actually be just fine from a performance angle, and certainly you
can count on any RedHat system having perl.
I hope you'll consider my zgrep replacement. I've tested all the features to
make sure that they work properly. For example, the invocation of the gzip
commands are such that either filenames or patterns can have embedded spaces.
Also, zgrep recognizes -h and -l and handles them. It also handles the case of
a single filename and of no filename given specially. My rewritten zgrep
behaves identically to /usr/bin/zgrep in all these cases (except that it doesn't
output a bunch of extra blank lines). My zgrep also remembers to unbuffer its
output which makes it more useful when piping to things.
I've attached my new zgrep to this bug report.
Created attachment 20199 [details]
perl version of zgrep that behaves better wrt sigpipe
A less severe fix would be to detect when the most recently run child command
has itself exit because of a sigpipe. If you're willing to "know" that SIGPIPE
= 13 (and that $? = 128 + signal when a child process is killed with a signal)
then you can simply replace the offending "echo" command with
test "$r" -eq 141 && exit $res
Try it. It works. (I really wish I had thought of this BEFORE spending 45
minutes rewriting zgrep in perl and posting it. Oh well.)
Fixed in gzip-1.3-13 - thanks for your patch, it was very helpful. You should
probably submit it to the gzip maintainer as well.
I have now sent a patch to firstname.lastname@example.org. Note that the whole trap business
actually becomes unnecessary since zgrep itself isn't generating any output.
The following patch is sufficient when applied to a clean gzip-1.3.tar.gz as
downloaded from alpha.gnu.org on 6/6/01 (as found from the source rpm). Of
course the trap stuff is harmless in case zgrep should ever itself generate
output, but it's also probably unnecessary since the default action on SIGPIPE
would be to exit.
--- /home/ejb/tmp/zgrep.in~ Tue Jun 5 21:44:20 2001
+++ /home/ejb/tmp/zgrep.in Tue Jun 5 21:44:28 2001
@@ -64,5 +64,6 @@
test "$r" -ne 0 && res="$r"
+ test "$r" -eq 141 && exit $res
For completeness, my message to bug-gzip also apologized for the hardcoding of
141 and suggested an autoconf test may be in order, but I didn't actually
*** Bug 52594 has been marked as a duplicate of this bug. ***
*** Bug 53975 has been marked as a duplicate of this bug. ***
*** Bug 54327 has been marked as a duplicate of this bug. ***