Bug 892206

Summary: Using a pipe in a command substitution triggers wrong return codes
Product: Red Hat Enterprise Linux 5 Reporter: Dennis Kanbier <dennis>
Component: kshAssignee: Michal Hlavinka <mhlavink>
Status: CLOSED ERRATA QA Contact: Martin Kyral <mkyral>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 5.8CC: fkrska, ovasik
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: ksh-20100621-13.el5 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-01 00:25:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 921048, 928849    
Attachments:
Description Flags
repro case for ksh bug? running under vmware (RH 5)
none
5.8 installed RPMS none

Description Dennis Kanbier 2013-01-05 21:02:38 UTC 
Description of problem:

Using grep in a ksh script after using a nested pipe in an exec statement can result in a false positive return code from grep later in the script. 

Version-Release number of selected component (if applicable):

ksh-20100621-5.el5_8.1

How reproducible:

Source of an example script, run multiple times and grep will report false positives:

#!/bin/ksh
#
# Command seems irrelevant, using a pipe in the exec
# statement results in a false positive on the grep
# commands below
#
#echo foo | cut -c $(echo bar | grep bar)	# FAILS
echo foo | grep $(echo bar | sort -u)		# FAILS
#echo foo | grep $(echo bar) 			# OK

# Greps
grep foo /etc/passwd  && echo "1: false positive?"
grep foo /etc/passwd  && echo "2: false positive?"
grep foo /etc/passwd  && echo "3: false positive?"
grep foo /etc/passwd  && echo "4: false positive?"
grep foo /etc/passwd  && echo "5: false positive?"
grep foo /etc/passwd  && echo "6: false positive?"

Steps to Reproduce:
1. Save the above code in file test.ksh
2. Make it executable
3. Run with "/usr/bin/watch -n1 test.ksh"
  
Actual results:

False positive messages on screen, indicating "foo" is present in /etc/passwd. Also the messages aren't persistent, they can change from like 1,2,3,4,5,6 to 3,4 ony 6 or none.

Expected results:

No messages on screen as foo is not present in /etc/passwd.

Additional info:

Also tested under bash, no messages are reported. Also tried under RHEL6, no messages are reported. If you don't use a pipe in the exec statement the problem goes away. You can test this by uncommenting the OK line and comment the FAILS line in the example code.
Comment 1 Dennis Kanbier 2013-01-07 09:30:51 UTC 
To make it even more simple:

host> ksh
host> grep bar /etc/foo;echo $?
grep: /etc/foo: No such file or directory
2
host> echo foo | grep $(echo foo | grep foo)
foo
host> grep bar /etc/foo;echo $?
grep: /etc/foo: No such file or directory
0
Comment 2 warkruid 2013-01-08 10:32:39 UTC 
Created attachment 674671 [details]
repro case for ksh bug? running under vmware (RH 5)

Tried this on:
Slackware 14 (hardware): result 0 2 2 with no variations
Puppy linux (hardware): result 0 2 2 with no variations

Red hat 5.8 on vmware: result 0 0 0 or 0 2 0 or 0 2 2 or 0 0 2
Comment 3 Dennis Kanbier 2013-01-08 13:13:37 UTC 
Created attachment 674812 [details]
5.8 installed RPMS

Was not able to reproduce on RH5.6 on vmware. 

After updating the virtual machine to RH5.8 the problem occurs again. Booted the 5.6 kernel in 5.8 but the problem persists.

Added the output of "rpm -qa" on the RH5.8 system.
Comment 4 Dennis Kanbier 2013-01-08 13:18:56 UTC 
In contrary to this bug's title it's not only gr
Comment 5 Dennis Kanbier 2013-01-09 10:08:45 UTC 
In contrary to this bug's title (edited) it's not only the return code of grep which is incorrect, but return codes in general:

[root@host ~]# ksh
# /bin/false; echo $?
1
# foo | grep $(echo foo | grep foo)
foo
# /bin/false; echo $?
1
# /bin/false; echo $?
0
# /bin/false; echo $?
1
#
Comment 6 Michal Hlavinka 2013-01-10 16:59:32 UTC 
reproducible
Comment 7 Dennis Kanbier 2013-01-15 20:27:48 UTC 
Reproducible on RHEL5.9 as well:

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.9 (Tikanga)
#
# ksh --version
  version         sh (AT&T Research) 93t+ 2010-06-21
#
# rpm -qa |grep ksh
ksh-20100621-12.el5
#
# echo foo | grep $(echo foo | sort)
foo
# /bin/false;echo $?
0
#
Comment 8 Michal Hlavinka 2013-01-16 13:25:46 UTC 
What do you mean by "as well"? AFAIK this is reproducible only in 5.9 and not in 6.4.
Comment 9 Dennis Kanbier 2013-01-16 13:44:40 UTC 
I noticed a RHEL5.9 release yesterday, the bug was discovered on RHEL5.8. So the issue is still reproducible in the newest RHEL5 release (5.9). 

It's not reproducible on RHEL6.4, that is correct.
Comment 10 Michal Hlavinka 2013-01-16 13:54:55 UTC 
Something fishy here. RHEL6 does not reproduce this, but RHEL6 ksh package rebuilt for RHEL5 does. Interesting...
Comment 11 Dennis Kanbier 2013-01-23 08:31:17 UTC 
Found anything yet? If it's not the KSH software it might be a library which is kind of worrying to me.
Comment 12 Michal Hlavinka 2013-01-23 16:27:07 UTC 
The difference is not in library, but in kernel. With kernel <= 2.6.31.12 it reproduces this problem. With kernel >= 2.6.32-0.14.rc0.git18 it works. I don't know what causes this difference yet. Using several (much) older kernels does not change reproducibility, so I guess it's not caused by any bug in kernel. Also I was unsuccessful with debugging this so far. Interesting that problem does not reproduce when I try to use strace.
Comment 13 Dennis Kanbier 2013-01-23 18:35:40 UTC 
All right then, I thought it was just me misinterpreting the strace output. Thank you for the update.
Comment 14 Michal Hlavinka 2013-01-25 14:17:24 UTC 
Found a bug in ksh. Here are test packages if you want to test them:
http://mhlavink.fedorapeople.org/bz892206/
Comment 16 Dennis Kanbier 2013-01-25 15:06:25 UTC 
Looking good, not reproducible with the new package as far as I can tell.
Comment 22 errata-xmlrpc 2013-10-01 00:25:38 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1351.html