Bug 103072

Summary: escapes (\\) in backquoted pipelines broken
Product: [Retired] Red Hat Linux Reporter: Tim Clarke <tclarke>
Component: tcshAssignee: Miloslav Trmač <mitr>
Status: CLOSED RAWHIDE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 9CC: hfuchi, tao
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: 6.14-1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2005-03-25 23:48:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tim Clarke 2003-08-26 13:43:50 UTC
Description of problem:

some tcsh scripts fail under RedHat 9. They worked under RedHat 7 and 6 and
works under tcsh/csh on Solaris/AIX systems 


Version-Release number of selected component (if applicable):

tcsh-6.12-4

How reproducible:

try the following (simplified) code fragment from tcsh on RedHat 9.0:-

set ddd=`date | \\
 awk '{print $1}'`

This is entered as two lines (i.e.; a newline follows the "\\")

The command will give the following error:-

 : Command not found.

Interestingly, if I remove one of the \ characters, the variable will be
correctly set with the desired value, but this will not work on other platforms
or RedHat 7. I will simply get the "Invalid null command." error.




Steps to Reproduce:
1.
2.
3.
    
Actual results:

 : Command not found.


Expected results:

ddd variable should be set


Additional info:

Comment 1 Miloslav Trmač 2004-08-20 08:40:47 UTC
tcsh changed behavior in 6.12 and fixing this is another incompatible
change...

Sent upstream, changing this at the distribution level would only
make the problem worse.

http://mx.gw.com/mailman/private/tcsh-bugs/2004-August/000274.html

Comment 3 Miloslav Trmač 2004-09-10 14:36:51 UTC
The tcsh-bugs post above is not available for list non-members,
so I am pasting it here for reference:
--------------------------------------------------------------
Hello,
I'd like to consult
http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=103072

Consider:
echo `a | \\
b`

The manual page is quite clear that the word with backquotes is parsed as
'a', ' ', '|', ' ', '\\', '\n', 'b'.
This forms a valid command string equivalent with "a | b", because
the backslash-newline is unquoted and thus disappears. This apparently
was the historical behavior.

What happens with tcsh-6.12 and 6.13 is something different:
backeval() converts all '\r' and '\n' characters to spaces, so we have
"a | \ b".

Looking at the changelog, this change seems to have appeared in
tcsh-6.11.01:
  8. `` commands with embedded newlines would ignore commands following
       the new line (Victor I. Pasko)

The manual page does not mention the issue of newlines and command
substitution at all, which makes me think that newlines in command
substitution should not be rewritten.

What will we do with this issue? We have already broken compatibility
with old tcsh, fixing this will break compatibility again.

There are three possibilities:
* Revert the change. Break scripts relying on the conversion.
* Do nothing. Scripts that broke with 6.12 will remain broken.
* Rewrite newlines to spaces only when unquoted. This "might" magically
  do the right thing for most scripts, but creates a third behavior,
  incompatible with either of the past variants.

  	Mirek

Comment 7 Tim Clarke 2004-10-11 17:35:41 UTC
Hmmmm, 

I used csh extensively on BSD (ConvexOS) from 1991-1995 and this
supported the \\ continuation.

I used csh, sh and ksh on AIX from 1992-2004. \\ works in csh while \
fails. \\ and \ both work in sh and ksh.

I used csh, sh and ksh on Solaris from 1997-2004. \\ works in csh
while \ fails. \\ and \ both work in sh and ksh.

Under tcsh-6.10-6 on Linux (RedHat7.2) (a csh derivative), \\ works
while \ fails. \\ and \ both work in bash and ksh

Under tcsh-6.12-4 on Linux (RedHat9) \\ fails while \ works. \\ and \
both work in bash and ksh.



So, \\ and \ both seem to work in all implementations of sh, ksh and
bash that I have tried.

\\ seems to work in all csh implementations that I have tried, while \
fails in all of them.

Under tcsh-6.10-6 \\ works while \ fails
Under tcsh-6.12-4 \ works while \\ fails

Conclusion - tcsh-6.12-4 reverses behaviour exhibited by all versions
of csh (an ancestral relative) and earlier versions of itself. This
change in behaviour has not even produced compatibility with other
shells found on Linux systems.

I would prefer to see the old behaviour back as I currently work in a
multi-OS environment and it is important that scripts work seamlessly
on different architectures. Other people may not fel the same way
about this.

Comment 8 Miloslav Trmač 2004-10-11 17:45:45 UTC
sh and ksh behavior is not relevant much, the underlying lexical
structure of csh is too different.

I completely agree with the "conclusion", but reverting the change
now would just add another instance of the same breakage.

I am working with upstream developers to hopefully find a solution.

(Historically, all the csh versions that support \\ have reportedly had
a bug in it, causing executing uninitialized data. So simple reversal
of the behavior would also reintroduce this bug in tcsh.)

Comment 9 Miloslav Trmač 2005-03-25 23:48:14 UTC
Handling newlines in command substution is controlled by the "csubstnonl"
variable in tcsh-6.14-1, which should appear in rawhide soon.

Comment 11 Tim Clarke 2005-03-29 10:33:57 UTC
I will test this when I see tcsh-6.14-1