This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 464156 - ksh does not declare shell variables with hyphen
ksh does not declare shell variables with hyphen
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: ksh (Show other bugs)
4.6
All Linux
urgent Severity high
: rc
: ---
Assigned To: Michal Hlavinka
BaseOS QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-09-26 10:40 EDT by Tomas Smetana
Modified: 2011-02-16 09:08 EST (History)
4 users (show)

See Also:
Fixed In Version: ksh-20100621-1.el4
Doc Type: Bug Fix
Doc Text:
Previously, if the variable name contained certain characters such as a hyphen or a space, ksh removed the variable from the environment. This error has been fixed, and although ksh still does not use environment variables with names that contain these characters, it keeps them available for sub-processes.
Story Points: ---
Clone Of:
: 488934 (view as bug list)
Environment:
Last Closed: 2011-02-16 09:08:02 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch for ksh-20080202 (531 bytes, patch)
2008-09-29 10:16 EDT, Tomas Smetana
no flags Details | Diff
keeps not used lines from the environment for the others (3.62 KB, patch)
2008-10-03 10:04 EDT, Michal Hlavinka
no flags Details | Diff
Backport from ksh93t beta for RHEL-4 ksh (1.66 KB, patch)
2008-10-10 04:05 EDT, Tomas Smetana
no flags Details | Diff

  None (edit)
Description Tomas Smetana 2008-09-26 10:40:25 EDT
Description of problem:
ksh removes from the environment variables that contain certain characters (like hyphen or space).

Version-Release number of selected component (if applicable):
ksh-20050202-0.5E.4

How reproducible:
always

Steps to Reproduce:
1. start tcsh
2. in the tcsh session set a variable containing hyphen:

# setenv a-b foo

3. start ksh and type printenv
4. start tcsh from the ksh session and type printenv
  
Actual results:
the variable a-b is not listed in any case

Expected results:
the printenv output should contain
a-b=foo

Additional info:

Excerpt from POSIX (http://www.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html):

Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names. Uppercase and lowercase letters shall retain their unique identities and shall not be folded together. The name space of environment variable names containing lowercase letters is reserved for applications. Applications can define any environment variables with names from this name space without modifying the behavior of the standard utilities.

Therefore ksh should not alter the environment even though it may not necessarily allow to access/set such variables.

Note that this bug is present even in the most recent version of ksh, therefore also in RHEL-5 and Fedora.
Comment 1 Tomas Smetana 2008-09-29 10:16:09 EDT
Created attachment 317983 [details]
Patch for ksh-20080202

Here's a patch for newer version of ksh.  It allows to have all the ascii characters in the environment variable name and seems to be working well.  I still have problems to create a patch for the older ksh-20050202 (always seem to break something else) but I'll keep trying.
Comment 3 Michal Hlavinka 2008-10-03 10:02:08 EDT
I've made patch for RHEL 4 version.

Patch fixes problem mentioned in description:

Ksh does not remove lines from the environment it does not understand any more.

These variables (lines) are kept in separated list and they are added to the environment for executed programs. Ksh itself does not see them because it does not know how to use them.

Conclusion:
if you: run tcsh, add variables to the environment, run ksh and again tcsh, environment won't be changed.
But... you can't use these variables via ksh's expressions.

After some time digging through ksh's sources, I can say that adding complete support for extra characters into ksh would be very difficult. It would be probably causing a lot of regressions. For example 'ab.cd' is not (can't be) the same as 'xyz' for ksh, because ksh already uses dot for something like struct in C and it's not just dot, what can cause problems here.
Comment 4 Michal Hlavinka 2008-10-03 10:04:23 EDT
Created attachment 319366 [details]
keeps not used lines from the environment for the others
Comment 6 Tomas Smetana 2008-10-10 04:05:56 EDT
Created attachment 319988 [details]
Backport from ksh93t beta for RHEL-4 ksh
Comment 7 RHEL Product and Program Management 2008-10-31 12:52:03 EDT
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 8 Ulrich Drepper 2008-11-13 21:27:20 EST
Who made the decision to go ahead with adding support for hyphens?  There are very good reasons why hyphens are not included in the required set of characters in envvar names.  Consider the following valid Bourne shell code:

$ a=5
$ b=1
$ a-b=3
$ printf "%d\n" ${a-b}
	should print "5\n" according to the standard

$ printf "%d\n" $(( a-b ))
	should print "4\n" according to the standard

$ unset a
$ printf "%s\n" ${a-b}
	should print "b\n" according to the standard


What will a ksh with support with hyphens in the name produce with this code?  Even if somehow this semantics is maintained, what about

   $ a=5
   $ a-a=3
   ${a-a-a}
   $(a-a-a)

??

Yes, the standard allows _existing_ values in the environment to contain hyphens.  But this is a simple extension:

    ${a-b}   will always be ${a - b} because the parser splits the text before the envvar is looked up

    $(a-a-a)  will always be $(a - a - a) for the same reason


BTW: I was pointed to this issue by one of my contacts at Sun.  They are apparently pressured to apply the same bad change.
Comment 9 Tomas Smetana 2008-11-14 03:34:00 EST
(In reply to comment #8)
> Who made the decision to go ahead with adding support for hyphens?  There are
> very good reasons why hyphens are not included in the required set of
> characters in envvar names.

In case the standard allow such a variable to exist in the environment and says ksh should "tolerate" it, what does that exactly mean?  Is it OK if ksh *removes* the variable?  The change that was made to ksh allows such a variable to exist in the environment (and passing it on), but ingnoring it otherwise.

We have discussed the problem with ksh upstream (AT&T) of course:
https://mailman.research.att.com/pipermail/ast-users/2008q3/002249.html
https://mailman.research.att.com/pipermail/ast-developers/2008q4/000349.html

It was the ksh upstream who agreed with the change otherwise we wouldn't let it in.

Let's go through your examples using ksh93t that contains the patch from comment #6 (ksh was started with 'env a-b=3 /bin/ksh'):

$ a=5
$ b=1
$ a-b=3
ksh: a-b=3: not found [No such file or directory]
$ printf "%d\n" ${a-b}
5
$ printf "%d\n" $((a-b))
4
$ unset a
$ printf "%s\n" ${a-b}  
b

The rest is clear -- 'a-a=3' is an invalid expression for ksh...  The difference from the previous versions is:

$ printenv
_=/usr/bin/printenv
a-b=3
...

Please let me know if you still see some problems with the proposed change.  It really doesn't introduce any changes in the ksh93 language syntax (or semantics).
Comment 10 Ulrich Drepper 2008-11-14 03:54:29 EST
(In reply to comment #9)
> $ a=5
> $ b=1
> $ a-b=3
> ksh: a-b=3: not found [No such file or directory]
> [...]

This means ksh will not ever try to look up such environment variable when the syntax _might_ seem to request it?  I.e., the parser hasn't been changed?

> $ printf "%d\n" ${a-b}
> 5
> $ printf "%d\n" $((a-b))
> 4
> $ unset a
> $ printf "%s\n" ${a-b}  
> b

Looks OK.


> The rest is clear -- 'a-a=3' is an invalid expression for ksh...

Shouldn't be different from a-b=3.  I.e., should be interpreted as a program name.


> The
> difference from the previous versions is:
> 
> $ printenv
> _=/usr/bin/printenv
> a-b=3
> ...
> 
> Please let me know if you still see some problems with the proposed change.

If it's really only the case that ksh doesn't crap out when it sees such environment variables and passes them on to child processes, that's fine.  This is what "tolerate" is supposed to mean.
Comment 11 Tomas Smetana 2008-11-14 04:33:19 EST
(In reply to comment #10)
> > $ a-b=3
> > ksh: a-b=3: not found [No such file or directory]
> > [...]
> 
> This means ksh will not ever try to look up such environment variable when the
> syntax _might_ seem to request it?  I.e., the parser hasn't been changed?

Yes. The patch reorders the environment array upon ksh startup so that the variables with invalid names (invalid from the ksh93 point of view) are put at the beginning and ksh uses a pointer that points beyond these variables as its environment, so ksh itself doesn't see the variables at all.  The pointer to the beginning of the environmet string array is then passed on to the child processes so they have chance to see all the variables.

> > The rest is clear -- 'a-a=3' is an invalid expression for ksh...
> 
> Shouldn't be different from a-b=3.  I.e., should be interpreted as a program
> name.

My fault, I meant it behaved exactly as the above, i.e. not a valid assignment expression, and will try to launch a program (so it ends up with "No such file or directory" in my case).

> If it's really only the case that ksh doesn't crap out when it sees such
> environment variables and passes them on to child processes, that's fine.

This is all the patch does.
Comment 18 Jaromir Hradilek 2010-09-08 07:51:23 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, if the variable name contained certain characters such as a hyphen or a space, ksh removed the variable from the environment. This error has been fixed, and although ksh still does not use environment variables with names that contain these characters, it keeps them available for sub-processes.
Comment 20 errata-xmlrpc 2011-02-16 09:08:02 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0226.html

Note You need to log in before you can comment on or make changes to this bug.