Description of problem: ksh removes from the environment variables that contain certain characters (like hyphen or space). Version-Release number of selected component (if applicable): ksh-20050202-0.5E.4 How reproducible: always Steps to Reproduce: 1. start tcsh 2. in the tcsh session set a variable containing hyphen: # setenv a-b foo 3. start ksh and type printenv 4. start tcsh from the ksh session and type printenv Actual results: the variable a-b is not listed in any case Expected results: the printenv output should contain a-b=foo Additional info: Excerpt from POSIX (http://www.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap08.html): Environment variable names used by the utilities in the Shell and Utilities volume of IEEE Std 1003.1-2001 consist solely of uppercase letters, digits, and the '_' (underscore) from the characters defined in Portable Character Set and do not begin with a digit. Other characters may be permitted by an implementation; applications shall tolerate the presence of such names. Uppercase and lowercase letters shall retain their unique identities and shall not be folded together. The name space of environment variable names containing lowercase letters is reserved for applications. Applications can define any environment variables with names from this name space without modifying the behavior of the standard utilities. Therefore ksh should not alter the environment even though it may not necessarily allow to access/set such variables. Note that this bug is present even in the most recent version of ksh, therefore also in RHEL-5 and Fedora.
Created attachment 317983 [details] Patch for ksh-20080202 Here's a patch for newer version of ksh. It allows to have all the ascii characters in the environment variable name and seems to be working well. I still have problems to create a patch for the older ksh-20050202 (always seem to break something else) but I'll keep trying.
I've made patch for RHEL 4 version. Patch fixes problem mentioned in description: Ksh does not remove lines from the environment it does not understand any more. These variables (lines) are kept in separated list and they are added to the environment for executed programs. Ksh itself does not see them because it does not know how to use them. Conclusion: if you: run tcsh, add variables to the environment, run ksh and again tcsh, environment won't be changed. But... you can't use these variables via ksh's expressions. After some time digging through ksh's sources, I can say that adding complete support for extra characters into ksh would be very difficult. It would be probably causing a lot of regressions. For example 'ab.cd' is not (can't be) the same as 'xyz' for ksh, because ksh already uses dot for something like struct in C and it's not just dot, what can cause problems here.
Created attachment 319366 [details] keeps not used lines from the environment for the others
Created attachment 319988 [details] Backport from ksh93t beta for RHEL-4 ksh
This request was evaluated by Red Hat Product Management for inclusion, but this component is not scheduled to be updated in the current Red Hat Enterprise Linux release. If you would like this request to be reviewed for the next minor release, ask your support representative to set the next rhel-x.y flag to "?".
Who made the decision to go ahead with adding support for hyphens? There are very good reasons why hyphens are not included in the required set of characters in envvar names. Consider the following valid Bourne shell code: $ a=5 $ b=1 $ a-b=3 $ printf "%d\n" ${a-b} should print "5\n" according to the standard $ printf "%d\n" $(( a-b )) should print "4\n" according to the standard $ unset a $ printf "%s\n" ${a-b} should print "b\n" according to the standard What will a ksh with support with hyphens in the name produce with this code? Even if somehow this semantics is maintained, what about $ a=5 $ a-a=3 ${a-a-a} $(a-a-a) ?? Yes, the standard allows _existing_ values in the environment to contain hyphens. But this is a simple extension: ${a-b} will always be ${a - b} because the parser splits the text before the envvar is looked up $(a-a-a) will always be $(a - a - a) for the same reason BTW: I was pointed to this issue by one of my contacts at Sun. They are apparently pressured to apply the same bad change.
(In reply to comment #8) > Who made the decision to go ahead with adding support for hyphens? There are > very good reasons why hyphens are not included in the required set of > characters in envvar names. In case the standard allow such a variable to exist in the environment and says ksh should "tolerate" it, what does that exactly mean? Is it OK if ksh *removes* the variable? The change that was made to ksh allows such a variable to exist in the environment (and passing it on), but ingnoring it otherwise. We have discussed the problem with ksh upstream (AT&T) of course: https://mailman.research.att.com/pipermail/ast-users/2008q3/002249.html https://mailman.research.att.com/pipermail/ast-developers/2008q4/000349.html It was the ksh upstream who agreed with the change otherwise we wouldn't let it in. Let's go through your examples using ksh93t that contains the patch from comment #6 (ksh was started with 'env a-b=3 /bin/ksh'): $ a=5 $ b=1 $ a-b=3 ksh: a-b=3: not found [No such file or directory] $ printf "%d\n" ${a-b} 5 $ printf "%d\n" $((a-b)) 4 $ unset a $ printf "%s\n" ${a-b} b The rest is clear -- 'a-a=3' is an invalid expression for ksh... The difference from the previous versions is: $ printenv _=/usr/bin/printenv a-b=3 ... Please let me know if you still see some problems with the proposed change. It really doesn't introduce any changes in the ksh93 language syntax (or semantics).
(In reply to comment #9) > $ a=5 > $ b=1 > $ a-b=3 > ksh: a-b=3: not found [No such file or directory] > [...] This means ksh will not ever try to look up such environment variable when the syntax _might_ seem to request it? I.e., the parser hasn't been changed? > $ printf "%d\n" ${a-b} > 5 > $ printf "%d\n" $((a-b)) > 4 > $ unset a > $ printf "%s\n" ${a-b} > b Looks OK. > The rest is clear -- 'a-a=3' is an invalid expression for ksh... Shouldn't be different from a-b=3. I.e., should be interpreted as a program name. > The > difference from the previous versions is: > > $ printenv > _=/usr/bin/printenv > a-b=3 > ... > > Please let me know if you still see some problems with the proposed change. If it's really only the case that ksh doesn't crap out when it sees such environment variables and passes them on to child processes, that's fine. This is what "tolerate" is supposed to mean.
(In reply to comment #10) > > $ a-b=3 > > ksh: a-b=3: not found [No such file or directory] > > [...] > > This means ksh will not ever try to look up such environment variable when the > syntax _might_ seem to request it? I.e., the parser hasn't been changed? Yes. The patch reorders the environment array upon ksh startup so that the variables with invalid names (invalid from the ksh93 point of view) are put at the beginning and ksh uses a pointer that points beyond these variables as its environment, so ksh itself doesn't see the variables at all. The pointer to the beginning of the environmet string array is then passed on to the child processes so they have chance to see all the variables. > > The rest is clear -- 'a-a=3' is an invalid expression for ksh... > > Shouldn't be different from a-b=3. I.e., should be interpreted as a program > name. My fault, I meant it behaved exactly as the above, i.e. not a valid assignment expression, and will try to launch a program (so it ends up with "No such file or directory" in my case). > If it's really only the case that ksh doesn't crap out when it sees such > environment variables and passes them on to child processes, that's fine. This is all the patch does.
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Previously, if the variable name contained certain characters such as a hyphen or a space, ksh removed the variable from the environment. This error has been fixed, and although ksh still does not use environment variables with names that contain these characters, it keeps them available for sub-processes.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2011-0226.html