Bug 138681 - [RHEL4] failsafe xterm has non-utf locale while environment is set to utf
Summary: [RHEL4] failsafe xterm has non-utf locale while environment is set to utf
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: gdm
Version: 4.0
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
: ---
Assignee: Ray Strode [halfline]
QA Contact: Mike McLean
URL:
Whiteboard:
Depends On:
Blocks: 156322
TreeView+ depends on / blocked
 
Reported: 2004-11-10 17:51 UTC by Dave Maley
Modified: 2007-11-30 22:07 UTC (History)
6 users (show)

Fixed In Version: RHBA-2005-644
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2005-10-05 15:32:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2005:644 0 qe-ready SHIPPED_LIVE gdm bug fix update 2005-10-05 04:00:00 UTC

Description Dave Maley 2004-11-10 17:51:24 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; rv:1.7.3) Gecko/20041020
Firefox/0.10.1

Description of problem:
If you choose "failsafe" from the sessions menu at the login screen
and login.  Then from the failsafe xterm run netconfig it looks all
messed up.  The encoding appears to be off.

Version-Release number of selected component (if applicable):
netconfig-0.8.21-1

How reproducible:
Always

Steps to Reproduce:
1. login to failsafe session
2. run netconfig
3.
    

Actual Results:  netconfig display has incorrect characters

Expected Results:  normal netconfig display

Additional info:

Comment 3 Petr Rockai 2005-04-07 15:22:28 UTC
Tried on RHEL4 and RHEL3 with ssh -Y, running failsafe session by hand and
couldn't reproduce. I will try on a local RHEL4 machine with gdm tomorrow, but i
don't see how could that break... Well, i'll report tomorrow.

Comment 4 Petr Rockai 2005-04-11 12:19:38 UTC
Ok, this problem is not a netcfg one but a configuration one. The failsafe
session is set up to run a regular (non-utf8) xterm and sets locale to
en_US.UTF-8 nevertheless which results in corrupted display. Reassigning to
xinitrc which is apparently the package responsible for setting up environment.
I suggest the Xsession script either sets up locale before running xterm or runs
uxterm instead (either will fix the problem).

Comment 6 Mike A. Harris 2005-04-12 23:18:56 UTC
The xterm changelog seems to suggest that xterm should already be able
to autodetect being ran in a UTF-8 environment and UTF-8 enable itself:


                                                                   Patch #126 -
2000/2/8 - XFree86 3.9.17c

     * patch  by  Markus  Kuhn: ensure that xterm will automatically activate
the UTF-8 mode whenever the name of the locale environment variable suggests
that a UTF-8
       locale  is  in  use.  This  will help that by simply setting LC_CTYPE an
entire system can be switched over to UTF-8, without users having to remember
the UTF-8
       command line options ("-u8", etc.) of the various applications. Command
line options and X resource entries can still be used to override this default
choice.



For some reason, it appears this doesn't appear to work.  Investigating...

Comment 7 Petr Rockai 2005-04-13 08:22:37 UTC
It does work (at least when running xterm manually from utf-8 locale). I believe
the locale is not set at the time xterm is exec'd or that the locale is not
exported to it. The shell started by xterm then loads up config and sets it's
own locale to UTF-8 and kablaam, it messes up.

Comment 8 Thomas E. Dickey 2005-04-13 09:58:56 UTC
Actually it's more complicated than that (there's the locale
resource for instance as a further refinement).  But as I
understand these comments, the preferred solution (rather than
the -u8 option, etc), would be for the app-defaults file to
be patched to reflect the fact that the whole system is running
in UTF-8 mode.

Comment 9 Mike A. Harris 2005-05-02 17:48:03 UTC
The problem with that, is that it hard codes either UTF-8 or non-UTF-8
as a default.  If we hard code non-UTF-8 by default (current default)
then people with UTF-8 setups will have a problem.  Since UTF-8 is our
standard system default, it makes more sense to set UTF-8 as the default
for our xterm too, and we can do that - however, if we hard code UTF-8
as the default for xterm, then people who do not use UTF-8 by overriding
the system default for LANG, will end up with a problem too.

Calling setlocale() and nl_langinfo() from inside xterm can tell it if
the current encoding is UTF-8 or not, and to automatically default to
using UTF-8.  I've examined xterm source briefly however and due to the
way the commandline args are parsed, I do not see an easy way to integrate
calls to setlocale() and nl_langinfo() to do this cleanly.

For a short term solution, we will probably change xinitrc to use
'uxterm' instead, or to patch the resources file.  That will make xterm
default to our system encoding default, however users who do not use
our system default, and reconfigure their systems to 8859-1 or something
else, will have a broken xterm still.  So this solution only substitutes
one problem that affects a number of people with an opposite problem that
will likely affect a slightly lower number of people if my assumption
is correct that the number of people who disable UTF-8 is small, and
intersection of that group of people with those who use xterm is smaller
than those who now use xterm in our UTF-8 default).



Comment 10 Thomas E. Dickey 2005-05-02 19:17:38 UTC
Supposedly if the locale resource is set, xterm will
do the right thing (perhaps there's some aspect of
that which I'm overlooking).  Recent changes (patch
#201 and upcoming #202) make it easier to switch
xterm into UTF-8 mode when the initial locale is not
UTF-8.

Comment 16 Mike A. Harris 2005-05-03 10:46:43 UTC
xterm-192-1.EL4.0 is now available for download via ftp, at the
following URL:  ftp://people.redhat.com/mharris/testing/4E

Please test this package as soon as possible, and update this report
with the status of testing results.

Comment 18 Mike A. Harris 2005-05-24 15:43:43 UTC
Please test the xinitrc package here, reboot the system, and test
wether the problem still occurs upon system startup with the
failsafe mode:

ftp://people.redhat.com/mharris/testing/4E/xinitrc/4.0.14.1-1/noarch

Please also paste back the output of the 'locale' command from within
the running xterm.

Setting status to "NEEDINFO", awaiting test results.

Comment 19 Mike A. Harris 2005-05-24 15:48:09 UTC
Oh, also, please revert the previous xterm resource change for this test,
or try both with and without it.  It's the 'utf8' resource we enabled.

TIA

Comment 21 Mike A. Harris 2005-05-26 02:06:11 UTC
Ok, I modified xinitrc to source lang.sh to ensure the environment is properly
set up, and it has been tested.  Aparently, even when LANG=en_US.UTF-8
is set prior to xterm being invoked, xterm still does not use UTF-8 mode,
even if the utf8 resource is enabled.  This bug is quite puzzling.

Comment 22 Thomas E. Dickey 2005-05-26 09:11:23 UTC
I'm missing some detail: how are you determining that xterm is
not using UTF-8 mode?  Perhaps there is some confusion regarding
the font.  That is the most visible detail (to see what xterm
actually decided, I'd have to turn on the debug trace).  The
usual problem with fonts is if someone's got a too-wide resource
pattern for the ISO-8859-1 fonts which overrides the ISO-10646-1
fonts which are specified in the *utf8Fonts subresources.

Comment 23 Mike A. Harris 2005-05-31 23:48:07 UTC
Reassigning bug back to xterm component from xinitrc, as the environment
is set up properly and confirmed, and the problem aparently still exists
even when the xterm utf8 resource is forced in the resources file.

Please indicate what font is being used in xterm when this problem is
reproduced.

Comment 24 Mike A. Harris 2005-05-31 23:58:12 UTC
Ok, I just tried the following:

1) From konsole, with LANG=en_US, run "xterm &"
2) Inside xterm, echo $LANG results in:  "en_US"
3) Run "netconfig", it works properly, with line drawing chars drawn correctly.
4) Run "LANG=en_US.UTF-8 netconfig", it draws european accented letters instead
   of line drawing chars.

Test 2:

1) From konsole:  export LANG=en_US.UTF-8
2) Run "xterm &"
3) Inside xterm, echo $LANG results in:  "en_US" (not en_US.UTF-8)
4) Run "netconfig", line drawing chars work properly.
5) Run "LANG=en_US.UTF-8 netconfig", line drawing chars work properly.

Test 3:

Same as test 2, except adding a step 3.5:
3.5) "export LANG=en_US.UTF-8"

This results in same result as test 2.



Comment 25 Thomas E. Dickey 2005-06-02 00:25:03 UTC
Test 1, step 4 - if xterm's not running in UTF-8 mode, an 
application that sends UTF-8 codes for line-drawing won't work.
(Everything under Test 1 looks correct - were you expecting some
other result?)

Test 2, step 3 has a problem, since xterm doesn't set $LANG.
What is setting it?

Test 2, steps 4 and 5 could amount to the same thing (xterm normally
recognizes the vt100-strings for line-drawing).  But I don't know
the dependencies for netconfig (ncurses/slang).

Test 3, setting $LANG after xterm's started won't change the coding
it's using (it couldn't).

I don't see a response to my comment about xterm's "locale" 
resource.

Comment 26 Mike A. Harris 2005-06-14 17:06:07 UTC
I've narrowed the problem down a slight bit.  The xinitrc startup scripts
are definitely not causing the problem, as xterm never gets invoked from
them when this problem occurs.  It appears to be directly invoked from
gdm hardcoded, and ignoring the environment and resource settings aparently.

Before I respond to the above stuff, I think it's worth doing a bit more
diagnosis so we're not chasing our tails.  ;o)

Comment 27 Mike A. Harris 2005-06-14 17:39:22 UTC
Ok, I've diagnosed this further with our gdm maintainer Ray Strode, and
after a fair bit more testing, we both believe this is a bug in the way
gdm sets up the environment in which xterm is ran, or a bug in the way
it invokes xterm.

Ray tracked down where it is invoked in gdm, in "slave.c".

I tried the following new tests:

- Fresh install of RHEL4U1 RC
- Defaults for everything, including English en_US.UTF-8 for i18n

Test1:
- Log in as root using failsafe from gdm
- LANG=en_US.UTF-8 in the shell inside xterm
- Initially reported problem reproduced when invoking netconfig.
- Invoking a second xterm without arguments, netconfig works properly.
Conclusion:
- The 2nd xterm does the right thing when invoked from en_US.UTF-8
  environment.
- This implies the first xterm is NOT being invoked with LANG=en_US.UTF-8

Test2:
- Changed xinitrc and Xclients xterm invocations to invoke uxterm
- Witnessed identical results to Test1
- Added some diagnostics to the xinitrc scripts
- Witnessed that xterm wasn't being invoked from that location
Conclusion:
- Problem is not due to the xinitrc scripts and can not be fixed from
  there.

Test3:
- Set the "*VT100*utf8: true" resource in .Xresources
- Repeated Test1 above with identical results.
- Changed the resource in the Xterm.ad
- Repeated Test1 with identical results
- Removed .Xresources and restored Xterm.ad to defaults
Conclusion:
- resources being completely ignored, assumption is that they're not
  being read, or that LANG is unset or otherwise non-UTF-8 when xterm
  gets invoked by gdm

Test4:
- Renamed xterm binary to xterm.orig, and created a wrapper "xterm" script
  which adds the "-u8" option automatically.
- Repeated test1 tests, and it was clear that xterm was still not operating
  with UTF-8.
Conclusion
- Same as for Test3

Test5:
- Modified the script from Test 4, by changing it to invoke:
        export LANG=en_US.UTF-8
        exec xterm -u8 $*
- Ran a failsafe session
- running netconfig now works properly as desired.


My final conclusion seems to be that gdm is not properly setting up
the i18n environment for xterm to run in.  Ray agrees this appears to
be a gdm issue, so reassigning to gdm component.


P.S.  Thanks for all of the troubleshooting suggestions Thomas, it's
      been quite helpful in diagnosing this.



Comment 28 Mike A. Harris 2005-06-14 17:47:21 UTC
One additional note:  I did one more test, by removing the -u8 option from
the xterm wrapper script and testing again.  The -u8 option is clearly not
required, as xterm seems to do the right thing.

In other words, as long as LANG=en_US.UTF-8, regardless of wether -u8 is
specified or not, xterm works properly and netconfig starts up with proper
line drawing characters.  If LANG is not set to UTF-8 when xterm is invoked,
from the wrapper script, then the problem occurs.

To confirm this, I added a line to log the output of the "locale" command
to the xterm wrapper script prior to manually setting LANG.  The results
are as follows:

# cat xterm-locale
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=


Hope this helps in further diagnosis of the issue.

Comment 35 Red Hat Bugzilla 2005-10-05 15:32:15 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2005-644.html



Note You need to log in before you can comment on or make changes to this bug.