Bug 1166428

Summary: gnome-terminal: please advertise proper 256-color $TERM description
Product: [Fedora] Fedora Reporter: Zbigniew Jędrzejewski-Szmek <zbyszek>
Component: vte291Assignee: Kalev Lember <kalevlember>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 21CC: debarshir, dennis.schridde, egmont, extras-qa, jistone, jonathan, kalevlember, lnykryn, mclasen, vpavlin, zbyszek, zeenix
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: vte291-0.38.2-3.fc21 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1165439 Environment:
Last Closed: 2014-12-12 04:34:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1165439    
Attachments:
Description Flags
Flip the default to TERM=xterm-256color none

Description Zbigniew Jędrzejewski-Szmek 2014-11-21 00:46:11 UTC
tl;dr gnome-terminal says that it supports 8 colors, but it supports 256 just fine. Let applications know that. gnome-terminal should set TERM=rxvt-256color (or some other appropriate value).

+++ This bug was initially created as a clone of Bug #1165439 +++

Description of problem:
I found that gnome-terminal no longer sets COLORTERM, and thus 256term.sh doesn't patch up TERM=xterm or TERM=screen to their 256color variants.

Version-Release number of selected component (if applicable):
initscripts-9.56.1-4.fc21.x86_64
gnome-terminal-3.14.2-1.fc21.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Open gnome-terminal
2. echo $TERM
3. Run screen/tmux
4. echo $TERM

Actual results:
First "xterm", then "screen".

Expected results:
On F20, it would get "xterm-256color" and "screen-256color".
COLORTERM was still being set by gnome-terminal at that time.

Additional info:

commit 1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf
Author: Christian Persch <chpe>
Date:   Sat Apr 26 19:13:21 2014 +0200

    screen: Stop setting COLORTERM env var
    
    COLORTERM is a long-obsolete slang-only variable used to work around
    broken termcap/terminfo entries.

https://git.gnome.org/browse/gnome-terminal/commit/?id=1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf

--- Additional comment from Josh Stone on 2014-11-18 20:05:50 EST ---

As a workaround, I've added this to my .bashrc, before global files are sourced:

if test -z "$COLORTERM" &&
    readlink /proc/$PPID/exe | grep -q gnome-terminal
then
    export COLORTERM=gnome-terminal
fi

Obviously, that can only check the immediate parent process, but so far it's ok.

--- Additional comment from Zbigniew Jędrzejewski-Szmek on 2014-11-20 19:42:33 EST ---

Whoah, what a mess ;)

So, gnome-terminal sets a $TERM entry which does not advertise full capabalities. Instead of changing that, we try to fix it up in the shell using a hacky scripts. This does not make any sense to me.

(Even if there were buggy programs which refused to work with TERM=xterm-256color, which I guess was the primal reason why it was not set properly when xterm or gnome-terminal gained 256 color support, they most likely have been fixed, since 256term.sh has been setting TERM=xterm-256color for many years now. So even if there was a reason no to do it, there isn't one now.)

Let's fix this properly. After-the-fact guessing whether a terminal supports 256 colors is not possible to do cleanly [1]. The only sane way is to set it properly from the terminal emulator program.

So I propose a two step procedure:

1. Make gnome-terminal, xterm, screen, tmux, and whatever else set a $TERM entry which advertises 256 colors. I'll file bugs for gnome-terminal and screen.

2. Wait a bit for 1. to happen, then drop 256term.* from initscripts.

The rationale is that when 1. is done, the 256term.* scripts will be noops, so doing things in this order should provide a smooth transition.

[1] http://unix.stackexchange.com/questions/23763/checking-how-many-colors-my-terminal-emulator-supports/23789#23789

Comment 1 Egmont Koblinger 2014-11-24 10:20:15 UTC
As far as I remember, the reason is that some people ssh into systems that don't have "xterm-256color" terminfo entry. "xterm" is more compatible with the crazy/ancient systems out there. (NB. I'd totally stay away from setting TERM to anything rxvt-related.)

If so far you relied on $COLORTERM for a workaround, you can now instead rely on $VTE_VERSION for the same workaround :)

I tend to agree that xterm-256color would be a better default, and if some system doesn't support it then the workaround should happen there.

On a slightly related note: gnome-terminal supports 16M true colors, but no terminfo entry exists for this feature.

Comment 2 Zbigniew Jędrzejewski-Szmek 2014-11-24 15:34:33 UTC
(In reply to Egmont Koblinger from comment #1)
> As far as I remember, the reason is that some people ssh into systems that
> don't have "xterm-256color" terminfo entry. "xterm" is more compatible with
> the crazy/ancient systems out there. (NB. I'd totally stay away from setting
> TERM to anything rxvt-related.)
But we're not using "xterm" anywhere in reality, since basically everyone runs 256colors.sh in their shell and changes "xterm" to "xterm-256color". The initial setting from gnome-terminal would only matter if someone run ssh directly in the terminal emulator and not through a shell.

> If so far you relied on $COLORTERM for a workaround, you can now instead
> rely on $VTE_VERSION for the same workaround :)
That's not satisfying either ;)

> I tend to agree that xterm-256color would be a better default, and if some
> system doesn't support it then the workaround should happen there.
Yes. Can we do the switch please?

> On a slightly related note: gnome-terminal supports 16M true colors, but no
> terminfo entry exists for this feature.
Then we should get the terminfo entry, make it widely available in Fedora and on Debian and advertise it to other systems, and then at some point we should be able to switch gnome-terminal to it fairly painlessly. But that's another story.

Comment 3 Egmont Koblinger 2014-11-24 17:13:19 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #2)

> But we're not using "xterm" anywhere in reality, since basically everyone
> runs 256colors.sh in their shell and changes "xterm" to "xterm-256color".

This is RedHat/Fedora-specific. Mainstream gnome-terminal should also consider users of other distros, the pros and cons they'd face.

> The initial setting from gnome-terminal would only matter if someone run ssh
> directly in the terminal emulator and not through a shell.

That's a good point, and this is where it becomes more important to advertise 256-color support.

> > If so far you relied on $COLORTERM for a workaround, you can now instead
> > rely on $VTE_VERSION for the same workaround :)
> That's not satisfying either ;)

Just for curiosity: how is it any worse than looking at XTERM_VERSION, ROXTERM_ID or KONSOLE_DBUS_SESSION, as already done in 256term.sh? ;)

> > I tend to agree that xterm-256color would be a better default, and if some
> > system doesn't support it then the workaround should happen there.
> Yes. Can we do the switch please?

Filed https://bugzilla.gnome.org/show_bug.cgi?id=740641

> Then we should get the terminfo entry, make it widely available in Fedora
> and on Debian and advertise it to other systems, and then at some point we
> should be able to switch gnome-terminal to it fairly painlessly. But that's
> another story.

This is a long complicated story. I probably wasn't clear at all: there's absolutely no terminfo capability anywhere that would define true colors, first we'd need to agree on one with terminfo's maintainer.

Another thing is that gnome-terminal (vte) uses xterm as the reference most of the time, and uses TERM=xterm*. There were times when it set it to TERM=vte or TERM=gnome or alike, but it was damn problematic, all systems lagging behind by ages in having this terminfo installed. And, guess what, xterm doesn't support 16M colors, so g-t would need to diverge and start using its own TERM with all its disadvantages...

Let's focus on 256 colors now :)

Comment 4 Zbigniew Jędrzejewski-Szmek 2014-11-24 18:11:35 UTC
(In reply to Egmont Koblinger from comment #3)
> (In reply to Zbigniew Jędrzejewski-Szmek from comment #2)
> > But we're not using "xterm" anywhere in reality, since basically everyone
> > runs 256colors.sh in their shell and changes "xterm" to "xterm-256color".
> 
> This is RedHat/Fedora-specific. Mainstream gnome-terminal should also
> consider users of other distros, the pros and cons they'd face.
I think all distros are pretty similar here. At least the ones that
are likely to package recent gnome3 versions, which tends to go with
having a fairly recent stack all over the place.

> > The initial setting from gnome-terminal would only matter if someone run ssh
> > directly in the terminal emulator and not through a shell.
> 
> That's a good point, and this is where it becomes more important to
> advertise 256-color support.
> 
> > > If so far you relied on $COLORTERM for a workaround, you can now instead
> > > rely on $VTE_VERSION for the same workaround :)
> > That's not satisfying either ;)
> 
> Just for curiosity: how is it any worse than looking at XTERM_VERSION,
> ROXTERM_ID or KONSOLE_DBUS_SESSION, as already done in 256term.sh? ;)
1. it is an additional script to run at every shell invocation
2. initscripts should be an optional package, we don't want it to influence behaviour of
   unrelated packages like terminal emulators
3. initscripts acts a centralized gatekeeper, and every new terminal emulator
   must "register" for proper support and every terminal which updates for 256 color
   support must coordinate with initscripts.
4. separate support must be provided for each shell.
   For example, gnome-terminal -e dash does not sport 256 colors. The same is
   probably true for many alternative shells.
5. (repeating from above) ssh and other programs run directly in a separate terminal
   window don't get support 'gnome-terminal -e mutt'...
6. If I ssh from outside, using an emulator which does not support 256 colors,
   I get silently "upgraded" to TERM=xterm-256color and things break.

> > > I tend to agree that xterm-256color would be a better default, and if some
> > > system doesn't support it then the workaround should happen there.
> > Yes. Can we do the switch please?
> 
> Filed https://bugzilla.gnome.org/show_bug.cgi?id=740641
Thanks.
 
> Let's focus on 256 colors now :)
+1

Comment 5 Egmont Koblinger 2014-11-24 18:52:13 UTC
I agree that 256term.sh is not the right approach for the long term. The proper approach would require xterm, konsole, gnome-terminal etc. to be modified.

However, until this happens, updating 256term.sh to check for VTE_VERSION is the easiest quickest workaround that brings back advertising 256 colors at least for shells running inside gnome-terminals, as it used to be.  This is something Fedora should do right away, to not introduce any regression in gnome-terminal.

Comment 6 Zbigniew Jędrzejewski-Szmek 2014-11-24 19:23:08 UTC
Any particular reason not to do sed s/xterm/xterm-256color/ in gnome-terminal or vte instead?

Comment 7 Egmont Koblinger 2014-11-24 22:43:14 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #6)
> Any particular reason not to do sed s/xterm/xterm-256color/ in
> gnome-terminal or vte instead?

Not that I'm aware of, but gnome-terminal's maintainer has the final word, and that's not me.

Even if this happens, it will arrive in Gnome 3.16 next spring, meaning it'll be broken in F21.

Fedora should apply a quick fix, either doing the sed in their vte, or replacing COLORTERM by VTE_VERSION in 256test.sh, so that it's fixed in F21.

Comment 8 Zbigniew Jędrzejewski-Szmek 2014-11-28 04:49:55 UTC
Created attachment 962336 [details]
Flip the default to TERM=xterm-256color

Comment 9 Egmont Koblinger 2014-12-01 23:04:17 UTC
Fixed in mainstream vte git.

Still, Fedora should somehow fix it for F21.

Comment 10 Fedora Update System 2014-12-02 18:26:52 UTC
vte291-0.38.2-3.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/vte291-0.38.2-3.fc21

Comment 11 Fedora Update System 2014-12-03 06:06:25 UTC
Package vte291-0.38.2-3.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing vte291-0.38.2-3.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-16166/vte291-0.38.2-3.fc21
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2014-12-12 04:34:13 UTC
vte291-0.38.2-3.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.