Bug 1165439 - profile.d/256term.sh can't detect gnome-terminal anymore
Summary: profile.d/256term.sh can't detect gnome-terminal anymore
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: initscripts   
(Show other bugs)
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords:
Depends On: 1166428 1166430 1166431
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-11-19 00:55 UTC by Josh Stone
Modified: 2018-02-13 13:13 UTC (History)
10 users (show)

Fixed In Version: initscripts-9.73-1.fc27
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1166428 1166430 1166431 (view as bug list)
Environment:
Last Closed: 2018-02-13 13:11:47 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Josh Stone 2014-11-19 00:55:43 UTC
Description of problem:
I found that gnome-terminal no longer sets COLORTERM, and thus 256term.sh doesn't patch up TERM=xterm or TERM=screen to their 256color variants.

Version-Release number of selected component (if applicable):
initscripts-9.56.1-4.fc21.x86_64
gnome-terminal-3.14.2-1.fc21.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Open gnome-terminal
2. echo $TERM
3. Run screen/tmux
4. echo $TERM

Actual results:
First "xterm", then "screen".

Expected results:
On F20, it would get "xterm-256color" and "screen-256color".
COLORTERM was still being set by gnome-terminal at that time.

Additional info:

commit 1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf
Author: Christian Persch <chpe@gnome.org>
Date:   Sat Apr 26 19:13:21 2014 +0200

    screen: Stop setting COLORTERM env var
    
    COLORTERM is a long-obsolete slang-only variable used to work around
    broken termcap/terminfo entries.

https://git.gnome.org/browse/gnome-terminal/commit/?id=1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf

Comment 1 Josh Stone 2014-11-19 01:05:50 UTC
As a workaround, I've added this to my .bashrc, before global files are sourced:

if test -z "$COLORTERM" &&
    readlink /proc/$PPID/exe | grep -q gnome-terminal
then
    export COLORTERM=gnome-terminal
fi

Obviously, that can only check the immediate parent process, but so far it's ok.

Comment 2 Zbigniew Jędrzejewski-Szmek 2014-11-21 00:42:33 UTC
Whoah, what a mess ;)

So, gnome-terminal sets a $TERM entry which does not advertise full capabalities. Instead of changing that, we try to fix it up in the shell using a hacky scripts. This does not make any sense to me.

(Even if there were buggy programs which refused to work with TERM=xterm-256color, which I guess was the primal reason why it was not set properly when xterm or gnome-terminal gained 256 color support, they most likely have been fixed, since 256term.sh has been setting TERM=xterm-256color for many years now. So even if there was a reason no to do it, there isn't one now.)

Let's fix this properly. After-the-fact guessing whether a terminal supports 256 colors is not possible to do cleanly [1]. The only sane way is to set it properly from the terminal emulator program.

So I propose a two step procedure:

1. Make gnome-terminal, xterm, screen, tmux, and whatever else set a $TERM entry which advertises 256 colors. I'll file bugs for gnome-terminal and screen.

2. Wait a bit for 1. to happen, then drop 256term.* from initscripts.

The rationale is that when 1. is done, the 256term.* scripts will be noops, so doing things in this order should provide a smooth transition.

[1] http://unix.stackexchange.com/questions/23763/checking-how-many-colors-my-terminal-emulator-supports/23789#23789

Comment 3 Josh Stone 2014-11-21 01:14:29 UTC
FWIW, I kind of understand why screen and tmux don't set TERM=screen-256color even when their initial TERM is capable -- they can't predict what the user TERM will be if it reattaches later, so TERM=screen is a safer bet.

That said, 256term.sh has been forcing them into 256 colors anyway, since the following Fedora 18 feature, and the world didn't end. :)
https://fedoraproject.org/wiki/Features/256_Color_Terminals

There's some good background info in that link too.

Comment 4 Zbigniew Jędrzejewski-Szmek 2014-11-21 14:52:59 UTC
(In reply to Josh Stone from comment #3)
> FWIW, I kind of understand why screen and tmux don't set
> TERM=screen-256color even when their initial TERM is capable -- they can't
> predict what the user TERM will be if it reattaches later, so TERM=screen is
> a safer bet.
This is somewhat true, but... how often does one start a screen or tmux on a capable terminal and then attaches from a less capable one? Most of the time you use the same terminal type (even if not the same terminal). If anything, the opposite direction is more likely: start a screen on a console when trouble shooting, and then connect to it later from a full environment. And apart from the linux console, almost all terminal emulators support 256 colors (including putty and macos thingies).

And anyway, as you say, 256colors.sh makes this all moot anyway.

> That said, 256term.sh has been forcing them into 256 colors anyway, since
> the following Fedora 18 feature, and the world didn't end. :)
> https://fedoraproject.org/wiki/Features/256_Color_Terminals
I read this page a couple of times already, and each time I understand the motivations less and less. "By keying on $COLORTERM rather than $TERM we both get accurate identification of the terminal, and non propagation by ssh" — no, if we are talking about ssh out of the system, since an exported variable $TERM is propagated. If we are talking about ssh in to the system, that makes more sense, since 256colors.sh will not mangle the $TERM variable then, but the alternate solution of setting $TERM properly in the first place work just as well in this case.

Comment 5 Josh Stone 2014-11-24 17:04:38 UTC
As far as maintaining the status quo, bug 1166428 comment 1 notes that checking for $VTE_VERSION should work at least as well as $COLORTERM did.  We should at least add this to 256term.sh until the better solution arrives.

Comment 6 Zbigniew Jędrzejewski-Szmek 2014-11-24 17:11:17 UTC
I'd prefer not to add a partial workaround if a full solution is possible which requires about the same amount of work.

Comment 7 Josh Stone 2014-11-24 17:25:38 UTC
It's a one-line patch in two files to add $VTE_VERSION to the mix.  The better solution is a negotiation with several other projects, at least gnome-terminal, screen, and tmux.  I don't call that the same amount of work. :)

Comment 8 Zbigniew Jędrzejewski-Szmek 2014-11-24 17:52:28 UTC
It's a better solution because it is technically correct. Whether it's 1 line in three projects each, or two lines in one project, I'd consider less important. 
But anyway, the good thing is that it can be done independently in gnome-terminal and in other emulators. So there's nothing stopping gnome-terminal from exporting a better $TERM value, even if screen and tmux wait. And if gnome-terminal does that, maybe it'll be easier for others to follow.

Comment 9 Josh Stone 2014-11-24 20:35:01 UTC
I agree your pursuit is better in the long run, yet I still think 256term.sh should check for $VTE_VERSION now to resolve the regression, so we're at least no worse than before.

But I'll stop arguing now -- my local files will be patched this way until it's resolved officially.

Comment 10 Zbigniew Jędrzejewski-Szmek 2014-11-24 20:39:05 UTC
Sure, I'll patch 256term.sh, but I don't want this to fall by the wayside just because patching initscripts is easier.

Comment 11 Egmont Koblinger 2014-11-27 20:30:04 UTC
(In reply to Josh Stone from comment #7)
> It's a one-line patch in two files to add $VTE_VERSION to the mix.  The
> better solution is a negotiation with several other projects, at least
> gnome-terminal, screen, and tmux.  I don't call that the same amount of
> work. :)

Totally agree.

The addition of VTE_VERSION should happen in Fedora ASAP to prevent a regression.

Refactoring the current design is still desirable, but is less important and is a significantly bigger task.

Comment 12 Pádraig Brady 2014-12-09 19:16:04 UTC
This breaks in Fedora 21 due to https://git.gnome.org/browse/gnome-terminal/commit/?id=1d5c1b6ca6373c1301494edbc9e43c3e6a9c9aaf

Upstream of gnome-terminal have indicated they're not willing
to replace COLORTERM (which will break other scripts I expect).

Adding in $VTE_VERSION (in addition to COLORTERM) is the best fix for now
as I did in https://bugzilla.redhat.com/attachment.cgi?id=960055

Comment 13 Egmont Koblinger 2014-12-09 19:50:52 UTC
By the way...

Just looking at the name of the variable ("COLORTERM") and a quick glimpse at slang's source code... I believe COLORTERM was a workaround to let apps know that the terminal supports _colors_. It was probably never used and never meant to be used to denote support of _256 colors_.

So probably Fedora's way of checking for it in 256term.sh was totally broken by design and just happened to work accidentally because all the emulators that set it to denote _color_ support have just happened to support _256 colors_ too.

Checking for VTE_VERSION is a way better approach obviously.

Comment 14 Pádraig Brady 2014-12-09 20:18:08 UTC
There was no accident. I audited all terminals that exported COLORTERM to check they supported 256 colors

Comment 15 Zbigniew Jędrzejewski-Szmek 2014-12-09 20:21:23 UTC
Actually this is will be fixed already when vte291-0.38.2-3.fc21 hits stable. The only thing to do in initscripts might be to add a Conflicts:vte291 < 0.38.2-3 to ensure that the fix is propagated.

Comment 16 Josh Stone 2014-12-09 21:43:25 UTC
The vte update will fix direct xterm-256color in gnome-terminal, but screen/tmux under vte still need the kludge of 256term.sh, at least until those packages also map to screen-256color themselves.

Comment 17 Zbigniew Jędrzejewski-Szmek 2014-12-09 21:46:11 UTC
Yes. But the kludge is still there, and should still work for screen and tmux.

Comment 18 Josh Stone 2014-12-09 22:04:36 UTC
How?  Even screen/tmux were relying on environment of the outer terminal, which was getting COLORTERM, so they now need to act on VTE_VERSION.

I just tried with vte291-0.38.2-3.fc21.x86_64 and a clean 256term.sh.  Now gnome-terminal does get TERM=xterm-256color, but both tmux and screen have TERM=screen.

Comment 19 Zbigniew Jędrzejewski-Szmek 2014-12-10 03:09:53 UTC
You're right, screen now uses the default TERM=xterm, as documented in the screen(1). I think this is OK, since this is the documented upstream behaviour. Let's wait for the reply of screen maintainer.

Comment 20 Pádraig Brady 2014-12-10 12:15:42 UTC
I disagree that screen/tmux having different defaults here is correct.
I.E. the update to 256term.{c,}sh is probably still best.

For reference here is the VTE change which was done independently to this bug:
http://git.gnome.org/browse/vte/commit/?id=82a8b069

Comment 21 Zbigniew Jędrzejewski-Szmek 2014-12-10 13:09:51 UTC
I kind of agree, it would be nicer for screen and tmux to default to 256 colors, since essentially all terminal emulators we use provide that.

Comment 22 Egmont Koblinger 2014-12-10 13:16:44 UTC
(In reply to Josh Stone from comment #18)
> How?  Even screen/tmux were relying on environment of the outer terminal,
> which was getting COLORTERM, so they now need to act on VTE_VERSION.

There are other terminals out there which didn't set COLORTERM, e.g. xterm. What's up with them? (I don't understand at all what's the trouble with screen/tmux or what kinds of problems vte's change still cause and why it actually causes so much trouble.)

> I just tried with vte291-0.38.2-3.fc21.x86_64 and a clean 256term.sh.  Now
> gnome-terminal does get TERM=xterm-256color, but both tmux and screen have
> TERM=screen.

Somewhere it was pointed out that promoting 256 colors inside screen/tmux is a bad idea, since the user might reattach from an 8-color terminal. Others might argue that it's a rare use case that shouldn't set back improvement, and users of this situation should fix it for themselves. Anyway, a consistent decision should be made and implemeted.

From screen's/tmux's point of view, I can't see why COLORTERM or VTE_VERSION should be relevant. They should - if necessary - look at TERM to detect 256 color capability.

(In reply to Zbigniew Jędrzejewski-Szmek from comment #19)
> You're right, screen now uses the default TERM=xterm, as documented in the
> screen(1). I think this is OK, since this is the documented upstream
> behaviour. Let's wait for the reply of screen maintainer.

Did it change recently, or is there a distro-specific patch somewhere? My screen(1) (Ubuntu Utopic, screen-4.2.1) says its default is TERM=screen.

Anyway, it's very important that *inside* tmux/screen TERM should be set to whatever the authors of tmux/screen say, rather than the value it receives from the outside terminal. I have seen tmux break in subtle ways when setting TERM=xterm inside, that one definitely wants to see TERM=screen or screen-256color, as it's clearly documented in its manual. I assume it's the same for screen.

(In reply to Pádraig Brady from comment #20)
> For reference here is the VTE change which was done independently to this
> bug:

Not quite independently :) It was inspired by one of the sibling tickets to this one.

Comment 23 Pádraig Brady 2014-12-10 13:26:59 UTC
screen-4.2.1-4.fc21.x86_64 here defaults to TERM=screen
and if we tweak /etc/profile.d/256term.sh as suggested above it will switch to screen-256color. Anything else would be wrong

Comment 24 Egmont Koblinger 2014-12-10 13:50:31 UTC
(In reply to Pádraig Brady from comment #23)
> screen-4.2.1-4.fc21.x86_64 here defaults to TERM=screen
> and if we tweak /etc/profile.d/256term.sh as suggested above it will switch
> to screen-256color. Anything else would be wrong

This makes sense. Was comment 19 wrong then (maybe a typo) saying screen's default is TERM=xterm? I assume so.

Comment 25 Zbigniew Jędrzejewski-Szmek 2014-12-10 13:59:49 UTC
(In reply to Egmont Koblinger from comment #24)
> (In reply to Pádraig Brady from comment #23)
> > screen-4.2.1-4.fc21.x86_64 here defaults to TERM=screen
> > and if we tweak /etc/profile.d/256term.sh as suggested above it will switch
> > to screen-256color. Anything else would be wrong
> 
> This makes sense. Was comment 19 wrong then (maybe a typo) saying screen's
> default is TERM=xterm? I assume so.
Yes, I means TERM=screen (as apposed to TERM=screen-256color).

Comment 26 Petr Hracek 2015-02-24 15:07:41 UTC
But good question, if screen is going to be connected to 8bit terminal then screen should work as before.
I did not tested.

The fix can be done in row termcap.c:874 (in rawhide).

I have checked the debian screen package and TERM=screen is present.

I don't thing that it is a good idea to change them to TERM_scree-256color.

Comment 27 Fedora End Of Life 2015-11-04 13:28:21 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 28 Zbigniew Jędrzejewski-Szmek 2015-11-04 14:09:16 UTC
Pfff, I'll try to have another look at this.

Comment 29 Jan Kurik 2016-02-24 13:17:19 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase

Comment 30 Fedora End Of Life 2017-07-25 18:44:29 UTC
This message is a reminder that Fedora 24 is nearing its end of life.
Approximately 2 (two) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 24. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '24'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 24 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.


Note You need to log in before you can comment on or make changes to this bug.