Bug 1436394

Summary: update to 4.8.19, switch from slang to ncurses, other cleanups
Product: [Fedora] Fedora Reporter: Tomasz Kłoczko <kloczko.tomasz>
Component: mcAssignee: Gwyn Ciesla <gwync>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: alekcejk, aliakc, cheese, dvlasenk, gwync, jskarvad, nerijus, novyjindrich, pahan, plroskin, slavazanko
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: mc-4.8.19-1.fc25 mc-4.8.19-2.fc26 mc-4.8.19-2.fc24 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-09 21:25:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
mc.spec patch
none
mc.spec.patch - go back to slang
none
/mc.spec.patch - go back to slang v2 none

Description Tomasz Kłoczko 2017-03-27 20:22:26 UTC
Created attachment 1266751 [details]
mc.spec patch

Please review changes in spec file (patch is in attachment).
If those changes will be approved please let me know should I request access to mc package repo or you will commit those changes.

Thx

Copy of the %changelog entry:

* Mon Mar 27 2017 Tomasz Kłoczko <kloczek> - 1:4.8.19-1
- updated to 4.8.19
- drop use slang and use ncurses. There are only few packages which are using
  slang. As ncurses support is fully working now it makes more sense to
  use it instead slang (Solaris 11.3 mc uses now ncurses)
- instead passing -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE to CFLAGS use
  --enable-largefile autoconf param
- removed Group, BuildRoot and %%clear (new packaging policy)
- added pkgconfig to BuildRequires
- replaced groff by groff-base in BuildRequires (only nfroff is used)
- use %%autosetup in %%prep
- added using %%{make_build} and %%{make_build} macros
- "rm -rf $RPM_BUILD_ROOT" on beginning %%install is no longer needed
- mcfs is no longer supported (removed --enable-vfs-mcfs autoconf option)
- added explicit enabled other VFSesess to force necessary checks

Comment 1 Gwyn Ciesla 2017-04-05 12:41:44 UTC
This looks good, I'll take care of it.  Thank you!

Comment 2 Fedora Update System 2017-04-05 13:13:09 UTC
mc-4.8.19-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-34bd61b9fa

Comment 3 Fedora Update System 2017-04-05 13:13:26 UTC
mc-4.8.19-1.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-b857531f8e

Comment 4 Fedora Update System 2017-04-05 13:13:37 UTC
mc-4.8.19-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-f831745890

Comment 5 Tomasz Kłoczko 2017-04-05 19:21:53 UTC
Thank you very much :)
Close this ticket or you are going to do this?

Comment 6 Fedora Update System 2017-04-05 19:54:46 UTC
mc-4.8.19-1.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-b857531f8e

Comment 7 Fedora Update System 2017-04-05 19:55:00 UTC
mc-4.8.19-1.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-34bd61b9fa

Comment 8 Fedora Update System 2017-04-05 21:55:58 UTC
mc-4.8.19-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-f831745890

Comment 9 Pavel Roskin 2017-04-06 18:44:01 UTC
Keys learned in "Learn Keys" are ignored in mc-4.8.19-1.fc26.x86_64, and I believe ncurses causing it.
https://bugzilla.redhat.com/show_bug.cgi?id=1439886

Comment 10 Fedora Update System 2017-04-07 03:48:52 UTC
mc-4.8.19-1.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 11 Nerijus Baliūnas 2017-04-07 10:21:52 UTC
Could you please take a look at bug 1383054? It is assigned to old maintainer as I understand.

Comment 12 Ali Akcaagac 2017-04-07 19:44:03 UTC
Fx keys broken (in combination with ctrl, shift, alt). Ncurses related:

https://bugzilla.redhat.com/show_bug.cgi?id=1440110
http://midnight-commander.org/ticket/3807
http://midnight-commander.org/ticket/3254

Comment 13 Tomasz Kłoczko 2017-04-07 20:56:11 UTC
Nevertheless those bugs needs to be fixed in ncurses because they will be affecting other applications as well. Sad that project so mature like ncurses still have such bugs :(
Will try to have look on this ncurses issue as well. Probably all what is needed here is few small fixes in terminfo.src in ncurses used to generate all binary terminal definitions.

Switching from SLang to ncurses is quite important from point of view minimizing number of dependencies. SLang is used only by few packages and ncurses is used by almost any tty capable application.

As kind of long term target it would be good as well rewrite all mc extfs scripts to use only SH. Now mc drags some perl dependencies on some minimal install profiles.

PS. My kind of private target is to remove SLang from @base and @core install profiles.
Outside of those two for example aalib can use ncurses and if it will possible to remove SLang dependency from gimp.

Comment 14 Pavel Roskin 2017-04-07 21:02:26 UTC
I don't think it's a bug in ncurses. I think it's a bug in the way mc uses ncurses. For learned keys to have effect, mc should use raw input and parse key sequences on its own. We know that mc can capture key sequences, so it's possible, it's just not implemented apart from the "Learn Keys" dialog.

Comment 15 Ali Akcaagac 2017-04-07 21:09:44 UTC
@Tomasz Kłoczko

While I understand your concerns, I need to inform that the spec changes also happened for Fedora 24 and Fedora 25 (stable distributions). This issue will render the user experience to break AND will in bad circumstances cause data loss. From what I know this is violating some of the Fedora release guidelines (Please don't ask me which ones - Something like "Never break user experiences")...

As explained here:
http://midnight-commander.org/ticket/3254

The issue might be more tricky than just providing "small fixes in terminfo.src" because - from what I understood by reading above ticket - that this might also be related on the tty used, how the kernel addresses the keys and so on... An issue that hasn't been touched for 8 and more years... (looking at that ticket and the ticket number its referring to inside that ticket).

After all... I am not really sure...

May I suggest, holding the packages for Fedora 24 and Fedora 25 back, until the issue has been brought up somewhere ? Pressing Shift+F6 accidently because the user expects "Move As" but gets "Delete what you selected and in worst case without feedback" might be - at least in my terms - a horrible "user experience".

Comment 16 Tomasz Kłoczko 2017-04-07 21:32:37 UTC
Feel tree to retrieve new mc which is using ncurses from F24/F25 if you think that it cannot wait day or two.
All what needs to be changed is change BuildRequires entry from ncurses-devel to slang-devel and another line change in %configure params to use back --with-screen=slang

I've added comment in mc ticket that all such bug reports should have included used $TERM value. This will allow precisely locate what needs to be corrected in terminfo.src. My i=understanding is that at least xterm-256color needs to be fixed as I'm able to reproduce it on gnome terminal. There are couple of other terminal applications so similar checks needs to be done on those term applications as well.

This issue should be reassigned to ncurses and/or some subticket in ncurses should be created. I'm not sure what is the normal bugzilla procedure in such cases when bug sits outside affected package (?)

Comment 17 Ali Akcaagac 2017-04-07 21:53:45 UTC
I think I found that document:

https://fedoraproject.org/wiki/Updates_Policy#All_other_updates
https://fedoraproject.org/wiki/Updates_Policy#Problems_or_issues_with_Updates

I usually avoid posting that stuff, since I don't even know myself for sure what's meant there... But these guidelines have some relevance...

It's ok to change ncurses and curses related terminal config files for a development release of Fedora... To test stuff, to revert mistakes, to improve etc... But I am not sure how this can be explained to already released Fedora versions and versions of Fedora that recently released Alpha images to the masses (Alpha Freeze)...

Just my thoughts here... But the "ncurses" mc packages for the stable Fedora releases should be removed - temporarely - until the issue can be explained or decided on... FESCO ?

Think about the users who read "F6 Move As" in the mc Keybar and then get their stuff wiped out because ncurses decided to put Shift F6 functionality on Shift F4.

Right now from my quick invesitgation

(Slang):

F1 = Help
F2 = User Menu
F3 = View (internal or external editor)
Shift F3 = Force internal editor
F4 = Edit
Shift F4 = Edit new
F5 = Copy
Shift F5 = Copy As
F6 = Move
Shift F6 0 Move As
F7 = ...

(NCurses)

F1 = Help
F2 = User Menu
F3 = View (internal or external editor)
Shift F3 = Copy As
F4 = Edit
Shift F4 = Move As
F5 = Copy
Shift F5 = ...
F6 = Move
Shift F6 = Delete
F7 = ...

(In reply to Tomasz Kłoczko from comment #16)
> I'm not sure what is the normal bugzilla procedure in
> such cases when bug sits outside affected package (?)

I think contacting FESCO is the appropriate way (as descriebed in the updates policy).

Comment 18 Pavel Roskin 2017-04-07 21:56:25 UTC
I would hate to see any other packages broken while fixing a bug in Midnight Commander. Please refrain from opening bugs in ncurses, tweaking terminfo and so on, unless there is a solid evidence that the issue is indeed outside Midnight Commander.

I don't care if Shift-F6 is called F16 or F18, I only care that it doesn't remove files in mc.

Comment 19 Tomasz Kłoczko 2017-04-07 23:18:25 UTC
Just made few experiments and looks like issue is really not so simple as I've been thinking initially.

It is possible to display terminal combination generated by exact key by press ctrl-v and then key.
This is what I see in bash command line inside gnome-terminal after pressing sequence ctrl-v, F6, space, ctrl-v shift-F6:

[tkloczko@domek SPECS]$ ^[[17~ ^[[17;2~

At the same time in terminal definition is:

[tkloczko@domek SPECS]$ infocmp xterm-256color | grep kf6=
	kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
[tkloczko@domek SPECS]$ infocmp xterm-256color | grep kf16=
	kf14=\E[1;2Q, kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~,

So looks like E[17~;2~ has been interpreted as \E[1;2S.

I'm asking to apply attached spec patch to move back to slang because investigating this issue may take more time than I have now.

Next time when I'll propose move mc to slang will include in ticket detailed tests results and will be recommending apply such change only to rawhide asking on fedora-devel list to test new mc.

Sorry for the trouble but I was not aware all those issues :(


Copy of the %changelog entry from the patch:

* Fri Apr 07 2017 Tomasz Kłoczko <kloczek> - 1:4.8.19-2
- go back to slang as it is serious issue with shift-f6 when ncurses is used
  displaying "Delete" instead "Move As" dialog (#1436394)
- reformat %%description to 80 cols
- really remove Group

Comment 20 Tomasz Kłoczko 2017-04-07 23:20:32 UTC
Created attachment 1269953 [details]
mc.spec.patch - go back to slang

Comment 21 Pavel Roskin 2017-04-07 23:33:15 UTC
Tomasz, I'd like to correct your explanation.

That's what the terminal sends

\E[17~ is F6
\E[17;2~ is Shift-F6 (2 is a modifier for Shift)

That's what infocmp says:

kf18=\E[17;2~

So Shift-F6 is interpreted as F18 by ncurses. That's OK. What's not OK is that I cannot teach mc to interpret \E[17;2~ as F16 if mc is linked against ncurses. That's because mc relies on ncurses for key interpretation instead of applying its own rules.

Slang is not so smart as ncurses, so mc does key parsing on its own and applies the learned rules.

Comment 22 Tomasz Kłoczko 2017-04-08 01:03:55 UTC
(In reply to Pavel Roskin from comment #21)
[..]
> So Shift-F6 is interpreted as F18 by ncurses. That's OK. What's not OK is
> that I cannot teach mc to interpret \E[17;2~ as F16 if mc is linked against
> ncurses. That's because mc relies on ncurses for key interpretation instead
> of applying its own rules.
> 
> Slang is not so smart as ncurses, so mc does key parsing on its own and
> applies the learned rules.

Pavel I don't think that this should be solved by mc learn kys function :)
But you are right about kf18 (I did not checked what is in this terminfo definition).

Theoretically terminal application should not have hardcoded generate exact terminal sequences and should read first terminfo definition to assign exact sequences to exact keys or key combinations. With this should be possible to change by terminal app $TERM value and all should remap to new layout of terminal sequences.
So this is I think (atm) that it may be an issue with gnome-terminal because it generates wrong terminal sequences (not this one which is in kf16.

None of the applications within terminal application should be relying on internally hard coded sequences so in this case looks like SLang maintainer instead push necessary change to terminfo.src probably "solved" the issue in SLang hiding bug in ncurses.
Similar issues is related to many problems for example in readline which had in the past (and maybe still has) in few places hardcoded own terminal sequences instead fully rely on terminal definitions. More than 10 years ago I've been able to fix readline by only remove such dirty hacks (will try to check how it is with current readline :)).

Comment 23 Ali Akcaagac 2017-04-08 07:19:34 UTC
(In reply to Tomasz Kłoczko from comment #20)
> Created attachment 1269953 [details]
> mc.spec.patch - go back to slang

Please change:
--with-screen=ncurses
to
--with-screen=slang

Comment 24 Nerijus Baliūnas 2017-04-08 09:19:36 UTC
And please add --with-x as well (see bug 1383054).

Comment 25 Tomasz Kłoczko 2017-04-08 10:46:19 UTC
Created attachment 1270000 [details]
/mc.spec.patch - go back to slang v2

Done.

Comment 26 Ali Akcaagac 2017-04-09 18:17:43 UTC
(In reply to Tomasz Kłoczko from comment #25)
> Created attachment 1270000 [details]
> /mc.spec.patch - go back to slang v2

What about the --with-x from comment #24. And please, it would help if the new spec is updated to git and new packages build...

Thanks

Comment 27 Gwyn Ciesla 2017-04-10 13:10:41 UTC
Sadly this has already gone to stable for f25.  Tomasz, do you intend to commit the move back to slang or shall I?

Comment 28 Tomasz Kłoczko 2017-04-10 17:31:24 UTC
Cannot promise but if mc package maintainer will not have look on this will try to check --with-x what is exact impact of this ption.
IMO if it would add dependencies from some X11 libraries would be better to keep --without-x.
As I see that most of the terminals have defined sequences from kf1 to kf60 it mean that terminal application may be able to send ctrl, shift and alt combinations.
Other alt keys should be already present as well.
Only advantage of X11 support may be that only on pressing alt, ctrl or shift on bottom F keys bar descriptions may be changing.
I've not been trying --with-x quite long time. Can you tell what more such version of mc can do?

Comment 29 Nerijus Baliūnas 2017-04-10 18:16:24 UTC
No, it will not add dependencies from some X11 libraries (see bug 1383054):

"I'm not quite sure why Fedora people have turned this off. Maybe they 
simply don't understand what it does and believe that this will introduce 
an install-time / run-time dependency on X11. That is not the case, mc 
will be able to install and run without X11 just as before, only in 
addition it will also dynamically load X11 libraries and use them if they 
are available."

Comment 30 Nerijus Baliūnas 2017-04-10 18:20:25 UTC
Here is the whole thread about --with-x. As you see, the mc devels agree that it is a good thing:

https://mail.gnome.org/archives/mc-devel/2016-October/msg00002.html

Comment 31 josef radinger 2017-04-17 17:54:28 UTC
maybe related:
ctrl-pgdn and ctrl-pgup behaves odd (no effect) in mcedit and mc with 4.8.19

the version 4.8.17 works ok

tested in fedora 25


[root@localhost ~]# cat
pgdown
pgdown
^[[6;5~



pgup
pgup
^[[5;5~

Comment 32 Tomasz Kłoczko 2017-04-17 18:07:54 UTC
(In reply to josef radinger from comment #31)
> maybe related:
> ctrl-pgdn and ctrl-pgup behaves odd (no effect) in mcedit and mc with 4.8.19
> 
> the version 4.8.17 works ok

If you will find more such issues please let me know on <kloczek>
Just opened https://bugzilla.redhat.com/show_bug.cgi?id=1442842 which should temporary solve tis issue as well.

Comment 33 Nerijus Baliūnas 2017-04-17 21:03:08 UTC
Shift+F7 does not work in viewer or editor (find next occurence).

Comment 34 Ali Akcaagac 2017-04-28 07:36:07 UTC
I'd like to inform, that I will be going to fill a ticket to FESCo if this issue isn't being fixed till 1st of may 2017. This situation exists for over 3 weeks now.

The situation is inacceptable since it affects "stable" distributions as well. Changing functionality of one of the most common used console applications as well as introducing a library change in said "stable" distributions. This is a clear violation of the release rules (which anyone can dig up on their own).

I'd really like to see this issue fixed, so we can avoid getting FESCo dealing with this.

Thanks

Comment 35 Jaroslav Škarvada 2017-04-28 09:08:42 UTC
This bug is closed as errata and I think it's correct resolution (because the subject is fullfilled and errata was released). Previously, I opened separate bug 1440856 for this issue.

I also thing it's unacceptable to break stable release with such changes. Unfortunately it's not yet fixed in the dist-git.

Comment 36 Fedora Update System 2017-05-05 13:07:55 UTC
mc-4.8.19-2.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-bf0dcd75c6

Comment 37 Fedora Update System 2017-05-05 13:08:39 UTC
mc-4.8.19-2.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-70c2e0ec52

Comment 38 Fedora Update System 2017-05-05 20:59:29 UTC
mc-4.8.19-2.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-70c2e0ec52

Comment 39 Fedora Update System 2017-05-05 22:39:15 UTC
mc-4.8.19-2.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-bf0dcd75c6

Comment 40 Fedora Update System 2017-05-09 21:25:10 UTC
mc-4.8.19-2.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 41 Fedora Update System 2017-05-17 05:56:33 UTC
mc-4.8.19-2.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.