Bug 1270534 - libncurses.so crashes with /usr/include/ncurses.h
Summary: libncurses.so crashes with /usr/include/ncurses.h
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: ncurses
Version: rawhide
Hardware: x86_64
OS: Linux
medium
low
Target Milestone: ---
Assignee: Miroslav Lichvar
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:8bd2959f123f8134bc86a2deb42...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-10 20:22 UTC by Marek Doležel
Modified: 2016-01-26 12:03 UTC (History)
10 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-01-26 10:54:03 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (76.44 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: cgroup (200 bytes, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: core_backtrace (6.46 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: dso_list (7.13 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: environ (3.39 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: exploitable (82 bytes, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: limits (1.29 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: maps (20.06 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: mountinfo (3.67 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: open_fds (448 bytes, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: proc_pid_status (1.15 KB, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details
File: var_log_messages (283 bytes, text/plain)
2015-10-10 20:22 UTC, Marek Doležel
no flags Details

Description Marek Doležel 2015-10-10 20:22:01 UTC
Description of problem:
1. run gdb from command line
2. tui enable
3. segmentation fault(core dumped)

Version-Release number of selected component:
gdb-7.10-28.fc24

Additional info:
reporter:       libreport-2.6.2
backtrace_rating: 4
cmdline:        gdb proj1
crash_function: wnoutrefresh
executable:     /usr/bin/gdb
global_pid:     11795
kernel:         4.3.0-0.rc4.git3.1.fc24.x86_64
runlevel:       N 5
type:           CCpp
uid:            1000

Truncated backtrace:
Thread no. 1 (10 frames)
 #0 wnoutrefresh at ../../ncurses/base/lib_refresh.c:258
 #1 wrefresh at ../../ncurses/base/lib_refresh.c:62
 #2 tui_refresh_win at ../../gdb/tui/tui-wingeneral.c:60
 #3 tui_show_locator_content at ../../gdb/tui/tui-stack.c:269
 #4 show_source_or_disasm_and_command at ../../gdb/tui/tui-layout.c:958
 #6 show_source_command at ../../gdb/tui/tui-layout.c:608
 #7 show_layout at ../../gdb/tui/tui-layout.c:104
 #8 tui_set_layout at ../../gdb/tui/tui-layout.c:149
 #10 tui_enable at ../../gdb/tui/tui.c:465
 #11 execute_command at ../../gdb/top.c:468

Comment 1 Marek Doležel 2015-10-10 20:22:07 UTC
Created attachment 1081654 [details]
File: backtrace

Comment 2 Marek Doležel 2015-10-10 20:22:09 UTC
Created attachment 1081655 [details]
File: cgroup

Comment 3 Marek Doležel 2015-10-10 20:22:11 UTC
Created attachment 1081656 [details]
File: core_backtrace

Comment 4 Marek Doležel 2015-10-10 20:22:12 UTC
Created attachment 1081657 [details]
File: dso_list

Comment 5 Marek Doležel 2015-10-10 20:22:14 UTC
Created attachment 1081658 [details]
File: environ

Comment 6 Marek Doležel 2015-10-10 20:22:15 UTC
Created attachment 1081659 [details]
File: exploitable

Comment 7 Marek Doležel 2015-10-10 20:22:17 UTC
Created attachment 1081660 [details]
File: limits

Comment 8 Marek Doležel 2015-10-10 20:22:19 UTC
Created attachment 1081661 [details]
File: maps

Comment 9 Marek Doležel 2015-10-10 20:22:21 UTC
Created attachment 1081662 [details]
File: mountinfo

Comment 10 Marek Doležel 2015-10-10 20:22:22 UTC
Created attachment 1081663 [details]
File: open_fds

Comment 11 Marek Doležel 2015-10-10 20:22:24 UTC
Created attachment 1081664 [details]
File: proc_pid_status

Comment 12 Marek Doležel 2015-10-10 20:22:26 UTC
Created attachment 1081665 [details]
File: var_log_messages

Comment 13 Jan Kratochvil 2015-10-11 17:57:09 UTC
I think it is probably rather in ncurses.  I have tried to upgrade to upstream ncurses-6.0-20151010.tgz but the crash remains the same, dump below is from 20151010:

Program received signal SIGSEGV, Segmentation fault.
wnoutrefresh (win=win@entry=0x555556a3f1a0) at ../../ncurses/base/lib_refresh.c:258
258			if (!CharEq(oline->text[src_col], nline->text[dst_col])) {
(gdb) bt
#0  wnoutrefresh (win=win@entry=0x555556a3dbb0) at ../../ncurses/base/lib_refresh.c:258
#1  0x00007ffff733da37 in wrefresh (win=0x555556a3dbb0) at ../../ncurses/base/lib_refresh.c:62
#2  0x00005555557af83a in tui_refresh_win (win_info=win_info@entry=0x5555560720a0 <_locator>) at ../../gdb/tui/tui-wingeneral.c:60
#3  0x00005555557acbe9 in tui_show_locator_content () at ../../gdb/tui/tui-stack.c:269
#4  0x00005555557a9b59 in show_source_or_disasm_and_command (layout_type=layout_type@entry=SRC_COMMAND) at ../../gdb/tui/tui-layout.c:958
#5  0x00005555557a9fe0 in show_source_or_disasm_and_command (layout_type=SRC_COMMAND) at ../../gdb/tui/tui-layout.c:898
#6  show_source_command () at ../../gdb/tui/tui-layout.c:608
#7  show_layout (layout=SRC_COMMAND) at ../../gdb/tui/tui-layout.c:104
#8  0x00005555557aa419 in tui_set_layout (layout_type=layout_type@entry=SRC_COMMAND) at ../../gdb/tui/tui-layout.c:149
#9  0x00005555557aa63a in tui_set_layout (layout_type=layout_type@entry=SRC_COMMAND) at ../../gdb/tui/tui-layout.c:131
#10 0x00005555557b0d5d in tui_enable () at ../../gdb/tui/tui.c:465
#11 0x000055555596e6f6 in execute_command (p=<optimized out>, p@entry=0x7fffffffdce7 "tui enable", from_tty=from_tty@entry=1) at ../../gdb/top.c:468
[...]
(gdb) l
253	
254		    /*
255		     * Copy the changed text.
256		     */
257		    for (; src_col <= last_src; src_col++, dst_col++) {
258			if (!CharEq(oline->text[src_col], nline->text[dst_col])) {
259			    nline->text[dst_col] = oline->text[src_col];
260			    CHANGED_CELL(nline, dst_col);
261			}
262		    }
(gdb) p oline->text
$1 = (chtype *) 0x56a3de70
(gdb) p oline->text[src_col]
value has been optimized out
(gdb) p *oline->text
Cannot access memory at address 0x56a3de70

Comment 14 Jan Kratochvil 2015-11-14 15:33:35 UTC
It cannot work, reproducible with Rawhide gdb-7.10.50.20151027-32.fc24.x86_64:
$ gdb
(gdb) focus cmd
Segmentation fault

This is because ncurses.spec:

%install
make -C narrowc5 DESTDIR=$RPM_BUILD_ROOT install.libs
make -C widec5 DESTDIR=$RPM_BUILD_ROOT install.libs
make -C narrowc6 DESTDIR=$RPM_BUILD_ROOT install.libs
make -C widec6 DESTDIR=$RPM_BUILD_ROOT install.{libs,progs,data,includes,man}

include files and *.so files from differently configured ncurses are NOT compatible while they are installed as if they were.

/usr/include/curses.h
struct _win_st
{
[...]
#if NCURSES_WIDECHAR
        cchar_t  _bkgrnd;       /* current background char/attribute pair */
#if 1
        int     _color;         /* current color-pair for non-space character */
#endif
#endif
};

GDB by default links with libncurses.so (probably as most of the applications) but then inlined functions access WINDOW::_bkgrnd but *.so functions do not allocate that field there -> buffer overrun.

==11223==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b00001ba24 at pc 0x0000009273c3 bp 0x7ffdd1e7b050 sp 0x7ffdd1e7b040
WRITE of size 4 at 0x60b00001ba24 thread T0
    #0 0x9273c2 in tui_show_locator_content tui/tui-stack.c:265
    #1 0x920d3f in show_source_or_disasm_and_command tui/tui-layout.c:970
    #2 0x91f2bb in show_source_command tui/tui-layout.c:610
    #3 0x91ddc9 in show_layout tui/tui-layout.c:104
    #4 0x91dff9 in tui_set_layout tui/tui-layout.c:149
    #5 0x934fee in tui_enable tui/tui.c:465
    #6 0x92c1bb in tui_set_focus_command tui/tui-win.c:1087
    #7 0x8d8561 in do_cfunc cli/cli-decode.c:105
    #8 0x8e0037 in cmd_func cli/cli-decode.c:1885
    #9 0xe0dfa9 in execute_command /home/jkratoch/redhat/gdb-test-master/gdb/top.c:475
    #10 0xb7396c in catch_command_errors /home/jkratoch/redhat/gdb-test-master/gdb/main.c:368
    #11 0xb760e2 in captured_main /home/jkratoch/redhat/gdb-test-master/gdb/main.c:1133
    #12 0xb69430 in catch_errors /home/jkratoch/redhat/gdb-test-master/gdb/exceptions.c:240
    #13 0xb7617f in gdb_main /home/jkratoch/redhat/gdb-test-master/gdb/main.c:1165
    #14 0x55ba6b in main /home/jkratoch/redhat/gdb-test-master/gdb/gdb.c:32
    #15 0x7fadefc775ff in __libc_start_main (/lib64/libc.so.6+0x205ff)
    #16 0x4ad638 in _start (/home/jkratoch/redhat/gdb-test-master/gdb/gdb+0x4ad638)

0x60b00001ba24 is located 28 bytes to the right of 104-byte region [0x60b00001b9a0,0x60b00001ba08)
allocated by thread T0 here:
    #0 0x528979 in __interceptor_calloc (/home/jkratoch/redhat/gdb-test-master/gdb/gdb+0x528979)
    #1 0x7fadf19fd5cd in _nc_makenew_sp (/lib64/libncurses.so.6+0x135cd)

Comment 15 Jan Kratochvil 2015-11-14 16:07:32 UTC
Workarounded in gdb-7.10.50.20151113-33.fc24 but it is an ncurses Bug:
  http://pkgs.fedoraproject.org/cgit/gdb.git/tree/gdb-fedora-libncursesw.patch

Comment 16 Miroslav Lichvar 2015-11-16 07:03:50 UTC
In reply to Jan Kratochvil from comment #14)
> include files and *.so files from differently configured ncurses are NOT
> compatible while they are installed as if they were.

The upstream intention is for the headers from the wide configuration to be compatible with headers from the non-wide configuration. That is how it was done in Fedora for a very long time. I suspect this is rather something related to the update to ncurses-6.0.

Comment 17 Thomas E. Dickey 2015-11-30 01:55:06 UTC
hmm - the patch changes the search rules.  I made that change to
allow me to build ncurses6 libraries (and headers) with different
suffixes so I could have both ncurses5 and ncurses6 development
and runtime on the same machine.

Since the ncurses5 versus ncurses6 headers differ with the size
of cchar_t, there's always a problem in mixing those.

I can experiment with the suggested patch, to see whether how
it fits in...

Comment 18 Marek Doležel 2016-01-12 19:43:39 UTC
Fixed. Works fine with ncurses 6.0-1.20150810.fc24

Closing bug

Comment 19 Jan Kratochvil 2016-01-16 03:27:12 UTC
Why fixed?  I see no new ncurses build.  You probably mean CLOSED-WORKSFORME.

But GDB "focus cmd" still crashes for me the same way on fresh Rawhide with:
  ncurses-6.0-1.20150810.fc24.x86_64
and GDB without the workaround.

For reproducibility you need older Rawhide GDB:
  https://koji.fedoraproject.org/koji/buildinfo?buildID=698893
  https://kojipkgs.fedoraproject.org//packages/gdb/7.10.50.20151027/32.fc24/src/gdb-7.10.50.20151027-32.fc24.src.rpm

as I have already workarounded this ncurses Bug in Rawhide GDB - see Comment 15.

Comment 21 Jan Kratochvil 2016-01-21 20:00:20 UTC
Sorry Marek Dolezel, that NEEDINFO was a typo.

Comment 22 Miroslav Lichvar 2016-01-22 11:56:56 UTC
The reason why it worked with ncurses5 seems to be that there were no extended colors. In ncurses6 it's enabled by default in the wide variant and the wattr macros access the _color field of the WINDOW structure, which is not allocated in the narrow variant.

I think it might be fixed in several different ways:

1) Remove the macros from ncurses.h. This would have an impact on performance, I'm not sure if that actually would be a problem.

2) Go back to separate headers for narrow and wide ncurses in /usr/include/ncurses{,w}. This would likely break packages that don't use pkg-config to get the ncurses CFLAGS and assume the headers are in some fixed location. I'm not sure how many packages like that we have in Fedora.

3) add a new flag that could be used by the macros to check if WINDOW has the _color field or not.

Thomas, as the upstream maintainer, do you have any suggestions?

Comment 23 Thomas E. Dickey 2016-01-23 19:23:06 UTC
Actually, extended colors are in two places in the header file:
the ext_color value in cchar_t and the _color value in WINDOW.
I have a hunch that ext_color contributes more to the reported
problem than _color

There's no simple solution (that I can see) which would allow
applications to do "#include <curses.h>" to bridge across ABI5/ABI6.
It's not the same as between ncurses/ncursesw (for which there was
a well-known symbol to define to get the latter: _XOPEN_SOURCE_EXTENDED).

That was why I spent some time making the configure scripts handle
suffixes for the library (and utilities).  In my test-packages, I
have /usr/include/ncursesw6/curses.h, which works with the ncursesw6-config

If you think it would be useful, I could add a symbol to turn _off_
the extended-color data fields in curses.h at compile-time (to allow
packages to build against ABI5 libraries), though that doesn't seem
like much of a solution.  Still, it's preferable to requiring a new
symbol to build with the new libraries.

Comment 24 Miroslav Lichvar 2016-01-25 11:21:19 UTC
(In reply to Thomas E. Dickey from comment #23)
> I have a hunch that ext_color contributes more to the reported
> problem than _color

FWIW, disabling the wattr* macros in ncurses.h and recompiling gdb (linked with libncurses.so.6) seems to fix the reported problem for me.

> There's no simple solution (that I can see) which would allow
> applications to do "#include <curses.h>" to bridge across ABI5/ABI6.

In the Fedora package we don't really need that. the ABI5 libraries are there just to provide compatibility for older packages until they are rebuilt with ABI6, there are no *.so or pkg-config files for the ABI5 libs.

The problem seems to be that the ABI6 ncursesw headers can't be safely used with applications linking to libncurses.so.6.

> If you think it would be useful, I could add a symbol to turn _off_
> the extended-color data fields in curses.h at compile-time (to allow
> packages to build against ABI5 libraries), though that doesn't seem
> like much of a solution.  Still, it's preferable to requiring a new
> symbol to build with the new libraries.

If there was a macro that would select the narrow variant, I think that would  work, at least for applications that use CFLAGS from the pkg-config file.

I'm wondering how big would be the performance impact if the wattr* macros were just removed. That would be a simple and reliable fix.

Comment 25 Jan Kratochvil 2016-01-25 12:27:05 UTC
Why Fedora ships both libncurses and libncursesw at all? If the include file is expected to be compatible with both and Fedora rebuilds everything from source then what is the point of libncurses?

Comment 26 Miroslav Lichvar 2016-01-25 12:50:48 UTC
I think the main reason for keeping separate libncurses/libncursesw was to save memory with applications that don't really need wchars. The cchar_t structure now seems to be 7 times larger than chtype. A couple of larger terminal windows could use few extra megabytes of memory. Is it worth it these days? I'm not sure.

Comment 27 Jan Kratochvil 2016-01-26 07:58:51 UTC
Any memory or performance optimization of anything like ncurses I find really useless nowadays.  Moreover when it complicates its use.

So the problem is solved, isn't it?  Drop libncurses and rename libncursesw->libncurses, the same with include directories and its config script.  I do not know if libncursesw variant is Fedora specific or whether upstream packages expect it?

Comment 28 Miroslav Lichvar 2016-01-26 08:16:04 UTC
(In reply to Jan Kratochvil from comment #27)
> So the problem is solved, isn't it?  Drop libncurses and rename
> libncursesw->libncurses, the same with include directories and its config
> script.  I do not know if libncursesw variant is Fedora specific or whether
> upstream packages expect it?

That's quite a radical change. Before doing that I'd like to see some discussion on the devel list. I think most if not all major distros still carry both libncurses and libncursesw.

For now, I think I'll just remove the macros from the header so it's compatible with libncurses.

Comment 29 Miroslav Lichvar 2016-01-26 10:54:03 UTC
As of ncurses-6.0-3.20160116.fc24, the wattr* macros in ncurses.h are disabled unless NCURSES_INTERNALS is defined. It may be a temporary solution. When upstream supports this combination of narrow and wide-char ncurses with one set of headers, or we decide to drop the narrow-char variant and switch everything to wide-char, the macros can be enabled again.

Comment 30 Thomas E. Dickey 2016-01-26 11:07:30 UTC
I don't have a better suggestion, at the moment.

Comment 31 Jan Kratochvil 2016-01-26 12:03:27 UTC
Thanks.  IIUC all applications with all versions of ncurses should always prefer ncursesw so that my workaround from Comment 15 is in fact good as it at least enables wide characters support.


Note You need to log in before you can comment on or make changes to this bug.