Bug 1656165

I've never used the readline library's tab-completion feature in the crash
utility (I didn't even consider it being enabled).  I'm certainly not
familiar with the library's internals, so don't hold your breath awaiting
a fix.

The failure can occur in multiple different paths, where the damage has
been done before the corruption is recognized.  Here's a couple more
relevant backtraces than the one in the attached file, where the failure
occurs while executing the readline() call:

crash> whatis mu*** Error in `./crash': free(): invalid pointer: 0x00007f095e62a000 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81489)[0x7f095d559489]
/lib64/libc.so.6(_IO_free_backup_area+0x1a)[0x7f095d55473a]
/lib64/libc.so.6(_IO_file_overflow+0x1d5)[0x7f095d553ea5]
/lib64/libc.so.6(_IO_file_xsputn+0xb0)[0x7f095d552810]
/lib64/libc.so.6(fputs+0xbb)[0x7f095d546e4b]
./crash[0x761e24]
./crash[0x762180]
./crash(fprintf_filtered+0x8c)[0x76320c]
./crash(throw_exception+0x63)[0x6a5cb3]
./crash[0x6a5f49]
./crash[0x6a6166]
./crash[0x760bc4]
./crash(c_parse_internal+0x32d6)[0x618a56]
./crash(c_parse+0x159)[0x618db9]
./crash[0x6d183a]
./crash(parse_expression_for_completion+0x71)[0x6d1b31]
./crash(expression_completer+0x76)[0x6b0726]
./crash[0x6afbf9]
./crash(readline_line_completion_function+0x59)[0x6b0669]
./crash(rl_completion_matches+0x61)[0x793331]
./crash(rl_complete_internal+0xf8)[0x793528]
./crash(_rl_dispatch_subseq+0x173)[0x78be63]
./crash(readline_internal_char+0x9f)[0x78c16f]
./crash(readline+0x45)[0x78c775]
./crash(process_command_line+0x1c3)[0x54f4a3]
./crash(main_loop+0x1e5)[0x467ed5]
./crash[0x6a7733]
./crash(catch_errors+0x7a)[0x6a645a]
./crash[0x6a86c6]
./crash(catch_errors+0x7a)[0x6a645a]
./crash(gdb_main_entry+0x47)[0x6a8a27]
./crash(main+0x775)[0x466265]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f095d4fa3d5]
./crash[0x46750e]

crash> whatis mu*** Error in `./crash': malloc(): memory corruption: 0x00007feeec3f7010 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x82c96)[0x7feeeb325c96]
/lib64/libc.so.6(+0x8382c)[0x7feeeb32682c]
/lib64/libc.so.6(realloc+0x1d2)[0x7feeeb328832]
./crash(xrealloc+0x1d)[0x7873ad]
./crash(vec_o_reserve+0x5f)[0x73bdff]
./crash[0x673fa6]
./crash(default_make_symbol_completion_list_break_on+0x3ac)[0x678a9c]
./crash(location_completer+0x322)[0x6b0222]
./crash(expression_completer+0x11e)[0x6b07ce]
./crash[0x6afbf9]
./crash(readline_line_completion_function+0x59)[0x6b0669]
./crash(rl_completion_matches+0x61)[0x793331]
./crash(rl_complete_internal+0xf8)[0x793528]
./crash(_rl_dispatch_subseq+0x173)[0x78be63]
./crash(readline_internal_char+0x9f)[0x78c16f]
./crash(readline+0x45)[0x78c775]
./crash(process_command_line+0x1c3)[0x54f4a3]
./crash(main_loop+0x1e5)[0x467ed5]
./crash[0x6a7733]
./crash(catch_errors+0x7a)[0x6a645a]
./crash[0x6a86c6]
./crash(catch_errors+0x7a)[0x6a645a]
./crash(gdb_main_entry+0x47)[0x6a8a27]
./crash(main+0x775)[0x466265]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7feeeb2c53d5]
./crash[0x46750e]

The rl_completion_matches() function is where the transition is made
from the readline library code to base gdb code.

I can't really figure out how to effectively debug this, given that the damage
has been done by the time the malloc/free/corruption is detected.  Staring
at the code doesn't show anything obvious.  My best guess is that it has
more to do with the embedded gdb completion code than the readline library
itself.  Or perhaps it's an issue related to the crash/gdb marriage, where
it is the only place where gdb code is invoked directly without the top-level
crash utility invoking gdb through its well-defined interface.  That alone is
a little bit disconcerting.

Anyway, I think I'll take a look at writing a readline completer plugin,
which would take gdb totally out of the picture.  It should be faster than
using the gdb completer, and would also remove the useless clutter of 
showing filenames as a completion option, which makes no sense.

> ...
> Anyway, I think I'll take a look at writing a readline completer plugin,
> which would take gdb totally out of the picture.  It should be faster than
> using the gdb completer, and would also remove the useless clutter of 
> showing filenames as a completion option, which makes no sense.

A patch has been applied upstream:

https://github.com/crash-utility/crash/commit/0f65ae0c36bf04e22219f28c32c3ae0cdee5acfe

  Implemented a new plugin function for the readline library's tab
  completion feature.  Without the patch, the use of the default plugin
  from the embedded gdb module has been seen to cause segmentation
  violations or other fatal malloc/free/corruption assertions.  The new
  plugin takes gdb out of the picture entirely, and also restricts the
  matching options to just symbol names, so as not to clutter the
  results with irrelevant filenames.
  (anderson)


Also, because the top-level crash code already has a symbol list, the new
plugin avoids having to do the malloc/realloc/frees that the gdb code
does in generating the list of matching options -- which is where I 
*believe* the reported problem lies.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2071