Bug 2019936

+++ This bug was initially created as a clone of Bug #2011520 +++

I have only a coredump, and no easy access to a cpu like this:

flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xsaves arat pku ospke md_clear flush_l1d arch_capabilities

Some data from the coredump:

0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007efd5d515c45 in __GI_abort () at abort.c:79
#2  0x000055fbe3324ba5 in std::__replacement_assert (__condition=0x55fbe335c5f0 "__builtin_expect(__n < this->size(), true)", __function=<synthetic pointer>, __line=950, __file=0x55fbe335c620 "/usr/include/c++/8/bits/stl_vector.h")
    at /usr/include/c++/8/x86_64-redhat-linux/bits/c++config.h:2391
#3  std::vector<reg, std::allocator<reg> >::operator[] (__n=67, this=<optimized out>) at /usr/include/c++/8/bits/stl_vector.h:950
#4  find_register_by_number (n=67, tdesc=<optimized out>) at ../../../gdb/gdbserver/regcache.c:204
#5  register_size (n=67, tdesc=<optimized out>) at ../../../gdb/gdbserver/regcache.c:291
#6  supply_register_zeroed (regcache=regcache@entry=0x55fbe4c331c0, n=n@entry=67) at ../../../gdb/gdbserver/regcache.c:344
#7  0x000055fbe3351b1d in i387_xsave_to_cache (regcache=0x55fbe4c331c0, buf=0x55fbe4c382b0) at ../../../gdb/gdbserver/i387-fp.c:836
#8  0x000055fbe333acbb in regsets_fetch_inferior_registers (regsets_info=0x55fbe3581a60 <x86_regsets_info>, regcache=regcache@entry=0x55fbe4c331c0) at ../../../gdb/gdbserver/linux-low.c:5410
#9  0x000055fbe333d9e5 in linux_fetch_registers (regcache=0x55fbe4c331c0, regno=<optimized out>) at ../../../gdb/gdbserver/linux-low.c:5712
#10 0x000055fbe332455e in get_thread_regcache (thread=0x55fbe4c33060, fetch=fetch@entry=1) at ../../../gdb/gdbserver/regcache.c:58
#11 0x000055fbe33266cf in prepare_resume_reply (buf=0x55fbe4c1f553 "", buf@entry=0x55fbe4c1f550 "T05", ptid=..., status=status@entry=0x55fbe4c33090) at ../../../gdb/gdbserver/remote-utils.c:1224
#12 0x000055fbe332cd92 in handle_status (own_buf=0x55fbe4c1f550 "T05") at ../../../gdb/gdbserver/server.c:3391
#13 process_serial_event () at ../../../gdb/gdbserver/server.c:4065
#14 handle_serial_event (err=<optimized out>, client_data=<optimized out>) at ../../../gdb/gdbserver/server.c:4391
#15 0x000055fbe331f31c in handle_file_event (event_file_desc=<optimized out>) at ../../../gdb/gdbserver/event-loop.c:418
#16 0x000055fbe331f7c4 in process_event () at ../../../gdb/gdbserver/event-loop.c:174
#17 start_event_loop () at ../../../gdb/gdbserver/event-loop.c:537
#18 0x000055fbe33100aa in captured_main (argv=<optimized out>, argc=<optimized out>) at ../../../gdb/gdbserver/server.c:3863
#19 main (argc=<optimized out>, argv=<optimized out>) at ../../../gdb/gdbserver/server.c:3950

(gdb) f 6
#6  supply_register_zeroed (regcache=regcache@entry=0x55fbe4c331c0, n=n@entry=67) at ../../../gdb/gdbserver/regcache.c:344
344		  register_size (regcache->tdesc, n));
(gdb) p regcache.tdesc.reg_defs
$1 = std::vector of length 67, capacity 128 = {{name = 0x55fbe4c2dd88 "eax", offset = 0, size = 32}, {name = 0x55fbe4c33ee8 "ecx", offset = 32, size = 32}, {name = 0x55fbe4c33f98 "edx", offset = 64, size = 32}, {
    name = 0x55fbe4c34028 "ebx", offset = 96, size = 32}, {name = 0x55fbe4c340b8 "esp", offset = 128, size = 32}, {name = 0x55fbe4c34148 "ebp", offset = 160, size = 32}, {name = 0x55fbe4c341d8 "esi", offset = 192, size = 32}, {
    name = 0x55fbe4c34268 "edi", offset = 224, size = 32}, {name = 0x55fbe4c342f8 "eip", offset = 256, size = 32}, {name = 0x55fbe4c34418 "eflags", offset = 288, size = 32}, {name = 0x55fbe4c344a8 "cs", offset = 320, size = 32}, {
    name = 0x55fbe4c34538 "ss", offset = 352, size = 32}, {name = 0x55fbe4c345c8 "ds", offset = 384, size = 32}, {name = 0x55fbe4c34658 "es", offset = 416, size = 32}, {name = 0x55fbe4c346e8 "fs", offset = 448, size = 32}, {
    name = 0x55fbe4c34778 "gs", offset = 480, size = 32}, {name = 0x55fbe4c34808 "st0", offset = 512, size = 80}, {name = 0x55fbe4c34388 "st1", offset = 592, size = 80}, {name = 0x55fbe4c34898 "st2", offset = 672, size = 80}, {
    name = 0x55fbe4c34928 "st3", offset = 752, size = 80}, {name = 0x55fbe4c349b8 "st4", offset = 832, size = 80}, {name = 0x55fbe4c34a48 "st5", offset = 912, size = 80}, {name = 0x55fbe4c34ad8 "st6", offset = 992, size = 80}, {
    name = 0x55fbe4c34b68 "st7", offset = 1072, size = 80}, {name = 0x55fbe4c34bf8 "fctrl", offset = 1152, size = 32}, {name = 0x55fbe4c34c88 "fstat", offset = 1184, size = 32}, {name = 0x55fbe4c34d18 "ftag", offset = 1216, size = 32}, {
    name = 0x55fbe4c34da8 "fiseg", offset = 1248, size = 32}, {name = 0x55fbe4c34e38 "fioff", offset = 1280, size = 32}, {name = 0x55fbe4c34ec8 "foseg", offset = 1312, size = 32}, {name = 0x55fbe4c34f58 "fooff", offset = 1344, 
    size = 32}, {name = 0x55fbe4c34fe8 "fop", offset = 1376, size = 32}, {name = 0x55fbe4c354d8 "xmm0", offset = 1408, size = 128}, {name = 0x55fbe4c35568 "xmm1", offset = 1536, size = 128}, {name = 0x55fbe4c355f8 "xmm2", offset = 1664, 
    size = 128}, {name = 0x55fbe4c35688 "xmm3", offset = 1792, size = 128}, {name = 0x55fbe4c35718 "xmm4", offset = 1920, size = 128}, {name = 0x55fbe4c357f8 "xmm5", offset = 2048, size = 128}, {name = 0x55fbe4c35888 "xmm6", 
    offset = 2176, size = 128}, {name = 0x55fbe4c35918 "xmm7", offset = 2304, size = 128}, {name = 0x55fbe4c359a8 "mxcsr", offset = 2432, size = 32}, {name = 0x55fbe4c35b28 "orig_eax", offset = 2464, size = 32}, {
    name = 0x55fbe4c35c38 "ymm0h", offset = 2496, size = 128}, {name = 0x55fbe4c35cc8 "ymm1h", offset = 2624, size = 128}, {name = 0x55fbe4c35d78 "ymm2h", offset = 2752, size = 128}, {name = 0x55fbe4c35e38 "ymm3h", offset = 2880, 
    size = 128}, {name = 0x55fbe4c35ec8 "ymm4h", offset = 3008, size = 128}, {name = 0x55fbe4c35f58 "ymm5h", offset = 3136, size = 128}, {name = 0x55fbe4c35fe8 "ymm6h", offset = 3264, size = 128}, {name = 0x55fbe4c36078 "ymm7h", 
    offset = 3392, size = 128}, {name = 0x55fbe4c36208 "k0", offset = 3520, size = 64}, {name = 0x55fbe4c362b8 "k1", offset = 3584, size = 64}, {name = 0x55fbe4c36368 "k2", offset = 3648, size = 64}, {name = 0x55fbe4c363f8 "k3", 
    offset = 3712, size = 64}, {name = 0x55fbe4c36488 "k4", offset = 3776, size = 64}, {name = 0x55fbe4c36568 "k5", offset = 3840, size = 64}, {name = 0x55fbe4c365f8 "k6", offset = 3904, size = 64}, {name = 0x55fbe4c36688 "k7", 
    offset = 3968, size = 64}, {name = 0x55fbe4c36718 "zmm0h", offset = 4032, size = 256}, {name = 0x55fbe4c36838 "zmm1h", offset = 4288, size = 256}, {name = 0x55fbe4c368c8 "zmm2h", offset = 4544, size = 256}, {
    name = 0x55fbe4c36958 "zmm3h", offset = 4800, size = 256}, {name = 0x55fbe4c369e8 "zmm4h", offset = 5056, size = 256}, {name = 0x55fbe4c36a78 "zmm5h", offset = 5312, size = 256}, {name = 0x55fbe4c36b08 "zmm6h", offset = 5568, 
    size = 256}, {name = 0x55fbe4c36b98 "zmm7h", offset = 5824, size = 256}, {name = 0x55fbe4c36c88 "pkru", offset = 6080, size = 32}}

(gdb) f 7
#7  0x000055fbe3351b1d in i387_xsave_to_cache (regcache=0x55fbe4c331c0, buf=0x55fbe4c382b0) at ../../../gdb/gdbserver/i387-fp.c:836
836		    supply_register_zeroed (regcache, i + zmm0h_regnum);
(gdb) p zmm0h_regnum 
$2 = 58

(gdb) list
831	      int zmm0h_regnum = find_regno (regcache->tdesc, "zmm0h");
832	
833	      if ((clear_bv & X86_XSTATE_ZMM_H) != 0)
834		{
835		  for (i = 0; i < num_avx512_zmmh_low_registers; i++)
836		    supply_register_zeroed (regcache, i + zmm0h_regnum);
837		}
838	      else
839		{
840		  p = (gdb_byte *) &fp->zmmh_low_space[0];
(gdb) p num_avx512_zmmh_low_registers
$3 = 16
(gdb) p i
$4 = 9

Steps to reproduce:

$ cat hello.c
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
        printf("Start\n");

        printf("Hello World\n");

        printf("End\n");
        return EXIT_SUCCESS;
}

$ gcc -m32 -g  hello.c -o hello

$ gdbserver :2345 hello

<< on another terminal >>

$ gdb
...
(gdb) target remote :2345


and the gdbserver fails with:

/usr/include/c++/8/bits/stl_vector.h:950: std::vector<_Tp, _Alloc>::const_reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) const [with _Tp = reg; _Alloc = std::allocator<reg>; std::vector<_Tp, _Alloc>::const_reference = const reg&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.

Please note that I could not reproduce the problem (no easy access to an
avx512* test system), but from the coredump it appears the issue is that
a too small buffer has been allocated, likely some issue with 32 bit binaries
and avx512 extensions.

System where problem happens has:

gdb-8.2-12.el8.x86_64
glibc-2.28-127.el8_3.2.x86_64
glibc-2.28-127.el8_3.2.i686
gcc-8.3.1-5.1.el8.x86_64

--- Additional comment from Keith Seitz on 2021-10-10 16:11:57 UTC ---

I have replicated the issue and am investigating a fix. Upstream does not contain this specific bug,
but gdbserver still attempts to query AVX512 registers.

--- Additional comment from Keith Seitz on 2021-10-29 18:09:03 UTC ---

I have a tentative patch to fix the problem. However, I run
into additional problems with RHEL8 gdbserver. I do not think
we will be able to target 8.6 for the system gdbserver.

However, the patch is sufficient to fix GTS 11. Is that
an option? [One need only install gcc-toolset-11-gdb-gdbserver
if one wants to continue to use the 3+ year old system gdb.]
We have a good chance of including this in GTS 11.1 release.

--- Additional comment from Paulo Andrade on 2021-10-29 19:42:14 UTC ---

This should be a very uncommon issue.

If GTS 11 resolves the issue, we can create a document describing
the condition and the workaround, of using GTS 11.

Since it is an issue only for developers, it should be OK to install
the newer gdb to debug 32 bit binaries.

This only happens with recent cpus, with extra avx512* capabilities.

--- Additional comment from Keith Seitz on 2021-10-29 20:19:57 UTC ---

To be clear: No version of gdbserver works, not even upstream.
I have a patch that I've tested everywhere, and it does not
work on our RHEL8 system gdbserver. It *does* work for GTS11,
though.

If we include a fix in GTS 11.1, users would need to install
gcc-toolset-11-gdb-gdbserver. It's the server that's broken,
not GDB itself. [GDB can handle native debugging in this
case just fine.]

If I'm reading you correctly, we should pursue getting GTS 11.1
the needed fix, and unless I hear otherwise, that's what I will do.

Thanks!

--- Additional comment from Paulo Andrade on 2021-10-29 20:42:09 UTC ---

Thanks for the clarification. I was a bit confused with comment #1
and understood upstream did not segfault, but still attempted to
read the registers.

Since it is a developer tool only, and is easy to access and install,
it should be completely fine to use the workaround.

--- Additional comment from Keith Seitz on 2021-11-03 16:22:29 UTC ---

Recategorizing to GTS 11 for inclusion in 11.1