Ashish, can you attach the output from 'xdpyinfo' with this server? My initial suspicion (that missing MIT-SHM was the problem) doesn't seem to be the case, and I'm curious what else is different about this configuration.

(In reply to Adam Jackson from comment #1)
> Ashish, can you attach the output from 'xdpyinfo' with this server? My
> initial suspicion (that missing MIT-SHM was the problem) doesn't seem to be
> the case, and I'm curious what else is different about this configuration.

Ping.

(In reply to Adam Jackson from comment #2)
> (In reply to Adam Jackson from comment #1)
> > Ashish, can you attach the output from 'xdpyinfo' with this server? My
> > initial suspicion (that missing MIT-SHM was the problem) doesn't seem to be
> > the case, and I'm curious what else is different about this configuration.
> 
> Ping.

Monthly-ish ping.

(In reply to Adam Jackson from comment #4)
> (In reply to Adam Jackson from comment #2)
> > (In reply to Adam Jackson from comment #1)
> > > Ashish, can you attach the output from 'xdpyinfo' with this server? My
> > > initial suspicion (that missing MIT-SHM was the problem) doesn't seem to be
> > > the case, and I'm curious what else is different about this configuration.
> > 
> > Ping.
> 
> Monthly-ish ping.

Ping yet again (also in direct email).

(EE) Backtrace:
(EE) 0: Xephyr (xorg_backtrace+0x55) [0x5b1ea5]
(EE) 1: Xephyr (0x400000+0x1b5f09) [0x5b5f09]
(EE) 2: /usr/lib64/libpthread.so.0 (0x7f1290848000+0xf370) [0x7f1290857370]
(EE) 3: /usr/lib64/libxcb-image.so.0 (xcb_image_create_native+0x38) [0x7f128fa28748]
(EE) 4: Xephyr (0x400000+0x2da9d) [0x42da9d]
(EE) 5: Xephyr (0x400000+0x15bb5e) [0x55bb5e]
(EE) 6: Xephyr (0x400000+0x79f05) [0x479f05]
(EE) 7: Xephyr (EnableDevice+0x1f0) [0x4a8950]
(EE) 8: Xephyr (0x400000+0xa9895) [0x4a9895]
(EE) 9: Xephyr (0x400000+0x767ab) [0x4767ab]
(EE) 10: /usr/lib64/libc.so.6 (__libc_start_main+0xf5) [0x7f128d58bb35]
(EE) 11: Xephyr (0x400000+0x2a159) [0x42a159]
(EE)
(EE) Segmentation fault at address 0x21

We're trying to create a "hardware" cursor here (ie, letting moba's X server draw the cursor image instead of Xephyr painting it into its own framebuffer). The crash is in xcb_image_create_native, in particular:

  case XCB_IMAGE_FORMAT_XY_BITMAP:
      if (depth != 1)
          return 0;
      /* fall through */
  case XCB_IMAGE_FORMAT_XY_PIXMAP:
      if (depth > 1) {
          fmt = find_format_by_depth(setup, depth);
          if (!fmt)
              return 0;
      }
      return xcb_image_create(width, height, format,
                              setup->bitmap_format_scanline_pad,
                              depth, depth, setup->bitmap_format_scanline_unit,
                              setup->image_byte_order,
                              setup->bitmap_format_bit_order,
                              base, bytes, data);

Note that ->bitmap_format_scanline_pad is 0x21 bytes from the start of the connection setup block. So I assume the connection has died for some reason and we're just not noticing until this point. Hmm.

Got access to a test environment for this. Upstream's Xephyr works fine, so I suspect this may already be fixed in 7.4....

Bisecting the upstream X server, this is the first "good" commit:

commit d51cce7992508f366758fa13dd93be24df0947ed
Author: Adam Jackson <ajax>
Date:   Tue Jul 5 13:07:09 2016 -0400

    xephyr: Don't crash if the server advertises zero xv adaptors

After this point Xephyr appears to work just fine with mobaxterm, and in particular I am unable to reproduce the crash from comment #7. This was tested with Fedora 25 not RHEL7.4; it's possible that some library in 7.3 (probably libxcb) had an additional bug that causes that crash.

If testing with a 7.4 chroot also fails to reproduce this, I think we can safely call this one fixed.

(In reply to Adam Jackson from comment #11)

> After this point Xephyr appears to work just fine with mobaxterm,

Ha ha no that's a lie. It still dies, and for essentially the same reason, just at a different point (xserver's mainloop was rewritten between 1.17 and 1.19, which affects when poll() happens and thus when errors are finally raised). The bug is we do not check for the XVideo extension at all before trying to make requests of it, which xcb treats as a connection error.

moving to VERIFIED:

used host:
rhel7.4
xorg-x11-server-1.19.3-7.el7.x86_64

client : win7
MobaXterm Home Edition v10.2

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1905