Bug 1474074 - [abrt] virt-viewer: g_socket_create_source(): virt-viewer killed by signal 11
Summary: [abrt] virt-viewer: g_socket_create_source(): virt-viewer killed by signal 11
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: spice-gtk
Version: 26
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Victor Toso
QA Contact: Fedora Extras Quality Assurance
URL: https://retrace.fedoraproject.org/faf...
Whiteboard: abrt_hash:7ce0ef06702733db22b729bd7a4...
: 1474627 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-23 15:41 UTC by Kyle Marek
Modified: 2018-05-29 11:59 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-05-29 11:59:38 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (32.74 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: cgroup (289 bytes, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: core_backtrace (12.11 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: cpuinfo (1.25 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: dso_list (19.43 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: environ (3.18 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: exploitable (82 bytes, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: limits (1.29 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: maps (91.23 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: open_fds (2.60 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: proc_pid_status (1.28 KB, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: var_log_messages (337 bytes, text/plain)
2017-07-23 15:41 UTC, Kyle Marek
no flags Details
File: virt-viewer-step-3-2.log (44.42 KB, text/plain)
2017-07-25 06:58 UTC, Kyle Marek
no flags Details
File: remote-viewer-step-3-2.log (37.41 KB, text/plain)
2017-07-25 06:58 UTC, Kyle Marek
no flags Details

Description Kyle Marek 2017-07-23 15:41:23 UTC
Description of problem:
1. Attempted to access the WebDAV mount in Windows Server 2012 R2
2. Realized I didn't share the directory yet
3. Shared the directory
4. Canceled the first access attempt in Windows
5. Tried to access WebDAV mount again
6. Segmentation fault occurs

Version-Release number of selected component:
virt-viewer-5.0-2.fc26

Additional info:
reporter:       libreport-2.9.1
backtrace_rating: 4
cmdline:        virt-viewer --connect qemu+ssh://server00.internal.gigabyteproductions.net/system win2k12r2
crash_function: g_socket_create_source
executable:     /usr/bin/virt-viewer
journald_cursor: s=728fb909ca664d05984379a008a0811f;i=97db;b=7e849ccf32774c9f82d32a0222be5bb4;m=19a9a9f12b;t=554fdc7d4bed7;x=be236fbd2b46e97c
kernel:         4.11.10-300.fc26.x86_64
rootdir:        /
runlevel:       N 5
type:           CCpp
uid:            1000

Truncated backtrace:
Thread no. 1 (9 frames)
 #0 g_socket_create_source at gsocket.c:3700
 #1 g_coroutine_socket_wait at gio-coroutine.c:59
 #2 spice_channel_iterate_read at spice-channel.c:2246
 #3 spice_channel_iterate at spice-channel.c:2291
 #4 spice_channel_coroutine at spice-channel.c:2583
 #5 coroutine_trampoline at coroutine_ucontext.c:63
 #6 continuation_trampoline at continuation.c:55
 #7 ??
 #8 ??

Comment 1 Kyle Marek 2017-07-23 15:41:31 UTC
Created attachment 1303251 [details]
File: backtrace

Comment 2 Kyle Marek 2017-07-23 15:41:32 UTC
Created attachment 1303252 [details]
File: cgroup

Comment 3 Kyle Marek 2017-07-23 15:41:33 UTC
Created attachment 1303253 [details]
File: core_backtrace

Comment 4 Kyle Marek 2017-07-23 15:41:34 UTC
Created attachment 1303254 [details]
File: cpuinfo

Comment 5 Kyle Marek 2017-07-23 15:41:36 UTC
Created attachment 1303255 [details]
File: dso_list

Comment 6 Kyle Marek 2017-07-23 15:41:37 UTC
Created attachment 1303256 [details]
File: environ

Comment 7 Kyle Marek 2017-07-23 15:41:38 UTC
Created attachment 1303257 [details]
File: exploitable

Comment 8 Kyle Marek 2017-07-23 15:41:40 UTC
Created attachment 1303258 [details]
File: limits

Comment 9 Kyle Marek 2017-07-23 15:41:42 UTC
Created attachment 1303259 [details]
File: maps

Comment 10 Kyle Marek 2017-07-23 15:41:43 UTC
Created attachment 1303260 [details]
File: open_fds

Comment 11 Kyle Marek 2017-07-23 15:41:44 UTC
Created attachment 1303261 [details]
File: proc_pid_status

Comment 12 Kyle Marek 2017-07-23 15:41:45 UTC
Created attachment 1303262 [details]
File: var_log_messages

Comment 13 Victor Toso 2017-07-24 12:21:33 UTC
(In reply to Kyle Marek from comment #0)
> Description of problem:
> 1. Attempted to access the WebDAV mount in Windows Server 2012 R2
> 2. Realized I didn't share the directory yet

The mount point is present in this situation?

> 3. Shared the directory
> 4. Canceled the first access attempt in Windows

Any visible error here?

> 5. Tried to access WebDAV mount again
> 6. Segmentation fault occurs

Can you reliable reproduce this? I don't have windows server to test it. I find it a bug the fact that you already have a mount point without sharing client's folder.

It would be great if you could provide the logs from virt-viewer while running from terminal with --debug --spice-debug.

Extra points if you provide which version of spice-webdavd you are using ;)

Thanks for the bug report!

Comment 14 Victor Toso 2017-07-24 12:22:38 UTC
PS: I tried several times on Fedora 25 guest. Mounting/Umounting, toggling on/off the 'share folder' option while copying files, etc.

Comment 15 Kyle Marek 2017-07-25 05:11:36 UTC
*** Bug 1474627 has been marked as a duplicate of this bug. ***

Comment 16 Kyle Marek 2017-07-25 06:19:40 UTC
Information about my test environment:

# Windows Server 2012 R2 Evaluation ISO
http://care.dlservice.microsoft.com/dl/download/6/2/A/62A76ABB-9990-4EFC-A4FE-C7D698DAEB96/9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_EN-US-IR3_SSS_X64FREE_EN-US_DV9.ISO

MD5: 5b5e08c490ad16b59b1d9fab0def883a
SHA1: 849734f37346385dac2c101e4aacba4626bb141c
SHA256: 6612b5b1f53e845aacdf96e974bb119a3d9b4dcb5b82e65804ab7e534dc7b4d5
SHA512: c16ca71c0a0afb8a884bb7b14f2b5784dfa337a9ccc750eeaf96451dd6167d6979014e3f993701a6df4b8a2d419b1b1a50cb8be3e21e2888e828bf6d9d365428


# SPICE guest tools
https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-latest.exe currently: https://www.spice-space.org/download/windows/spice-guest-tools/spice-guest-tools-0.132/spice-guest-tools-0.132.exe

MD5: 20e10d38b5cf512c2e8c67b75c6046e5
SHA1: 6a85ad37548c8b65db14a0b57787fc773ba0c16c
SHA256: 5e2a86615fc7af3628b6866f4770524e93e39fb0e9f95196d15548927ce83cb2
SHA512: dfd94af10cf2c8afc9344f69c87f6ec5141f82058a7a20d0d4a873d3693c8cda29497e058ab2f54294be571d8e7f1d0a7b1d779710eb9c601b0607228208b69f


# SPICE WebDAV daemon
https://www.spice-space.org/download/windows/spice-webdavd/spice-webdavd-x64-latest.msi currently: https://www.spice-space.org/download/windows/spice-webdavd/spice-webdavd-x64-2.2.msi

MD5: 6570bbf5f455991d64d05451b3e7af4b
SHA1: f5311f2b3b7b347218bd23ce6e8d12964aa28053
SHA256: 45baa7bce5e135172f9fad7aac011adb711f59902b7c53b9aafadbceb0d202bd
SHA512: 8a5c2550df89d9023faefbed4711d53df0db4320b9772c78ec471baaa57a05959e062cd1f5e04daceb51fcfe58e320e4349171f9c1a21371de470ef326de182f


kmarek.gigabyteproductions.net ~
$ rpm --query --whatprovides "$(which qemu-system-x86_64)" "$(which remote-viewer)" "$(which smbd)"
qemu-system-x86-core-2.9.0-1.fc26.1.x86_64
virt-viewer-5.0-2.fc26.x86_64
samba-4.6.5-0.fc26.x86_64

kmarek.gigabyteproductions.net ~
$ cat spice.webdav.bugtest.qemu.sh
#!/bin/sh
cd "$(dirname "$0")"
qemu-system-x86_64 \
    -nodefaults \
    -nodefconfig \
    -no-user-config \
    -machine q35,accel=kvm,usb=off \
    -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 \
    -no-hpet \
    -cpu qemu64 -smp cores=$(nproc) \
    -m 1G \
    -rtc base=localtime,driftfix=slew \
    -device rtl8139,netdev=eth0 -netdev user,id=eth0,smb="$HOME/win" \
    -device qxl-vga \
    -device nec-usb-xhci,id=xhci \
    -drive media=cdrom,file="$HOME/9600.17050.WINBLUE_REFRESH.140317-1640_X64FRE_SERVER_EVAL_EN-US-IR3_SSS_X64FREE_EN-US_DV9.ISO",format=raw \
    -drive file="spice.webdav.bugtest.qed",format=qed \
    -device usb-tablet,bus=xhci.0 \
    -monitor stdio \
    -display none -spice unix,addr="spice.webdav.spice.sock",disable-ticketing \
    -device virtio-serial-pci -device virtserialport,chardev=spicechannel0,name=com.redhat.spice.0 -chardev spicevmc,id=spicechannel0,name=vdagent \
    -device virtio-serial-pci,id=virtio-serial0 -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel1,id=channel1,name=org.spice-space.webdav.0 -chardev spiceport,name=org.spice-space.webdav.0,id=charchannel1


Steps to replicate my environment:
1. run once: qemu-img create -f qed spice.webdav.bugtest.qed 32G
2. let run: ./spice.webdav.bugtest.qemu.sh
3. let run: remote-viewer "spice+unix:///.$(realpath "spice.webdav.spice.sock")"
   Note: Unix sockets have a path length limit.
     See: https://stackoverflow.com/questions/34829600/why-is-the-maximal-path-length-allowed-for-unix-sockets-on-linux-108
     See: https://web.archive.org/web/20170725060522/https://stackoverflow.com/questions/34829600/why-is-the-maximal-path-length-allowed-for-unix-sockets-on-linux-108
     See: man 7 unix
4. in guest: install Windows Server 2012 R2 Standard Evaluation (Server with a GUI)
   Note: because of smb= in qemu.sh, files can be shared from host by running in the guest:
   net use * \\10.0.2.4\qemu
5. in guest: install spice-guest-tools-latest.exe
6. in guest: install spice-webdav-x64-latest.msi
7. Make Windows able to mount WebDAV
   Note: See: https://superuser.com/questions/505191/mounting-webdav-drive-in-windows-server-2012/869730#869730
   Note: See: https://web.archive.org/web/20170725044554/https://superuser.com/questions/505191/mounting-webdav-drive-in-windows-server-2012/869730#869730
7.1. run in guest: powershell -command "Add-WindowsFeature Desktop-Experience"
7.2. run in guest: shutdown /r /t 0

Steps to test:
1. Freshen everything
1.1. run in guest: net use y: /delete
1.2. run in guest: net stop spice-webdavd
1.3. run in guest: net stop webclient
1.4. close viewer
1.5. open viewer
2. Create functional mount
2.1. Share a directory in preferences of remote-viewer
2.2. run in guest: net start spice-webdavd
2.3. run in guest: net use y: http://127.0.0.1:9843
2.4. run in guest: explorer y:
2.5. in guest: close y: window
3. Break mount
3.1. Quit viewer
3.2. Start viewer (but don't re-enable sharing)
3.3. run in guest: explorer y:
3.4. in guest: Receive "Y:\ is not accessible." error
3.5. Share directory
3.6. run in guest: explorer y:
3.7. remote-viewer crashes immediately.


Understanding that virt-viewer and remote-viewer share a code base, and in the interest of eliminating libvirt as a factor, I tested against remote-viewer. I was able to apply this process to a libvirt-managed QEMU virtual machine running on a Fedora 25 host and was able to reproduce the issue with virt-viewer from the same virt-viewer-5.0-2.fc26.x86_64 as well.

Comment 17 Kyle Marek 2017-07-25 06:56:57 UTC
To clarify comment #16's test:

(In reply to Victor Toso from comment #13)
> (In reply to Kyle Marek from comment #0)
> > Description of problem:
> > 1. Attempted to access the WebDAV mount in Windows Server 2012 R2
> > 2. Realized I didn't share the directory yet
> 
> The mount point is present in this situation?

Yes. Mount was operational until I had kicked myself off of virt-viewer while accessing the same VM in virt-manager.
 
> > 3. Shared the directory
> > 4. Canceled the first access attempt in Windows
> 
> Any visible error here?

At the time: no explicit error. The "canceling" was my stopping of the pretending-to-progress bar that indicated it was still trying to connect.

> > 5. Tried to access WebDAV mount again
> > 6. Segmentation fault occurs
> 
> Can you reliable reproduce this? I don't have windows server to test it. I
> find it a bug the fact that you already have a mount point without sharing
> client's folder.
> 

Ability to reproduce is hit-and-miss depending on things like if a significant amount of memory pages operating the guest has been swapped out.

Mount point exists because of operations previous to the *re-opening* of the SPICE client. However, it is still a bug that a guest has any influence over a SPICE client to jump to memory addresses.

> It would be great if you could provide the logs from virt-viewer while
> running from terminal with --debug --spice-debug.
> 

Attaching shortly.

> Extra points if you provide which version of spice-webdavd you are using ;)
> 

spice-webdavd-x64-2.2.msi. Hashes in comment #16.

Comment 18 Kyle Marek 2017-07-25 06:58:03 UTC
Created attachment 1304039 [details]
File: virt-viewer-step-3-2.log

Comment 19 Kyle Marek 2017-07-25 06:58:45 UTC
Created attachment 1304040 [details]
File: remote-viewer-step-3-2.log

Comment 20 Victor Toso 2017-07-25 14:16:15 UTC
Reproduced a few times the crash but with different backtrace. It could be different bug or not. I plan to investigate further.

Thread 1 "remote-viewer" received signal SIGSEGV, Segmentation fault.
0x00007fffefdf6a14 in __memmove_avx_unaligned_erms () from /lib64/libc.so.6
(gdb) bt full
#0  0x00007fffefdf6a14 in __memmove_avx_unaligned_erms () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff4e46fc2 in spice_channel_read_sasl (channel=0xca8130, data=0xfc2050, len=6) at spice-channel.c:1122
        c = 0xca7740
        __func__ = "spice_channel_read_sasl"
#2  0x00007ffff4e470c1 in spice_channel_read (channel=0xca8130, data=0xfc2050, length=6) at spice-channel.c:1149
        c = 0xca7740
        len = 6
        ret = 0
        __func__ = "spice_channel_read"
#3  0x00007ffff4e49f45 in spice_channel_recv_msg (channel=0xca8130, msg_handler=0x7ffff4e52e3c <spice_webdav_handle_msg>, data=0x0) at spice-channel.c:2031
        c = 0xca7740
        in = 0xfc2040
        msg_size = 161
        msg_type = 101
        sub_list_offset = 0
#4  0x00007ffff4e4aaa2 in spice_channel_iterate_read (channel=0xca8130) at spice-channel.c:2338
        c = 0xca7740
#5  0x00007ffff4e4acb5 in spice_channel_iterate (channel=0xca8130) at spice-channel.c:2376
        c = 0xca7740
#6  0x00007ffff4e4ba6b in spice_channel_coroutine (data=0xca8130) at spice-channel.c:2664
        channel = 0xca8130
        c = 0xca7740
        verify = 0
        rc = 0
        delay_val = 1
        ssl_options = 33554432
        __func__ = "spice_channel_coroutine"
#7  0x00007ffff4e92d49 in coroutine_trampoline (cc=0xca77e0) at coroutine_ucontext.c:63
        co = 0xca77b0
#8  0x00007ffff4e92a01 in continuation_trampoline (i0=13268960, i1=0) at continuation.c:55
        arg = {p = 0xca77e0, i = {13268960, 0}}
        cc = 0xca77e0
#9  0x00007fffefce7600 in ?? () from /lib64/libc.so.6

Comment 21 Kyle Marek 2017-07-25 19:07:24 UTC
I noticed a similar occurrence when testing against remote-viewer instead of virt-viewer. This bug's backtrace is regarding virt-viewer, and the one I filed with ABRT on remote-viewer is https://bugzilla.redhat.com/show_bug.cgi?id=1474627

Comment 22 Victor Toso 2017-07-28 14:31:37 UTC
Just a small update. This can also happen on linux (Fedora 25/GNOME) but the automount there works better and when you don't have the shared folder enabled in remote-viewer chances are high that the mount point will not exist. Still, for fedora I have a fix.

The same fix is not working well in windows guest, still need more debugging.

Comment 23 Victor Toso 2017-08-01 13:03:19 UTC
Sent two patches that should avoid the crash in the client:
https://lists.freedesktop.org/archives/spice-devel/2017-August/039109.html

Note that the reason for this to happen is related to some poor behavior in spice-webdavd for windows and the guest itself, you can compare with Fedora/GNOME.

Very likely the shared folder feature will be broken if you try to access a mount point while the client has this option disabled - on windows. On linux with patch mentioned above, everything seems okay after you enable shared folder again.

Thanks again for testing and reporting this issue.

Comment 24 Kyle Marek 2017-08-01 13:20:59 UTC
Out of curiosity, with these patches applied, are SPICE clients susceptible to making jumps to arbitrary memory addresses by means of maliciously malformed channel data, or was the jump due to uninitialized data that is now guaranteed to be initialized?

Comment 25 Victor Toso 2017-08-02 09:56:21 UTC
(In reply to Kyle Marek from comment #24)
> Out of curiosity, with these patches applied, are SPICE clients susceptible
> to making jumps to arbitrary memory addresses by means of maliciously
> malformed channel data, or was the jump due to uninitialized data that is
> now guaranteed to be initialized?

That should not happen, hopefully. This piece of code uses an internal protocol (not in spice-protocol) to exchange data messages between SPICE client and the webdav daemon so the client will try to get id, data-size, payload from this stream of bytes. If the stream of bytes is bad, we will not have a valid WebDAV payload in the end and that should make the webdav server in the client to issue warnings/fail or close the connection.

We might need extra checks and possibly dropping data somewhere before spice-gtk receives it... But in the end spice-gtk should just parse/demux.

Comment 26 Fedora End Of Life 2018-05-03 08:17:09 UTC
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 27 Fedora End Of Life 2018-05-29 11:59:38 UTC
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.