Bug 1281455

Summary: use-after-free at hard VM reset (virsh reset/system_reset)
Product: Red Hat Enterprise Linux 7 Reporter: David Jaša <djasa>
Component: spiceAssignee: Victor Toso <victortoso>
Status: CLOSED ERRATA QA Contact: SPICE QE bug list <spice-qe-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: cfergeau, rduda, sraje, tpelka, victortoso
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: spice-0.12.4-17.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 03:44:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
symbolized output, log snippet & full backtrace none

Description David Jaša 2015-11-12 14:57:51 UTC
Created attachment 1093345 [details]
symbolized output, log snippet & full backtrace

Description of problem:
When running address sanitizer-instrumented qemu and spice-server, I encountered use-after free during hard system reset (triggered by qemu command system_reset)

Version-Release number of selected component (if applicable):
spice-server-0.12.4-15.el7.sanitizer.x86_64
qemu-kvm-rhev 

How reproducible:
didn't try to reproduce

Steps to Reproduce:
1. I rebuilt qemu-kvm and spice-server with address sanitizer enabled
2. I got BSOD in windows VM, so I used qemu's system_reset to get back faster
3. right after VM reset, ASAN detected use-after-free

Actual results:
==10014== ERROR: AddressSanitizer: heap-use-after-free on address 0x600c002c2830 at pc 0x7fffee3887bc bp 0x7fffffffccd0 sp 0x7fffffffccc0
[Thread 0x7fff7acf9700 (LWP 10051) exited]
READ of size 4 at 0x600c002c2830 thread T0
[Thread 0x7fff5d118700 (LWP 10841) exited]
[Thread 0x7fff5fe45700 (LWP 10286) exited]
[Thread 0x7fff5e726700 (LWP 10839) exited]
    #0 0x7fffee3887bb in spice_char_device_client_free /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/char_device.c:166
    #2 0x7fffee38bdfb in spice_char_device_state_destroy /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/char_device.c:712
    #4 0x7fffee43cb09 in reds_reset_vdp /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/reds.c:512
    #5 0x7fffee43cb09 in reds_agent_remove /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/reds.c:683
...
0x600c002c2830 is located 16 bytes inside of 56-byte region [0x600c002c2820,0x600c002c2858)
freed by thread T0 here:
    #2 0x7fffee38bf74 in spice_char_device_write_buffer_free /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/char_device.c:693
    #3 0x7fffee38bf74 in spice_char_device_state_destroy /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/char_device.c:704
    #5 0x7fffee43cb09 in reds_reset_vdp /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/reds.c:512
    #6 0x7fffee43cb09 in reds_agent_remove /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/reds.c:683
...
previously allocated by thread T0 here:
    #2 0x7fffee4bc99f in spice_malloc0 /home/djasa/rpmbuild/BUILD/spice-0.12.4/spice-common/common/mem.c:109
    #4 0x7fffee3875f1 in __spice_char_device_write_buffer_get /home/djasa/rpmbuild/BUILD/spice-0.12.4/server/char_device.c:510
...

Expected results:
no use-after-free occurs

Additional info:
virtio-scsi has to be used for block device because virtio-blk uses coroutines that are incompatible with address sanitizer

Comment 1 Victor Toso 2015-11-13 10:18:48 UTC
Sent some patches which should fix it:
http://lists.freedesktop.org/archives/spice-devel/2015-November/023565.html

Comment 2 David Jaša 2015-11-13 10:33:46 UTC
I wasn't able to reproduce at subsequent retries. The fact that system boot takes 20-30 minutes with asan-enabled qemu isn't exactly helpful either.

Comment 4 Victor Toso 2015-11-13 11:13:12 UTC
Patches pushed upstream related to this:

c429574bb6aebcbbddcc9714f994afd6b0ae7186
08230ee1715ff07bb0bdc8e6661b53a6752854d5

The following patch seems necessary as well:

d7bee1bc56e2d3ea6af399ba8479cb4b849d4b15

Comment 5 Mike McCune 2016-03-28 23:43:17 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 6 Victor Toso 2016-05-10 07:00:10 UTC
*** Bug 1330394 has been marked as a duplicate of this bug. ***

Comment 10 errata-xmlrpc 2016-11-04 03:44:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2324.html