Bug 1281849 - asan-instrumented qemu running order of magnitude slower than qemu without address sanitizer
asan-instrumented qemu running order of magnitude slower than qemu without ad...
Status: NEW
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: gcc (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Marek Polacek
Depends On:
Blocks: 1413146
  Show dependency treegraph
Reported: 2015-11-13 10:25 EST by David Jaša
Modified: 2017-07-18 14:33 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description David Jaša 2015-11-13 10:25:13 EST
Description of problem:
asan-instrumented qemu running order of magnitude slower than uninstrumented qemu

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. rebuild qemu with asan (aka with -fsanitize=address )
2. run a real VM in qemu instrumented with address sanitizer

Actual results:
qemu feels even slower than qemu under valgrind/memcheck.

Expected results:
qemu runs at ~1/2 the usual speed as asan-instrumented programs should

Additional info:
1. how to rebuild qemu (rpm):
    1. download & install srpm, insert these lines to start of the %build stage:
%global optflags %(echo %{optflags} | sed 's/ -g / -ggdb -fno-omit-frame-pointer -fsanitize=address -Wno-error=maybe-uninitialized -Wno-unused-but-set-variable /')
%global __global_ldflags %(echo "%{__global_ldflags} -fsanitize=address -lasan")
%global __strip %{nil}
%global debug_package %{nil}
    2. rebuild using rpmbuild -ba ~rpmbuild/SPECS/qemu-kvm.spec. The build will
        fail at "make check" but usable build will be in BUILDROOT anyway

2. how to run qemu
    1. always run qemu within gdb: gdb ~/rpmbuild/.../usr/libexec/qemu-kvm
        otherwise you'll run into OOMs
    2. always use -monitor stdio (or elsewhere) because qemu under gdb requires
        "c" command to continue booting
    3. use virtio-scsi block devices, virtio-blk seems incompatible with asan
Comment 2 David Jaša 2015-11-13 10:37:47 EST
Example qemu command line for such a real VM (adopted from libvirt):

ASAN_OPTIONS=symbolize=1:verbosity=10:fast_unwind_on_fatal=1:fast_unwind_on_malloc=1:abort_on_error=1 \
ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer LC_ALL=C G_SLICE=always-malloc \
perf record -a --call-graph fp -s   -o tmp/qemu_with_asan.raw -- gdb \
rpmbuild/BUILDROOT/qemu-kvm-rhev-2.3.0-31.el7_2.1.sanitizer.x86_64-gcc/usr/libexec/qemu-kvm \
--ex "handle SIGPIPE nostop noprint pass" --ex "-name win7u -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -cpu Nehalem -m 1536 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid UUID -no-user-config -nodefaults -monitor stdio -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on  -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/PATH/TO/IMAGE,if=none,id=drive-scsi0-0-0-0,format=FORMAT,discard=unmap -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1  -netdev tap,ifname=TAP_DEVICE_NAME,script=no,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=MAC_ADDRESS,bus=pci.0,addr=0x3 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -spice port=5800,addr=::,disable-ticketing,seamless-migration=on -vga qxl -global qxl-vga.ram_size=134217728 -global qxl-vga.vram_size=33554432 -global qxl-vga.vgamem_mb=32 -device qxl,id=video1,bus=pci.0,addr=0x9 -device qxl,id=video2,bus=pci.0,addr=0xa -device qxl,id=video3,bus=pci.0,addr=0xb -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -msg timestamp=on"

to pre-create tap device, run as root:
ip tuntap add dev TAP_DEVICE_NAME mode tap user USER group GROUP
ip l set dev TAP_DEVICE_NAME up
ip l set dev TAP_DEVICE_NAME master virbr0    // or br0

Note You need to log in before you can comment on or make changes to this bug.