Bug 1493304

Summary: SIGSEGV in qemu-arm
Product: [Fedora] Fedora Reporter: John Reiser <jreiser>
Component: qemuAssignee: Fedora Virtualization Maintainers <virt-maint>
Status: CLOSED UPSTREAM QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 27CC: amit, berrange, cfergeau, crobinso, dwmw2, itamar, pbonzini, rjones, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-20 22:34:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
"./foo" 32-bit ARM executable none

Description John Reiser 2017-09-19 21:10:26 UTC
Description of problem: qemu-arm itself gets a SIGSEGV between the first emulated system call and the second, although the emulated program runs many system calls just fine natively on 32-bit ARM RaspberryPi 3B under Fedora 27.


Version-Release number of selected component (if applicable):
qemu-user-2.10.0-1.fc27.x86_64


How reproducible: every time


Steps to Reproduce:
1. qemu-arm ./foo  ## ./foo will be attached (1/4 MB); is standalone (no PT_INTERP)
2.
3.

Actual results:
$ qemu-arm ./foo
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)


Expected results:
no SIGSEGV

Additional info:
Running on x86_64:
$ gdb qemu-arm
GNU gdb (GDB) Fedora 8.0.1-26.fc27
(gdb) run ./foo
Starting program: /usr/bin/qemu-arm ./foo
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.26-8.fc27.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff591f700 (LWP 2965)]

Thread 1 "qemu-arm" received signal SIGSEGV, Segmentation fault.
0x0000555555993665 in static_code_gen_buffer ()
Missing separate debuginfos, use: dnf debuginfo-install glib2-2.54.0-1.fc27.x86_64 gperftools-libs-2.6.1-3.fc27.x86_64 libgcc-7.2.1-2.fc27.x86_64 libstdc++-7.2.1-2.fc27.x86_64 libunwind-1.2-3.fc27.x86_64 pcre-8.41-1.fc27.2.x86_64 zlib-1.2.11-4.fc27.x86_64
(gdb) bt
#0  0x0000555555993665 in static_code_gen_buffer ()
#1  0x00005555555f96ed in cpu_tb_exec (itb=<optimized out>, itb=<optimized out>, 
    cpu=<optimized out>) at /usr/src/debug/qemu-2.10.0-1.fc27.x86_64/accel/tcg/cpu-exec.c:166
#2  cpu_loop_exec_tb (tb_exit=<synthetic pointer>, last_tb=<synthetic pointer>, tb=<optimized out>, 
    cpu=<optimized out>) at /usr/src/debug/qemu-2.10.0-1.fc27.x86_64/accel/tcg/cpu-exec.c:578
#3  cpu_exec (cpu=<optimized out>)
    at /usr/src/debug/qemu-2.10.0-1.fc27.x86_64/accel/tcg/cpu-exec.c:676
#4  0x00005555555fc750 in cpu_loop (env=0x5555582b6290)
    at /usr/src/debug/qemu-2.10.0-1.fc27.x86_64/linux-user/main.c:570
#5  0x00005555555c8240 in main (argc=<optimized out>, argv=0x7fffffffdfa8, envp=<optimized out>)
    at /usr/src/debug/qemu-2.10.0-1.fc27.x86_64/linux-user/main.c:4862

(gdb) x/i $pc
=> 0x555555993665 <static_code_gen_buffer+13333>:	mov    %ebp,%gs:(%ebx)
(gdb) info reg
rax            0x1	1
rbx            0xf70373ac	4144198572
rcx            0x0	0
rdx            0x55555597b880	93824996587648
rsi            0x555555993640	93824996685376
rdi            0x5555582b6290	93825039819408
rbp            0xe3833022	0xe3833022
rsp            0x7fffffffd2c0	0x7fffffffd2c0
r8             0x0	0
r9             0x0	0
r10            0x55555597b7f0	93824996587504
r11            0x206	518
r12            0x555555993640	93824996685376
r13            0xf703704c	4144197708
r14            0x5555582b6290	93825039819408
r15            0x5555582ae5b8	93825039787448
rip            0x555555993665	0x555555993665 <static_code_gen_buffer+13333>
eflags         0x10286	[ PF SF IF RF ]
cs             0x33	51
ss             0x2b	43
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
(gdb) x/12i $pc-0x20
   0x555555993645 <static_code_gen_buffer+13301>:	in     (%dx),%eax
   0x555555993646 <static_code_gen_buffer+13302>:	
    jl     0x555555993828 <static_code_gen_buffer+13784>
   0x55555599364c <static_code_gen_buffer+13308>:	mov    $0xf70373ac,%ebp
   0x555555993651 <static_code_gen_buffer+13313>:	mov    %gs:0x0(%ebp),%ebp
   0x555555993656 <static_code_gen_buffer+13318>:	mov    0xc(%r14),%ebx
   0x55555599365a <static_code_gen_buffer+13322>:	or     %ebx,%ebp
   0x55555599365c <static_code_gen_buffer+13324>:	mov    %ebp,0x8(%r14)
   0x555555993660 <static_code_gen_buffer+13328>:	mov    $0xf70373ac,%ebx
=> 0x555555993665 <static_code_gen_buffer+13333>:	mov    %ebp,%gs:(%ebx)
   0x555555993669 <static_code_gen_buffer+13337>:	mov    0x34(%r14),%ebp
   0x55555599366d <static_code_gen_buffer+13341>:	mov    %gs:0x0(%ebp),%ebx
   0x555555993672 <static_code_gen_buffer+13346>:	mov    %ebx,0x10(%r14)
(gdb) q

=====

$ qemu-arm -strace ./foo
3184 mmap2(0xf7000000,228092,PROT_EXEC|PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS|MAP_FIXED,-1,0) = 0xf7000000
--- SIGSEGV {si_signo=SIGSEGV, si_code=1, si_addr=0xf70373ac} ---
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
$ 

=====

When run on 32-bit ARM (RaspberryPi 3B) under Fedora 27:
$ strace ./foo
execve("./foo", ["./foo"], 0xbee3b150 /* 52 vars */) = 0
mmap2(0xb6fa2000, 228092, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6fa2000
cacheflush(0xb6fd904c, 0xb6fd9afc, 0, 0xb6fd904c, 0) = 0
readlink("/proc/self/exe", "/home/jreiser/foo", 4095) = 17
open("/proc/self/exe", O_RDONLY)        = 3
cacheflush(0xbefae71c, 0xbefae830, 0, 0xbefae71c, 0) = 0
mmap2(NULL, 1122304, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6d7e000
mmap2(0xb6d7e000, 397204, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6d7e000
cacheflush(0xb6d7e000, 0xb6d7e114, 0, 0xb6d7e000, 0) = 0
cacheflush(0xb6d7e114, 0xb6ddef94, 0, 0xb6d7e114, 0) = 0
cacheflush(0xb6ddef94, 0xb6ddef9c, 0, 0xe1a0f00e, 0xb6ddef94) = 0
mprotect(0xb6d7e000, 397204, PROT_READ|PROT_EXEC) = 0
mmap2(0xb6def000, 524608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6def000
cacheflush(0xb6def000, 0xb6e6f140, 0, 0xb6def000, 0) = 0
mprotect(0xb6def000, 524608, PROT_READ|PROT_WRITE) = 0
mmap2(0xb6e70000, 127324, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb6e70000
open("/usr/local/bin/upx-linux-musl-gcc-7.2.0-toolchains-20170914/linux-musl-gcc-7.2.0-20170914-default-pie/arm-linux-musleabihf-gcc-7.2.0/arm-linux-musleabihf/lib/ld-musl-armhf.so.1", O_RDONLY) = -1 ENOENT (No such file or directory)
exit(127)                               = ?
+++ exited with 127 +++

Comment 1 John Reiser 2017-09-19 21:12:58 UTC
Created attachment 1328167 [details]
"./foo" 32-bit ARM executable

Runs natively on 32-bit ARM Fedora 27 (see strace at end of Description), gets SIGSEGV under qemu-arm on x86_64.

Comment 2 John Reiser 2017-09-19 22:23:06 UTC
The ARM instructions in the vicinity of the SIGSEGV during emulation by qemu-arm:
=====
0xf703704c:
        ldr r2,mflg_here  // pc+856
        orr r2,r2,r3  @ modify the instruction
=>      str r2,mflg_here  // pc+848    the faulting instruction

     [[snip about 848 bytes]]

0xf70373ac:
  mflg_here:  // The next instruction is re-written once.
        orr r3,r3,#0  @ flags |= MAP_{PRIVATE|ANON}  [QNX vs Linux]
=====

The modified instruction at 0xf70373ac is covered by the
   mmap2(0xf7000000,228092,PROT_EXEC|PROT_READ|PROT_WRITE, ...
because 0x373ac is 226220 which is less than 228092, so the address has PROT_WRITE access.  The word at 0xf70373ac was written (by storing 4 consecutive bytes individually) exactly once before, may have been read afterwards but never has been executed or otherwise written, and at 848 bytes ahead of pc is outside the range of any prefetch for instruction caching.  Therefore the access should be allowed, and an actual 32-bit armv7hl ARM processor does allow it.

Comment 3 Cole Robinson 2017-09-19 22:33:03 UTC
Thanks for the report. Is this a regression, or is it your first time trying qemu for this?

If it isn't a regression, I suggest taking this to qemu-devel, it can probably get straightened out quickly

Comment 4 John Reiser 2017-09-19 23:21:31 UTC
This is the first time.  Inquiry sent to qemu-devel.

Comment 5 John Reiser 2017-09-20 22:34:15 UTC
Distilled from [qemu-devel]: qemu-arm wants 0xf7000000 and above for itself, and doesn't always protect it enough.  There may be a workaround using something like "-R 0xfffe0000" but it is easier just to avoid 0xf7000000 and above.  Note also that qemu-arm on a program which is ET_DYN (-fpie) puts the stack below the code (which is just below 0xf7000000.)