Bug 437358 (XenPvops32emu)

Summary: F-10 pv_ops xen: Add ia32 emulation
Product: [Fedora] Fedora Reporter: Eduardo Habkost <ehabkost>
Component: kernel-xenAssignee: Eduardo Habkost <ehabkost>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 9CC: bjorn.sund, markmc, mb--redhat, orion, xen-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 2.6.25.3-2.fc9 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-06-16 23:32:12 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 442569    
Attachments:
Description Flags
0001-ia32entry.S-Replace-cli-sti-cases-with-DISABLE_INTE.patch
none
0002-Introduce-xen_ia32_syscall.patch
none
0003-Use-xen_ia32_syscall-if-CONFIG_XEN-is-enabled-FIX.patch
none
0004-Make-the-int80-and-syscall-vdso32-implementations-se.patch
none
0005-Use-vdso32-int80-by-default-if-Xen-is-enabled-FIXME.patch none

Description Eduardo Habkost 2008-03-13 17:29:02 UTC
Currently, 32-bit binaries cause the x86_64 pvops kernel-xen to crash on 
ia32_syscall:

general protection fault: 0000 [1]
CPU 0
Modules linked in:
Pid: 1404, comm: mbchk Not tainted 2.6.25-rc3-pvops #90
RIP: e030:[<ffffffff802219c0>]  [<ffffffff802219c0>] ia32_syscall+0x0/0x44
RSP: e02b:ffff8800ba7fdfc8  EFLAGS: 00010296
RAX: 000000000000007a RBX: 00000000ffffdaf6 RCX: ffffffff802219c0
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: 00000000ffffdcf8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000296 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  00007ffff7fe86f0(0000) GS:ffffffff8058b000(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b CR0: 000000008005003b
CR2: 0000000008054d40 CR3: 00000000b853c000 CR4: 0000000000002620
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000000
Process mbchk (pid: 1404, threadinfo ffff8800ba7fc000, task ffff8800ba690af0)
Stack:  00000000ffffe000 0000000000000000 0000000008054d4d 0000000000000023
 0000000000000296 00000000ffffdacc 000000000000002b
Call Trace:


Code: 8b 64 24 18 48 8b 6c 24 20 48 8b 5c 24 28 48 83 c4 30 41 87 e9 44 8b 44 
24 68 e9 f5 fe ff ff 48 c7 c0 f2 ff ff ff eb 65 0f 1f 00 <0f> 01 f8 fb 89 c0 
50 fc 48 83 ec 48 48 89 7c 24 40 48 89 74 24
RIP  [<ffffffff802219c0>] ia32_syscall+0x0/0x44
 RSP <ffff8800ba7fdfc8>

Comment 1 Eduardo Habkost 2008-03-13 17:34:23 UTC
ia32 emulation will be temporarily disabled on x86_64 kernel-xen until this 
bug is fixed.

Comment 2 Mark McLoughlin 2008-04-10 13:21:25 UTC
Basically, this means that ia32 binaries don't work on x86_64 xen right now.

We're not sure yet whether we'll get this fixed by GA.

Comment 3 Mark McLoughlin 2008-04-15 15:39:36 UTC
Looks like we'll have to punt this to F10 (or a post GA F9 update)

Comment 4 Mark McLoughlin 2008-04-16 14:14:46 UTC
*** Bug 442685 has been marked as a duplicate of this bug. ***

Comment 5 Mark McLoughlin 2008-04-16 14:16:38 UTC
bug #442685 details how not having ia32 emulation means that guest installs hang
at the very end when grub (a 32 bit binary) can't be executed ...

This is less critical than it sounds - pygrub doesn't require grub-install to
have been run, so when the install hangs you should be able to reboot without
any problems. Very annoying, though.

Comment 6 Eduardo Habkost 2008-04-16 20:57:19 UTC
The experimental code on the ia32emu-dev[1] branch on xen-pvops-64.git is able
to run simple ia32 binaries, but it is not completely stable. nash segfaults
when loading a 32-bit initrd and the kernel oopses when booting using a 64-bit
initrd and 32-bit root filesystem.

It is able to run grub, though.  :)

[1] http://git.et.redhat.com/?p=xen-pvops-64.git;a=shortlog;h=ia32emu-dev

Comment 7 Mark McLoughlin 2008-04-17 17:45:28 UTC
Okay, these look fairly sane to me - the changes are:

  - always use int80 for ia32 syscalls, never sysenter
  - handle the different stack format for int80 entry under xen
  - use pv_ops irq_enable/disable in ia32 syscall asm

I'll give them a test tomorrow, but if it looks like it fixes grub-install on
x86_64 and doesn't cause any other regressions, then I think we should include them.

Comment 8 Mark McLoughlin 2008-04-17 17:49:57 UTC
Created attachment 302774 [details]
0001-ia32entry.S-Replace-cli-sti-cases-with-DISABLE_INTE.patch

Comment 9 Mark McLoughlin 2008-04-17 17:50:25 UTC
Created attachment 302775 [details]
0002-Introduce-xen_ia32_syscall.patch

Comment 10 Mark McLoughlin 2008-04-17 17:50:59 UTC
Created attachment 302776 [details]
0003-Use-xen_ia32_syscall-if-CONFIG_XEN-is-enabled-FIX.patch

Comment 11 Mark McLoughlin 2008-04-17 17:51:24 UTC
Created attachment 302777 [details]
0004-Make-the-int80-and-syscall-vdso32-implementations-se.patch

Comment 12 Mark McLoughlin 2008-04-17 17:51:53 UTC
Created attachment 302778 [details]
0005-Use-vdso32-int80-by-default-if-Xen-is-enabled-FIXME.patch

Comment 13 Eduardo Habkost 2008-04-18 04:15:59 UTC
The risk of enabling ia32 emulation as it is now is security: it is very easy 
to Oops the kernel when running ia32 binaries.

Comment 14 Mark McLoughlin 2008-04-18 08:00:04 UTC
Okay, that's a fair point

I've reopened bug #442685, and am putting back on the F10_XenPvOps tracker for now

Comment 15 Orion Poplawski 2008-04-24 16:29:34 UTC
Would it be possible to add a command line argument to enable ia32 emulation? 
That way installs could be run with it, but then turned off for normal operation.

Comment 16 Eduardo Habkost 2008-04-25 22:01:23 UTC
Got it a little better with an additional fix.

Now, half of the initscripts from a 32-bit filesystem image run, but with 
random segfaults and hangs.

http://git.et.redhat.com/?p=xen-pvops-64.git;a=commit;h=c7e49e971a9d27e196550979a7c3fea36f887473

Comment 17 Bug Zapper 2008-05-14 06:01:59 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 18 Eduardo Habkost 2008-05-27 14:31:11 UTC
*** Bug 448465 has been marked as a duplicate of this bug. ***

Comment 19 Mark McLoughlin 2008-05-29 16:42:14 UTC
Pulled in Eduardo's patches from here:

http://git.et.redhat.com/?p=xen-pvops-64.git;a=commit;h=e1bc320d3f1713f0964c70adf15971295a298f27

and re-enabled CONFIG_IA32_EMULATION

Will be in kernel-xen-2.6-2.6.25.2-4.fc10 and kernel-xen-2.6-2.6.25.3-2.fc9

* Thu May 29 2008 Mark McLoughlin <markmc>
- Enable ia32 emulation (ehabkost, #437358)


Comment 20 Fedora Update System 2008-05-29 18:31:50 UTC
kernel-xen-2.6-2.6.25.3-2.fc9 has been submitted as an update for Fedora 9

Comment 21 Need Real Name 2008-05-30 06:07:27 UTC
Thanks, that seems to have fixed it (bug 448465, installing glibc.i686) for me.

Comment 22 Mark McLoughlin 2008-05-30 06:59:03 UTC
(In reply to comment #21)
> Thanks, that seems to have fixed it (bug 448465, installing glibc.i686) for me.

Thanks for testing; could you bump the update's karma here:

https://admin.fedoraproject.org/updates/F9/pending/kernel-xen-2.6-2.6.25.3-2.fc9




Comment 23 Eduardo Habkost 2008-06-15 16:39:03 UTC
*** Bug 451520 has been marked as a duplicate of this bug. ***

Comment 24 Fedora Update System 2008-06-16 23:32:09 UTC
kernel-xen-2.6-2.6.25.3-2.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.