Bug 177430 - 32bit app crashes x86_64 2.6.9-22.0.1.ELsmp kernel
Summary: 32bit app crashes x86_64 2.6.9-22.0.1.ELsmp kernel
Status: CLOSED DUPLICATE of bug 171778
Alias: None
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel
Version: 4.0
Hardware: x86_64
OS: Linux
Target Milestone: ---
: ---
Assignee: Kernel Maintainer List
QA Contact: Brian Brock
Depends On:
TreeView+ depends on / blocked
Reported: 2006-01-10 17:13 UTC by James Pearson
Modified: 2007-11-30 22:07 UTC (History)
0 users

Clone Of:
Last Closed: 2006-01-12 21:53:14 UTC

Attachments (Terms of Use)

Description James Pearson 2006-01-10 17:13:13 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040524

Description of problem:
Running a third party 32bit application on a x86_64 install using the 2.6.9-22.0.1.EL (SMP) x86_64 kernel crashes the machine with:

Kernel BUG at prio_tree:528
invalid operand: 0000 [1] SMP 
CPU 0 
Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 i2c_dev i2c_core nfs lockd sunrpc dm_mod button battery ac ohci_hcd ehci_hcd tg3 floppy ext3 jbd sata_nv libata sd_mod scsi_mod
Pid: 3173, comm: app.bin Not tainted 2.6.9-22.0.1.ELsmp
RIP: 0010:[<ffffffff8015e780>] <ffffffff8015e780>{vma_prio_tree_add+70}
RSP: 0018:000001007d4ade80  EFLAGS: 00010216
RAX: 0000000000000016 RBX: 00000100349be4b8 RCX: 0000000000000000
RDX: 0000000000000434 RSI: 000001003c6d11f8 RDI: 00000100349be4b8
RBP: 000001007efb2040 R08: 0000000000000016 R09: 0000000000000000
R10: ffffffff803ef900 R11: 0000010001003950 R12: 0000010034ab8298
R13: 000001013f667ba0 R14: 0000010034ab82a8 R15: 0000010034ab8268
FS:  0000002a959b6de0(0000) GS:ffffffff804d3080(005b) knlGS:00000000f1a316c0
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000e6209000 CR3: 0000000000101000 CR4: 00000000000006e0
Process app.bin (pid: 3173, threadinfo 000001007d4ac000, task 000001007fdd67f0)
Stack: ffffffff80169022 0000000000000000 00000100349be4b8 0000000000000000 
       0000000000100077 000001013d9bc7c0 00000000e620a000 0000000000435000 
       ffffffff8016a9a7 0000000000000435 
Call Trace:<ffffffff80169022>{vma_link+204} <ffffffff8016a9a7>{do_mmap_pgoff+1444} 
       <ffffffff80126f00>{sys32_mmap2+253} <ffffffff8012500f>{cstar_do_call+27} 

Code: 0f 0b 4e eb 31 80 ff ff ff ff 10 02 48 c7 47 60 00 00 00 00 
RIP <ffffffff8015e780>{vma_prio_tree_add+70} RSP <000001007d4ade80>
 <0>Kernel panic - not syncing: Oops

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Run application


Actual Results:  Oops as above

Expected Results:  Doesn't oops ...

Additional info:

I think it's related to to the linux-2.6.9-x86_64-flexmmap.patch (and linux-2.6.9-x86_64-task_size-32bit.patch) - backing out both these patches and rebuilding a kernel and the application doesn't oops.

However, with the default kernel, 32 bit applications can grab up to 4Gb of memory - with these patches removed, they can only get about 2.7Gb

Also, using a FC4 kernel (2.6.11-1.1369_FC4smp) kernel on the same machine and it doesn't oops ...

Comment 1 James Pearson 2006-01-12 17:16:56 UTC
Just tried kernel-smp-2.6.9-27.EL from U3 beta - and it doesn't oops ...

Does anyone have any idea which one(s) of the newer patches 'fixes' this problem?

Comment 2 Jason Baron 2006-01-12 21:53:14 UTC
indeed. I was going to mention to try the beta, see bugzillas:

Bugzilla: 171778
Bugzilla: 173981
Bugzilla: 175728

*** This bug has been marked as a duplicate of 171778 ***

Note You need to log in before you can comment on or make changes to this bug.