Bug 177430

Summary: 32bit app crashes x86_64 2.6.9-22.0.1.ELsmp kernel
Product: Red Hat Enterprise Linux 4 Reporter: James Pearson <james-p>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED DUPLICATE QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.0   
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2006-01-12 21:53:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description James Pearson 2006-01-10 17:13:13 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040524

Description of problem:
Running a third party 32bit application on a x86_64 install using the 2.6.9-22.0.1.EL (SMP) x86_64 kernel crashes the machine with:

Kernel BUG at prio_tree:528
invalid operand: 0000 [1] SMP 
CPU 0 
Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 i2c_dev i2c_core nfs lockd sunrpc dm_mod button battery ac ohci_hcd ehci_hcd tg3 floppy ext3 jbd sata_nv libata sd_mod scsi_mod
Pid: 3173, comm: app.bin Not tainted 2.6.9-22.0.1.ELsmp
RIP: 0010:[<ffffffff8015e780>] <ffffffff8015e780>{vma_prio_tree_add+70}
RSP: 0018:000001007d4ade80  EFLAGS: 00010216
RAX: 0000000000000016 RBX: 00000100349be4b8 RCX: 0000000000000000
RDX: 0000000000000434 RSI: 000001003c6d11f8 RDI: 00000100349be4b8
RBP: 000001007efb2040 R08: 0000000000000016 R09: 0000000000000000
R10: ffffffff803ef900 R11: 0000010001003950 R12: 0000010034ab8298
R13: 000001013f667ba0 R14: 0000010034ab82a8 R15: 0000010034ab8268
FS:  0000002a959b6de0(0000) GS:ffffffff804d3080(005b) knlGS:00000000f1a316c0
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000e6209000 CR3: 0000000000101000 CR4: 00000000000006e0
Process app.bin (pid: 3173, threadinfo 000001007d4ac000, task 000001007fdd67f0)
Stack: ffffffff80169022 0000000000000000 00000100349be4b8 0000000000000000 
       0000000000100077 000001013d9bc7c0 00000000e620a000 0000000000435000 
       ffffffff8016a9a7 0000000000000435 
Call Trace:<ffffffff80169022>{vma_link+204} <ffffffff8016a9a7>{do_mmap_pgoff+1444} 
       <ffffffff80126f00>{sys32_mmap2+253} <ffffffff8012500f>{cstar_do_call+27} 
       

Code: 0f 0b 4e eb 31 80 ff ff ff ff 10 02 48 c7 47 60 00 00 00 00 
RIP <ffffffff8015e780>{vma_prio_tree_add+70} RSP <000001007d4ade80>
 <0>Kernel panic - not syncing: Oops


Version-Release number of selected component (if applicable):
kernel-smp-2.6.9-22.0.1.EL

How reproducible:
Always

Steps to Reproduce:
1. Run application

  

Actual Results:  Oops as above

Expected Results:  Doesn't oops ...

Additional info:

I think it's related to to the linux-2.6.9-x86_64-flexmmap.patch (and linux-2.6.9-x86_64-task_size-32bit.patch) - backing out both these patches and rebuilding a kernel and the application doesn't oops.

However, with the default kernel, 32 bit applications can grab up to 4Gb of memory - with these patches removed, they can only get about 2.7Gb

Also, using a FC4 kernel (2.6.11-1.1369_FC4smp) kernel on the same machine and it doesn't oops ...

Comment 1 James Pearson 2006-01-12 17:16:56 UTC
Just tried kernel-smp-2.6.9-27.EL from U3 beta - and it doesn't oops ...

Does anyone have any idea which one(s) of the newer patches 'fixes' this problem?



Comment 2 Jason Baron 2006-01-12 21:53:14 UTC
indeed. I was going to mention to try the beta, see bugzillas:

Bugzilla: 171778
Bugzilla: 173981
Bugzilla: 175728

*** This bug has been marked as a duplicate of 171778 ***