RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 670907 - [RHEL6.1][Kernel] BUG: unable to handle kernel NULL pointer dereference, IP: [<ffffffff814115f0>] get_rps_cpu+0x290/0x340
Summary: [RHEL6.1][Kernel] BUG: unable to handle kernel NULL pointer dereference, IP: ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: kernel
Version: 6.1
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: rc
: ---
Assignee: Neil Horman
QA Contact: Liang Zheng
URL:
Whiteboard:
Depends On:
Blocks: 6.1KnownIssues
TreeView+ depends on / blocked
 
Reported: 2011-01-19 16:19 UTC by Jeff Burke
Modified: 2011-08-05 21:08 UTC (History)
6 users (show)

Fixed In Version: kernel-2.6.32-112.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-05-23 20:38:10 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
fix for rps oops (707 bytes, patch)
2011-01-24 18:07 UTC, Neil Horman 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2011:0542 0 normal SHIPPED_LIVE Important: Red Hat Enterprise Linux 6.1 kernel security, bug fix and enhancement update 2011-05-19 11:58:07 UTC

Description Jeff Burke 2011-01-19 16:19:50 UTC 
Description of problem:
 While running tests on a scratch kernel with a "Receive Packet Steering patch" the system Oops

Version-Release number of selected component (if applicable):
2.6.32-100.el6Scratch (test kernel)

How reproducible:
Always

Actual results:
igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX 
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready 
br0: port 1(eth0) entering learning state 
br0: port 1(eth0) entering forwarding state 
BUG: unable to handle kernel NULL pointer dereference at 0000000000000010 
IP: [<ffffffff814115f0>] get_rps_cpu+0x290/0x340 
PGD 46bc3b067 PUD 46c7b8067 PMD 0  
Oops: 0000 [#1] SMP  
last sysfs file: /sys/devices/system/cpu/cpu23/cache/index2/shared_cpu_map 
CPU 1  
Modules linked in: sit tunnel4 ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT bridge stp llc autofs4 sunrpc cpufreq_ondemand acpi_cpufreq freq_table ipv6 xt_physdev iptable_filter ip_tables dm_mirror dm_region_hash dm_log kvm_intel kvm i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support ioatdma i7core_edac edac_core sg igb dca ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif ahci megaraid_sas dm_mod [last unloaded: microcode] 
 
Modules linked in: sit tunnel4 ip6table_filter ip6_tables ebtable_nat ebtables xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack ipt_REJECT bridge stp llc autofs4 sunrpc cpufreq_ondemand acpi_cpufreq freq_table ipv6 xt_physdev iptable_filter ip_tables dm_mirror dm_region_hash dm_log kvm_intel kvm i2c_i801 i2c_core iTCO_wdt iTCO_vendor_support ioatdma i7core_edac edac_core sg igb dca ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif ahci megaraid_sas dm_mod [last unloaded: microcode] 
Pid: 0, comm: swapper Not tainted 2.6.32-100.el6scratch.x86_64 #1 QSSC-S4R 
RIP: 0010:[<ffffffff814115f0>]  [<ffffffff814115f0>] get_rps_cpu+0x290/0x340 
RSP: 0018:ffff88048e403b80  EFLAGS: 00010246 
RAX: 0000000000000001 RBX: 0000000000000001 RCX: ffff88086bc80000 
RDX: ffffffff81f02368 RSI: 0000000000000206 RDI: 0000000000000000 
RBP: ffff88048e403bc0 R08: ffffffff81411a60 R09: ffff88048e403ba8 
R10: 0000000000000000 R11: ffff88086a8c55f8 R12: ffff88086a8c55c0 
R13: 0000000000000006 R14: ffff88086b7c2022 R15: ffff88086a8c55f8 
FS:  0000000000000000(0000) GS:ffff88048e400000(0000) knlGS:0000000000000000 
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b 
CR2: 0000000000000010 CR3: 000000046bd86000 CR4: 00000000000026e0 
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 
Process swapper (pid: 0, threadinfo ffff880c6dde8000, task ffff88086de9e040) 
Stack: 
 ffffffff81411a60 ffff88086bc80000 000000078e416f00 ffff88086a8c55c0 
<0> ffff88086a8c55c0 ffff88086bc806c0 ffff88086b7c2022 ffff88086a8c55f8 
<0> ffff88048e403be0 ffffffff81411a7d ffff88086b7c2022 ffff88086a8c55c0 
Call Trace: 
 <IRQ>  
 [<ffffffff81411a60>] ? netif_receive_skb+0x0/0x50 
 [<ffffffff81411a7d>] netif_receive_skb+0x1d/0x50 
 [<ffffffffa02fe3c8>] br_handle_frame_finish+0x198/0x260 [bridge] 
 [<ffffffffa02fe63a>] br_handle_frame+0x1aa/0x250 [bridge] 
 [<ffffffff8140caa3>] __netif_receive_skb+0x1c3/0x670 
 [<ffffffff81411aa0>] netif_receive_skb+0x40/0x50 
 [<ffffffff81411ba8>] napi_skb_finish+0x58/0x70 
 [<ffffffff814120a9>] napi_gro_receive+0x39/0x50 
 [<ffffffffa01128c6>] igb_poll+0x8d6/0xf30 [igb] 
 [<ffffffff8104f0ba>] ? enqueue_entity+0x13a/0x340 
 [<ffffffff810561ec>] ? try_to_wake_up+0xcc/0x400 
 [<ffffffff81412273>] net_rx_action+0x103/0x2a0 
 [<ffffffff8108d373>] ? hrtimer_get_next_event+0xc3/0x100 
 [<ffffffff8106bcb7>] __do_softirq+0xb7/0x1e0 
 [<ffffffff810d0dc0>] ? handle_IRQ_event+0x60/0x170 
 [<ffffffff8100c2cc>] call_softirq+0x1c/0x30 
 [<ffffffff8100df05>] do_softirq+0x65/0xa0 
 [<ffffffff8106bab5>] irq_exit+0x85/0x90 
 [<ffffffff814ce455>] do_IRQ+0x75/0xf0 
 [<ffffffff8100bad3>] ret_from_intr+0x0/0x11 
 <EOI>  
 [<ffffffff812ad4ba>] ? intel_idle+0xda/0x160 
 [<ffffffff812ad49d>] ? intel_idle+0xbd/0x160 
 [<ffffffff813dc2a7>] cpuidle_idle_call+0xa7/0x140 
 [<ffffffff81009e96>] cpu_idle+0xb6/0x110 
 [<ffffffff814c01fc>] start_secondary+0x1fc/0x23f 
Code: 85 c0 48 8b 7d c8 0f 84 11 fe ff ff 83 bf f0 03 00 00 01 48 c7 c1 24 f5 7e 81 8b 9f a0 06 00 00 48 0f 44 cf 48 8b bf 30 04 00 00 <48> 8b 57 10 48 89 4d c0 48 89 55 c8 e8 9f 5c f1 ff 48 8b 4d c0  
RIP  [<ffffffff814115f0>] get_rps_cpu+0x290/0x340 
 RSP <ffff88048e403b80> 
CR2: 0000000000000010 

Expected results:
System should not Oops

Additional info:
This needs br0 interface to reproduce
Comment 2 Neil Horman 2011-01-21 21:23:22 UTC 
So this is a wierd one, for some reason, when I add a bridge into the mix, we oops in get_rps_cpu.  The bridge sends us through a path in get_rps_cpu that we other wise wouldn't which causes a printk to be triggered, which is where we seem to oops
but the poniters that that printk interrogates are all valid, so I'm not at all certain whats going on here yet.
Comment 3 Neil Horman 2011-01-21 21:46:04 UTC 
ok, it appears the netdev_warn macro is trying to dereference something in the netdev structure of the bridge interface that isn't set properly.  If i remove the call everything works fine.  I'll figure out what it is and get a patch together shortly.
Comment 4 Neil Horman 2011-01-22 02:08:29 UTC 
note to self.  its net_device->dev.parent thats null, and netdev_warn presumes that its always non-null.  I'll need to patch that.
Comment 5 Neil Horman 2011-01-24 15:57:06 UTC 
looks like upstream commit 08c801f8d45387a1b46066aad1789a9bb9c4b645 inadvertently fixed this when RPS was origionally introduced.  'll have a patch shortly.
Comment 6 Neil Horman 2011-01-24 18:07:09 UTC 
Created attachment 475004 [details]
fix for rps oops
Comment 7 RHEL Program Management 2011-01-24 22:50:22 UTC 
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.
Comment 8 Aristeu Rozanski 2011-02-03 16:38:34 UTC 
Patch(es) available on kernel-2.6.32-112.el6
Comment 13 Aristeu Rozanski 2011-03-02 15:23:25 UTC 
this patch went in 104, released on 112. a followup patch was submitted and got
into 115.
(625487)
Comment 15 errata-xmlrpc 2011-05-23 20:38:10 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0542.html
Note You need to log in before you can comment on or make changes to this bug.