Bug 545411 (CVE-2009-4272)

Summary: CVE-2009-4272 kernel: emergency route cache flushing leads to node deadlock
Product: [Other] Security Response Reporter: Konstantin Khorenko <khorenko>
Component: vulnerabilityAssignee: Red Hat Product Security <security-response-team>
Status: CLOSED ERRATA QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: atangrin, bressers, dhoward, dwu, hjia, jlieskov, kolyshkin, kzhang, lwang, nhorman, rkhan, security-response-team, tao, vdanen, vvs
Target Milestone: ---Keywords: Security
Target Release: ---   
Hardware: All   
OS: Linux   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-10 16:20:49 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 545662, 545663, 553538    
Bug Blocks:    
Description Flags
initial revision of a patch
additional patch to fix uninitialized rp issue none

Description Konstantin Khorenko 2009-12-08 14:15:33 UTC
Recently Parallels Virtuozzo Containers team has found a kernel bug that leads to a node freeze.
We were able to get the root of the problem: it is linux-2.6-net-allow-for-on-demand-emergency-route-cache-flushing.patch, added in 2.6.18-164.x kernels (i've checked 164.6.1.el5, it still contains the bug).

The calltrace of the deadlocked process:
             for (i = rt_hash_mask; i >= 0; i--) {

deadlock, bh is disabled => all the node quickly becomes "freezed".

We believe this is a security issue, cause this is a remote DOS.

Original raw calltrace:

Route hash chain too long!
Adjust your secret_interval!
SysRq HELP : loglevel0-8 vsced_stAte reBoot Crashdump tErm Full debuG kIll thaw-filesystems(J) saK showMem Nice powerOff showPc unRaw Sync showTasks Unmount shoWcpus
SysRq: Show Regs
----------- IPI show regs -----------
Pid: 23107, comm:              dcpumon, oopses: 0
EIP: 0060:[<0261abf6>] CPU: 3, VCPU: 9665:2
EIP is at _spin_lock_bh+0xf/0x18
 EFLAGS: 00000282    Tainted:  P       (2.6.18-028stab066.8-ent #1 028stab066)
EAX: c00bc000 EBX: d9f25950 ECX: 00000088 EDX: c00bc000
ESI: 00000000 EDI: 00000000 EBP: 00001950 DS: 007b ES: 007b
CR0: 80050033 CR2: b7dfa03c CR3: 00799000 CR4: 000006f0
 [<025d0da4>] rt_run_flush+0x74/0x123
 [<025d1bd9>] rt_intern_hash+0x4a4/0x6f2
 [<025d26bf>] __ip_route_output_key+0x898/0x909
 [<025d2743>] ip_route_output_flow+0x13/0x24f
 [<025de3b9>] inet_csk_route_req+0xb8/0x1c7
 [<025edc9a>] tcp_v4_send_synack+0x17/0x169
 [<025f072d>] tcp_v4_conn_request+0x4cc/0x517
 [<0261ab57>] _read_lock_bh+0x8/0x17
 [<025e7b52>] tcp_rcv_state_process+0x5d/0xd66
 [<025eebb1>] tcp_v4_do_rcv+0x2c3/0x323
 [<025f1781>] tcp_v4_rcv+0x95e/0x9b1
 [<025d60e5>] ip_local_deliver+0x1a6/0x26d
 [<025d5efc>] ip_rcv+0x505/0x548

 [<025b8b7e>] netif_receive_skb+0x3b2/0x3f0
 [<025ba6f7>] process_backlog+0x7e/0xd1
 [<025ba8c6>] net_rx_action+0x9c/0x1a7
 [<024275c4>] __do_softirq+0xe0/0x193
 [<024276ad>] do_softirq+0x36/0x3a
 [<024277c8>] local_bh_enable+0x74/0x7f
 [<025baf37>] dev_queue_xmit+0x34f/0x380
 [<025bf6f4>] neigh_resolve_output+0x1a7/0x1d1
 [<025db407>] ip_output+0x299/0x2c2
 [<025daa33>] ip_queue_xmit+0x4cf/0x515
 [<025d26bf>] __ip_route_output_key+0x898/0x909
 [<025ef380>] tcp_v4_send_check+0x77/0xbd
 [<025e9da6>] tcp_transmit_skb+0x687/0x6b5
 [<025ec3e2>] tcp_connect+0x341/0x3c1
 [<025f0040>] tcp_v4_connect+0x708/0x89f
 [<024690a1>] kunmap_high+0x14/0x84
 [<025fad1d>] inet_stream_connect+0x7d/0x208
 [<02475c7a>] get_user_size+0x30/0x57
 [<025afa9e>] sys_connect+0x6d/0x98
 [<0246cfea>] follow_page_pte+0x173/0x195
 [<02475997>] __kmap_atomic_user_page+0xdb/0xf3
 [<024758a8>] __kunmap_atomic_user_page+0x10/0x24
 [<02475c2f>] rw_vm+0x10d/0x128
 [<025afbe6>] sys_socketcall+0x8e/0x1af
 [<02407400>] do_syscall_trace+0xab/0xb1
----------- IPI show regs -----------

Best regards,

Konstantin Khorenko,
PVC/OpenVZ developer,

Comment 6 Thomas Graf 2009-12-09 11:43:34 UTC
Created attachment 377148 [details]
initial revision of a patch

This is how a fix could look like, I have not tested it yet because I was not able to reproduce the issue myself yet.


Comment 15 Thomas Graf 2009-12-15 14:11:54 UTC
Created attachment 378505 [details]
additional patch to fix uninitialized rp issue

This is a follow up patch based on the upstream commits

to address the uninitialized rp issues when rt caching is disabled.

Comment 19 errata-xmlrpc 2010-01-19 23:38:34 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Linux 5

Via RHSA-2010:0046 https://rhn.redhat.com/errata/RHSA-2010-0046.html

Comment 22 errata-xmlrpc 2010-02-09 15:24:34 UTC
This issue has been addressed in following products:

  Red Hat Enterprise Virtualization for RHEL-5

Via RHSA-2010:0095 https://rhn.redhat.com/errata/RHSA-2010-0095.html