Attachment 148240 Details for Bug 229081 – logs with sysrq

logs with sysrq

messages (text/plain), 651.68 KB, created by Josef Bacik on 2007-02-16 20:35:27 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Josef Bacik

Created: 2007-02-16 20:35:27 UTC

Size: 651.68 KB

patch

obsolete

>Feb 16 14:59:40 rh5cluster2 openais[2365]: [TOTEM] entering OPERATIONAL state. 
>Feb 16 14:59:40 rh5cluster2 openais[2365]: [CLM  ] got nodejoin message 10.10.1.12 
>Feb 16 14:59:40 rh5cluster2 openais[2365]: [CPG  ] got joinlist message from node 2 
>Feb 16 15:00:04 rh5cluster2 fenced[2381]: fence "rh5cluster1.gsslab.rdu.redhat.com" success
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] entering GATHER state from 11. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] Creating commit token because I am the rep. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] Saving state aru 17 high seq received 17 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] entering COMMIT state. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] entering RECOVERY state. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] position [0] member 10.10.1.12: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] previous ring seq 6336 rep 10.10.1.12 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] aru 17 high delivered 17 received flag 0 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] position [1] member 10.10.1.13: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] previous ring seq 4 rep 10.10.1.13 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] aru 9 high delivered 9 received flag 0 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] Did not need to originate any messages in recovery. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] Storing new sequence id for ring 18c4 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] Sending initial ORF token 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] CLM CONFIGURATION CHANGE 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] New Configuration: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] 	r(0) ip(10.10.1.12)  
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] Members Left: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] Members Joined: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [SYNC ] This node is within the primary component and will provide service. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] CLM CONFIGURATION CHANGE 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] New Configuration: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] 	r(0) ip(10.10.1.12)  
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] 	r(0) ip(10.10.1.13)  
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] Members Left: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] Members Joined: 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] 	r(0) ip(10.10.1.13)  
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [SYNC ] This node is within the primary component and will provide service. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [TOTEM] entering OPERATIONAL state. 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] got nodejoin message 10.10.1.12 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CLM  ] got nodejoin message 10.10.1.13 
>Feb 16 15:02:23 rh5cluster2 openais[2365]: [CPG  ] got joinlist message from node 2 
>Feb 16 15:02:28 rh5cluster2 kernel: dlm: connecting to 1
>Feb 16 15:08:56 rh5cluster2 ccsd[2359]: Update of cluster.conf complete (version 2 -> 3). 
>Feb 16 15:17:50 rh5cluster2 kernel: dlm: connecting to 1
>Feb 16 15:17:50 rh5cluster2 kernel: dlm: got connection from 1
>Feb 16 15:17:58 rh5cluster2 kernel: dlm: connecting to 1
>Feb 16 15:17:58 rh5cluster2 kernel: dlm: got connection from 1
>Feb 16 15:18:07 rh5cluster2 clvmd: Cluster LVM daemon started - connected to CMAN
>Feb 16 15:18:44 rh5cluster2 kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "rhel5cluster:gfs2lv"
>Feb 16 15:18:44 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: Joined cluster. Now mounting FS...
>Feb 16 15:18:45 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: jid=1, already locked for use
>Feb 16 15:18:45 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: jid=1: Looking at journal...
>Feb 16 15:18:45 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: jid=1: Done
>Feb 16 15:19:37 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: fatal: assertion "!atomic_read(&gl->gl_ail_count)" failed
>Feb 16 15:19:37 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1:   function = gfs2_meta_inval, file = fs/gfs2/meta_io.c, line = 101
>Feb 16 15:19:37 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: about to withdraw this file system
>Feb 16 15:19:37 rh5cluster2 kernel: GFS2: fsid=rhel5cluster:gfs2lv.1: telling LM to withdraw
>Feb 16 15:22:09 rh5cluster2 kernel: 273fa8 00000000 00000000 c9204448 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9272a90 c0420b4c 00000001 c9272a90 c06ae440 d331f65e 0000016a 000005e5 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9272bb4 c9204400 00000000 cc818ac0 c9272f54 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044de5e>] watchdog+0x0/0x52
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044dea1>] watchdog+0x43/0x52
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: events/0      S C9274F78  2672     5      1             6     4 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c9274f8c 00000046 00000002 c9274f78 c9274f74 00000000 c925b5e0 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000001 c9274f64 0000000a c9272560 c06ae440 9aa249e1 0000016a 00012f27 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9272684 c9204400 00000000 cc818ac0 00000000 c925b5c0 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: khelper       S 00000003  3332     6      1             7     5 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c9276f8c 00000046 00000000 00000003 c925b4e0 000000d8 c925b4e0 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000001 c9276f64 0000000a c9272030 cafcca90 eba6a1e6 0000001c 000073b1 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9272154 c9204400 00000000 cf9b5040 00000000 00000000 c042f8b3 c9276fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kthread       S 00000003  3328     7      1    44     434     6 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c927af8c 00000046 cc248ce8 00000003 c92781e0 00000003 c92781e0 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000001 c927af64 00000009 c9279a90 cfd9f560 9a7e0b62 0000013b 00000588 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9279bb4 c9204400 00000000 cbe91740 00000000 00000000 c042f8b3 c927afa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kblockd/0     S 00000003  3412    44      7            45       (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd5af8c 00000046 00000002 00000003 cfd30660 00000000 cfd30660 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000001 cfd5af64 0000000a cfd47560 ce487030 ce18380a 0000015f 00009403 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd47684 c9204400 00000000 cbe913c0 00000000 00000000 ffffffff cfd5afa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kacpid        S C0406FB4  3784    45      7           137    44 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd5ef8c 00000046 00000000 c0406fb4 c0422130 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd5efbc 00000000 00000001 cfd47030 c9272030 5926fa42 0000000a 00001ae0 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd47154 c9204400 00000000 c06ae280 cfd056c4 00000000 c042f8b3 cfd5efa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: cqueue/0      S 00000086  3784   137      7           138    45 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd25f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24560 c9279a90 5aa26eca 0000000a 000008de 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd24684 c9204400 00000000 c06ae280 cfd05184 00000000 c042f8b3 cfd25fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: ksuspend_usbd S 00000086  3784   138      7           141   137 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd28f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24030 c9272030 5aa310e1 0000000a 00000567 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd24154 c9204400 00000000 c06ae280 c92bac04 00000000 c042f8b3 cfd28fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: khubd         S 00000004  3456   141      7           143   138 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd31f3c 00000046 cfde2660 00000004 c05788f0 00049544 000000a3 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        80000180 cfdf9400 0000000a cfd32a90 cfd15030 fe28ddce 0000000a 00001d24 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd32bb4 c9204400 00000000 cfdafac0 00000004 000003e8 cfd31f9c c06cf940 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05788f0>] usb_control_msg+0xc5/0xcf
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c057676a>] hub_thread+0x940/0x9a7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0575e2a>] hub_thread+0x0/0x9a7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kseriod       S CFD46F7C  3176   143      7           166   141 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd46f90 00000046 00000002 cfd46f7c cfd46f78 00000000 00000000 c054941b 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000004 c06d0a4c 0000000a cfd47a90 c06ae440 eaa67105 0000000a 00000d93 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd47bb4 c9204400 00000000 cfdafc80 c06d089c c06d08a0 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c054941b>] driver_probe_device+0x44/0xa5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0587f66>] serio_thread+0x25b/0x2a9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0587d0b>] serio_thread+0x0/0x2a9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: pdflush       S CFD71F88  3668   166      7           167   143 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd71f9c 00000046 00000002 cfd71f88 cfd71f84 00000000 c07a5c00 00000286 
>Feb 16 15:22:09 rh5cluster2 kernel:        c042ef10 00000000 0000000a cfd6e560 c06ae440 19e968fa 00000133 00001974 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd6e684 c9204400 00000000 cf9b5740 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04570ae>] pdflush+0xb0/0x19f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: pdflush       D CFD74EEC  2420   167      7           168   166 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd74f00 00000046 00000002 cfd74eec cfd74ee8 00000000 c91a8440 c91b7d80 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd74f8c 00000000 0000000a cfd75a90 c06ae440 05c0f89c 0000016b 000010ab 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd75bb4 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d504>] io_schedule_timeout+0x3f/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045a78e>] congestion_wait+0x50/0x64
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0456d64>] wb_kupdate+0x98/0xde
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0457107>] pdflush+0x109/0x19f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0456ccc>] wb_kupdate+0x0/0xde
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kswapd0       S CFD2AF34  3340   168      7           169   167 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd2af48 00000046 00000002 cfd2af34 cfd2af30 00000000 00000000 05f37243 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9258d4c 00000000 0000000a cfd2ba90 c06ae440 2d4d6955 00000133 000018ee 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd2bbb4 c9204400 00000000 cc818ac0 c045a799 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045a799>] congestion_wait+0x5b/0x64
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04594e2>] kswapd+0xb4/0x3d5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045942e>] kswapd+0x0/0x3d5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aio/0         S 00000086  3784   169      7           329   168 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd33f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd32560 c9272030 6ae0b51c 0000000a 00000d4e 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd32684 c9204400 00000000 c06ae280 cffc7c04 00000000 c042f8b3 cfd33fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kpsmoused     S 00000086  3784   329      7           362   169 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd18f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000009 cfd15560 cfb73a90 e3191df7 0000000a 00000b35 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd15684 c9204400 00000000 c06ae280 cffc7184 00000000 c042f8b3 cfd18fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kmirrord      S 00000086  3784   362      7           367   329 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cff9af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cff7b730 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd6e030 cff7b560 8757d833 0000000e 0000097a 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd6e154 c9204400 00000000 cfdaf900 cfc3d184 00000000 c042f8b3 cff9afa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: ksnapd        S 00000086  3784   367      7           370   362 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cff73f8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd75200 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9201700 00000000 00000005 cff7b030 cfd75030 877b12a3 0000000e 0000095c 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff7b154 c9204400 00000000 cfdaf900 cfc3d6c4 00000000 c042f8b3 cff73fa8 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kjournald     S CFD93F88  2364   370      7           400   367 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd93f9c 00000046 00000002 cfd93f88 cfd93f84 cdb09740 00000000 cff98800 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 0000000a cff72030 cfd75a90 acbf3d26 00000169 0000186f 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff72154 c9204400 00000000 cc818ac0 c0420985 00000000 cfd93fac cf406a68 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kauditd       S CFF70F60  3700   400      7          1294   370 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cff70fa0 00000046 00000001 cff70f60 c041fa15 00000000 00000001 cb71d218 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 cb71d218 0000000a cfb73a90 cfd15a90 8ddbdcb4 0000015e 00001510 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfb73bb4 c9204400 00000000 cbe913c0 c0420985 00000000 00000000 cff70fb0 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044986d>] kauditd_thread+0x103/0x132
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044976a>] kauditd_thread+0x0/0x132
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: udevd         S CFF78B48  2580   434      1          1977     7 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cff78b5c 00000086 00000002 cff78b48 cff78b44 00000000 00000004 cff78b18 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000011 c044f208 00000004 c9279030 c06ae440 1b53b7b6 00000148 0001b087 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9279154 c9204400 00000000 cfdaf3c0 00000203 00000200 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c048fb82>] inotify_poll+0x3a/0x40
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04dedbb>] number+0x105/0x24d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047007b>] copy_strings+0xe7/0x1b9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04376c6>] bit_waitqueue+0x16/0x48
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04824fc>] seq_printf+0x2e/0x4b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c049f22a>] show_stat+0x9ec/0x9f9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0483a12>] simple_empty+0xe/0x50
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04293f5>] do_wait+0x8ba/0x95e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: kjournald     S CFD16F88  3184  1294      7          2941   400 (L-TLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd16f9c 00000046 00000002 cfd16f88 cfd16f84 00000000 00000000 cfa56000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000002 c9279560 c06ae440 4b25b168 0000001a 0000132b 
>Feb 16 15:22:09 rh5cluster2 kernel:        c9279684 c9204400 00000000 cce333c0 c0420985 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: dhclient      S CFD63B48  2476  1977      1          2106   434 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd63b5c 00000082 00000002 cfd63b48 cfd63b44 00000000 cc130e08 d0859b10 
>Feb 16 15:22:09 rh5cluster2 kernel:        0063c853 00000000 0000000a ccea3030 c06ae440 946642af 00000018 00007ef7 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccea3154 c9204400 00000000 cf9b5c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0457b8d>] activate_page+0x61/0x84
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0457d23>] mark_page_accessed+0x1c/0x30
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: auditd        S CCAC9B48  2476  2106      1  2108    2107  1977 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ccac9b5c 00000086 00000002 ccac9b48 ccac9b44 00000000 c041ff8c d0859b10 
>Feb 16 15:22:09 rh5cluster2 kernel:        0063d05f ccac9bec 00000006 cfd15a90 c06ae440 8662758d 00000165 00002fc4 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd15bb4 c9204400 00000000 cfdaf200 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477612>] poll_freewait+0x18/0x4c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477948>] do_sys_poll+0x302/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04224cb>] __cond_resched+0x16/0x34
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05c4e5b>] netlink_unicast+0x171/0x1a9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05c45d2>] netlink_sendskb+0x19/0x30
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05c5012>] netlink_ack+0x17f/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044973e>] audit_receive+0x944/0x970
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043b209>] wake_futex+0x42/0x4c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046d3c6>] do_readv_writev+0x17d/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a75c7>] sock_aio_write+0x0/0x102
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: auditd        S CFC562BC  2176  2107      1          2127  2106 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc8a0e80 00000086 000000cf cfc562bc cfd2b030 0000000a cc8a0000 cc8a0f64 
>Feb 16 15:22:09 rh5cluster2 kernel:        cc8a0f64 cc8a0ee0 00000007 cfd2b030 cb3d6030 8dddc5e1 0000015e 00016596 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd2b154 c9204400 00000000 cfdaf200 cfdaf200 c043b07d cc8a0f48 c0419ccd 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e3ca3>] percpu_counter_sum+0x37/0x52
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046b6df>] vfs_statfs+0x47/0x5f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046c5ce>] sys_fstatfs+0x4b/0x54
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: python        S CFF60B48  1820  2108   2106                     (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cff60b5c 00200082 00000002 cff60b48 cff60b44 00000000 0000019a 02cf6a5d 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000075 00000002 cfd6ea90 c06ae440 8e91f09a 0000015e 000381b1 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd6ebb4 c9204400 00000000 cf9b5e40 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c06074c2>] unix_write_space+0x12/0x71
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: syslogd       S CCFF0B48  1732  2127      1          2130  2107 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ccff0b5c 00000086 00000002 ccff0b48 ccff0b44 00000000 cff596bc 00000003 
>Feb 16 15:22:09 rh5cluster2 kernel:        c04d9696 00000001 00000007 ccfe8030 c06ae440 61c1357e 00000167 00007438 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccfe8154 c9204400 00000000 cf9b5200 c04d021f cff4cb30 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04d9696>] cfq_insert_request+0x6d/0x391
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04d021f>] elv_insert+0x146/0x1ec
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ae8c5>] datagram_poll+0x14/0xb6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040418b>] setup_sigcontext+0x10b/0x18b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0439a0b>] enqueue_hrtimer+0x4b/0x6a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0439e11>] hrtimer_start+0xb2/0xbc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042a28a>] do_setitimer+0x166/0x4d6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: portmap       S 00000046  2488  2183      1          2222  2130 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cceeebfc 00000082 c9205900 00000046 00000046 c9205880 00000246 00000246 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfa57d80 c060e744 00000002 cfd75560 cfd75030 de9fcb2d 00000018 000087fb 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd75684 c9204400 00000000 cce33e40 cfa57d80 c06c9600 cceeec68 cb71d998 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a826f>] sys_sendmsg+0x192/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8c6a>] sys_recvmsg+0x14b/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b2666>] netdev_run_todo+0x1ec/0x1f3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05eedea>] inet_sock_destruct+0x16a/0x1b0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e862>] _write_lock_bh+0x8/0x1a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: rpc.statd     S CC908B48  1600  2222      1          2265  2183 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc908b5c 00000082 00000002 cc908b48 cc908b44 00000000 00ef8300 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 cf4606c0 00000001 cfd75030 c06ae440 dea35352 00000018 00003bd7 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd75154 c9204400 00000000 cce33580 03a57d00 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: rpc.idmapd    S CC990F20  3064  2265      1          2359  2222 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc990f34 00200086 00000002 cc990f20 cc990f1c 00000000 00000000 bf8329cc 
>Feb 16 15:22:09 rh5cluster2 kernel:        cc990f60 cf9806bc 0000000a cff7ba90 c06ae440 e49ec5b9 00000169 00006e27 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff7bbb4 c9204400 00000000 cf9b53c0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c049043f>] sys_epoll_wait+0x13e/0x37f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: ccsd          S CA943B48  2364  2359      1          2360  2265 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ca943b5c 00000086 00000002 ca943b48 ca943b44 00000000 0000003e 0b19380a 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000365 00000007 cfd9aa90 cfd9f560 94ab951a 0000013b 00005f7a 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd9abb4 c9204400 00000000 cce33ac0 0000012d 00000000 ca943c0c caf5e218 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0ecaad3>] ip6_push_pending_frames+0x34c/0x3d6 [ipv6]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05acc76>] skb_queue_purge+0x11/0x17
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060666d>] unix_sock_destructor+0xe/0xd2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05aa1d3>] sk_free+0xb2/0xc7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: ccsd          S C9D7FB48  2260  2360      1          2365  2359 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c9d7fb5c 00000086 00000002 c9d7fb48 c9d7fb44 00000000 00000086 c042ee07 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff4cb9c 00000000 00000008 cfb73560 c06ae440 e564e91c 000000b2 0001c961 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfb73684 c9204400 00000000 cce33ac0 00000003 c04d2cce ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04d2cce>] __generic_unplug_device+0x1d/0x1f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0908098>] tg3_start_xmit_dma_bug+0x581/0x64f [tg3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ac932>] skb_clone+0x3d/0x1f1
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060a167>] packet_rcv_spkt+0x0/0x139
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b127c>] dev_hard_start_xmit+0x1bc/0x21b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e36f7>] tcp_v4_send_check+0x74/0xaa
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05d6e05>] tcp_close+0x504/0x523
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CC991BE8  2104  2365      1          2366  2360 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc991bfc 00000086 00000002 cc991be8 cc991be4 00000000 000001f7 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 cff72560 c06ae440 037039a8 0000016b 00012caa 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff72684 c9204400 00000000 cce333c0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 10 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8b07>] move_addr_to_user+0x50/0x68
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CAFC7BE8  2740  2366      1          2367  2365 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cafc7bfc 00000086 00000002 cafc7be8 cafc7be4 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 cfb73030 c06ae440 4a55bda4 00000019 00005293 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfb73154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S C992BE6C  3024  2367      1          2374  2366 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c992be80 00000086 00000002 c992be6c c992be68 00000000 cfd878c0 cff2b114 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd878c0 cfd87920 00000006 cfd32030 c06ae440 842192fd 00000057 00002982 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd32154 c9204400 00000000 cce333c0 c047513c c992bd90 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CFD9BBE8  2740  2374      1          2375  2367 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd9bbfc 00000086 00000002 cfd9bbe8 cfd9bbe4 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccea3560 c06ae440 ed7134ee 00000157 000004ff 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccea3684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CD78DBE8  2688  2375      1          2373  2374 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cd78dbfc 00000086 00000002 cd78dbe8 cd78dbe4 00000000 000001cc 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 cd78dbc0 00000001 ccfe8560 cc24ea90 1aabccf2 00000148 000004a7 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccfe8684 c9204400 00000000 cce333c0 c04057e7 c0a3f680 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CD073BE8  2740  2394      1          2395  2387 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cd073bfc 00000086 00000002 cd073be8 cd073be4 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 cd117560 ce487030 1ae0cde0 00000148 000007ea 
>Feb 16 15:22:09 rh5cluster2 kernel:        cd117684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CCDC2BE8  2740  2395      1          2393  2394 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ccdc2bfc 00000086 00000002 ccdc2be8 ccdc2be4 00000000 00000194 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1a90 c06ae440 1a7d9fb8 00000148 00000406 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccde1bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CCEA0000  2784  2396      1          2397  2393 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ccdc4bfc 00000086 00000000 ccea0000 cfacba80 00000000 000000b1 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1560 cfd9f560 93ff7cdd 0000013b 00000590 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccde1684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S 00000000  2868  2397      1          2400  2396 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c08c7bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccfe8a90 cff72560 9d870693 00000019 00000a7a 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccfe8bb4 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S 00000000  2868  2400      1          2401  2397 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cd093bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 cd117030 cfd2b560 0348822d 00000058 00000802 
>Feb 16 15:22:09 rh5cluster2 kernel:        cd117154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CCEA0000  2800  2401      1          2419  2400 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc4c4bfc 00000086 00000000 ccea0000 cfa51c80 00000000 000000c2 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccadb560 cfd2ba90 dc3cfdfa 00000019 000004e0 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccadb684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e007b>] __copy_to_user_ll+0x32/0xd6
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S 00000000  2868  3311      1          3312  3081 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        ccb6ebfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 cafcc030 c9279030 d3e6a84d 00000130 00000758 
>Feb 16 15:22:09 rh5cluster2 kernel:        cafcc154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: aisexec       S CCEA0000  2800  3312      1          3302  3311 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc59cbfc 00000086 00000000 ccea0000 cfa47980 00000000 00000019 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000000 00000000 00000001 ccc20a90 c9279030 d3e077b4 00000130 000003ca 
>Feb 16 15:22:09 rh5cluster2 kernel:        ccc20bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: groupd        S CC8A9BE8  2476  2373      1          2381  2375 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cc8a9bfc 00000082 00000002 cc8a9be8 cc8a9be4 00000000 cff4cb30 00000020 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000046 c0406fb4 00000002 cff7b560 ccea3560 ed712fef 00000157 00003fc5 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff7b684 c9204400 00000000 cce33040 c062ad80 cc8a9ea4 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 7 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: fenced        S C0A5FBE8  2476  2381      1          2387  2373 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        c0a5fbfc 00000086 00000002 c0a5fbe8 c0a5fbe4 00000000 cff4cb30 00000008 
>Feb 16 15:22:09 rh5cluster2 kernel:        00300000 00000000 00000009 cfd2b560 c06ae440 036ac9b9 00000058 00002bf0 
>Feb 16 15:22:09 rh5cluster2 kernel:        cfd2b684 c9204400 00000000 cce33200 00001000 00000001 ffffffff 00000000 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: dlm_controld  S CFF72A90  2476  2387      1          2394  2381 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        caef8bfc 00000082 00000117 cff72a90 c9204400 caef8bb8 c0423f8f 00000046 
>Feb 16 15:22:09 rh5cluster2 kernel:        00000046 00000000 00000001 cff72a90 cb3d6a90 1ac52279 00000148 0000612f 
>Feb 16 15:22:09 rh5cluster2 kernel:        cff72bb4 c9204400 00000000 cfdaf580 0807b008 caef8e9c 00000018 c04057e7 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05ae166>] memcpy_toiovec+0x27/0x4a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05c59dd>] netlink_recvmsg+0x285/0x2a4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: gfs_controld  S 00000012  2184  2393      1          2396  2395 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cd784bfc 00000082 00000000 00000012 00000000 00000100 cceb5b54 08262176 
>Feb 16 15:22:09 rh5cluster2 kernel:        cd117a90 00000003 00000001 cd117a90 cd117560 1ae0c5f6 00000148 000082a4 
>Feb 16 15:22:09 rh5cluster2 kernel:        cd117bb4 c9204400 00000000 cfdaf740 0000000e 0000000b 00000001 cd784ebc 
>Feb 16 15:22:09 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:09 rh5cluster2 last message repeated 5 times
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:09 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:09 rh5cluster2 kernel:  =======================
>Feb 16 15:22:09 rh5cluster2 kernel: dbus-daemon   S CCED6BE8  2308  2419      1          2474  2401 (NOTLB)
>Feb 16 15:22:09 rh5cluster2 kernel:        cced6bfc 00000082 00000002 cced6be8 cced6be4 00000000 cff4cb30 00000008 
>Feb 16 15:22:10 rh5cluster2 kernel:        00300000 ccadba90 00000009 ccadba90 c06ae440 6b9635d1 00000020 0000c09e 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccadbbb4 c9204400 00000000 cfdafc80 00001000 00000001 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 5 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060da42>] __wait_on_bit_lock+0x4b/0x52
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hcid          S CA93ABE8  2128  2474      1          2487  2419 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca93abfc 00000086 00000002 ca93abe8 ca93abe4 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        cf4606c0 cbc75e00 00000001 cfd24a90 c06ae440 c8e71f0d 0000001c 0002c8d2 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd24bb4 c9204400 00000000 cfdafac0 ca93ac0c ffffffff ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sdpd          S 00000000  2660  2487      1          2498  2474 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc21b28 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 00000000 00000001 cbc98a90 ccadb030 c5a36808 0000001c 0004ebc9 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc98bb4 c9204400 00000000 cf9b5580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478111>] sys_pselect7+0xdd/0x208
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782a9>] sys_pselect6+0x6d/0x79
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: krfcommd      S CBC38680  3480  2498      1          2547  2487 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc6aaf20 00000046 00000000 cbc38680 d0a2b75f 00000000 ffffffea d0a90079 
>Feb 16 15:22:10 rh5cluster2 kernel:        c924fa00 00000286 00000005 cc24e560 ccea3a90 cfe5f3ca 0000001c 00002ad9 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc24e684 c9204400 00000000 cf9b5740 00000000 00000246 00000000 c992d4c0 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a2b75f>] l2cap_sock_create+0x64/0x6b [l2cap]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a90079>] bt_sock_create+0x79/0x8f [bluetooth]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b28dad>] rfcomm_run+0x184/0x1097 [rfcomm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cfcc>] audit_syscall_exit+0x294/0x2b0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0407880>] do_syscall_trace+0x57/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b28c29>] rfcomm_run+0x0/0x1097 [rfcomm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: pcscd         S D084A4C7  2108  2547      1          2551  2498 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbcfdb5c 00000082 d0845080 d084a4c7 cd276e80 c048b0ae cff4c02c cd276e80 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 00001000 00000001 cc24e030 ccbb9560 e429912c 0000001c 0004fdce 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc24e154 c9204400 00000000 cf9b5740 d084aff0 00000000 00000001 00000008 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060db04>] __wait_on_bit+0x50/0x58
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d085d1f8>] find_revoke_record+0x42/0x6c [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d085d6c8>] journal_cancel_revoke+0x7a/0x9c [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d085dce1>] journal_add_journal_head+0x1c/0x136 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04572cd>] __do_page_cache_readahead+0x84/0x1bd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0452679>] do_generic_mapping_read+0x3f4/0x43b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04ddfdc>] rb_insert_color+0x8c/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045ecbd>] __vma_link+0x44/0x4f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04243cb>] sched_fork+0x73/0xaa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0425dbf>] copy_process+0xefa/0x10b1
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0424753>] wake_up_new_task+0x1af/0x1b7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: pcscd         S CFD0EF24  3188  2551      1          2572  2547 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd0ef38 00000082 00000002 cfd0ef24 cfd0ef20 00000000 cfd0eefc c920554c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c0439f90 00000007 ccbb9560 c06ae440 f91782e1 0000016a 0001f0cc 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccbb9684 c9204400 00000000 cf9b5740 7ba68370 00000161 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hidd          S 00000000  2812  2572      1          2592  2551 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cf969bc0 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 00000000 00000001 cba58560 ccadb030 f010d4e3 0000001c 000085f6 
>Feb 16 15:22:10 rh5cluster2 kernel:        cba58684 c9204400 00000000 cbeffe40 00000000 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a903a5>] bt_sock_poll+0x14/0xf4 [bluetooth]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047843a>] sys_ppoll+0xd9/0x1fc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: automount     S CFBAFC00  2104  2592      1          2593  2572 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc60ed8 00000082 00000000 cfbafc00 c92a6800 00000004 c9177220 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfbafc00 c0472500 00000001 cbc98030 cba58030 08c2edd5 0000001d 00007fd4 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc98154 c9204400 00000000 cfdafe40 c04ddf9c 00000004 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472500>] pipe_write+0x393/0x3ec
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04ddf9c>] rb_insert_color+0x4c/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0430d0d>] dequeue_signal+0x36/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04319fe>] sys_rt_sigtimedwait+0x1cd/0x2a8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: automount     S CC50AE6C  3384  2593      1          2594  2592 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc50ae80 00000082 00000002 cc50ae6c cc50ae68 00000000 cb81cf34 cb81cf0c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cc50ae58 00000007 cafcca90 c06ae440 06e137f7 00000164 000014c4 
>Feb 16 15:22:10 rh5cluster2 kernel:        cafccbb4 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: automount     S CB81CE6C  2316  2594      1          2597  2593 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cb81ce80 00000082 00000002 cb81ce6c cb81ce68 00000000 cf764f34 cf764f0c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000254 c06b6280 0000000a ccbb9030 c06ae440 000b52fd 0000016b 0000344f 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccbb9154 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: automount     S 00000086  2376  2597      1          2600  2594 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cba67bfc 00000082 c0564b78 00000086 c042ee07 cfb76200 c07a5c00 00000086 
>Feb 16 15:22:10 rh5cluster2 kernel:        c042ef10 00000000 0000000a cba58a90 ccbb9030 8a40c87b 00000160 000020be 
>Feb 16 15:22:10 rh5cluster2 kernel:        cba58bb4 c9204400 00000000 cfdafe40 cba67c70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0564b78>] ide_build_sglist+0x2b/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05645da>] ide_dma_exec_cmd+0x1f/0x22
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: automount     S CFF4C02C  2184  2600      1          2616  2597 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cfc4bbfc 00000082 c048b0ae cff4c02c cbc75500 00000001 00001000 030c0020 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 00000000 0000000a cba58030 ccbb9030 06e0b73b 00000164 00002206 
>Feb 16 15:22:10 rh5cluster2 kernel:        cba58154 c9204400 00000000 cfdafe40 cfc4bc70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: acpid         S CC50BBE8  2108  2616      1          2632  2600 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc50bbfc 00000082 00000002 cc50bbe8 cc50bbe4 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        cf4606c0 cbc75480 00000007 ccadb030 ca6a4560 e17f9d47 0000001e 00020054 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccadb154 c9204400 00000000 cf9b5040 00001000 00000000 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08596cf>] journal_stop+0x1b6/0x1c2 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891bde>] ext3_journal_dirty_data+0x0/0x32 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453322>] generic_file_buffered_write+0x4ca/0x5f8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453a08>] generic_file_aio_write+0x5e/0xb3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d088ffdc>] ext3_file_write+0x24/0x8f [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hpiod         S 3537FFFF  2476  2632      1          2637  2616 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cadf9e3c 00200086 00000000 3537ffff 00003736 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cbc98560 00000001 cbc98560 ccbb9a90 186c89d0 0000001d 00005699 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc98684 c9204400 00000000 cf9b5900 cadf9e94 cadf9e95 cadf9e94 ffffffff 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9d7f>] lock_sock_nested+0x8a/0x92
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d4088>] inet_csk_accept+0x9b/0x200
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b7de>] d_instantiate+0x5c/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05eebea>] inet_accept+0x1f/0xb5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a90a0>] sys_accept+0xee/0x1c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a89ec>] sys_connect+0x82/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9248>] sys_socketcall+0xd6/0x261
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: python        S CA721B48  2476  2637      1          2654  2632 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca721b5c 00200086 00000002 ca721b48 ca721b44 00000000 0003ea1c c04057e7 
>Feb 16 15:22:10 rh5cluster2 kernel:        c0406fb4 00200286 0000000a cbea7560 c06ae440 bbd33096 0000016a 00016ace 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbea7684 c9204400 00000000 cbeff900 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: cupsd         S CC31EB48  2040  2654      1          2673  2637 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc31eb5c 00200082 00000002 cc31eb48 cc31eb44 00000000 cc130c68 d0859b10 
>Feb 16 15:22:10 rh5cluster2 kernel:        0063f04f 00000000 00000007 cbea7030 c06ae440 9a21bfcf 0000001d 00008dde 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbea7154 c9204400 00000000 cce33740 cc31eb6c ffffffff ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ff5c>] signal_wake_up+0x1e/0x2c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04301fb>] __group_send_sig_info+0x74/0x7f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04313fc>] sys_kill+0xee/0xfa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sshd          S CB443B48  2476  2673      1  3166    2699  2654 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cb443b5c 00200082 00000002 cb443b48 cb443b44 00000000 cb203a2c d0859b10 
>Feb 16 15:22:10 rh5cluster2 kernel:        00638802 00000000 00000007 cbea7a90 c06ae440 8c927a69 0000015e 000067b3 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbea7bb4 c9204400 00000000 cbeffc80 d0891698 00001000 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437720>] __wake_up_bit+0x28/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0461d88>] anon_vma_prepare+0x11/0xa5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sendmail      S CBC37B48  2476  2699      1          2707  2673 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc37b5c 00200082 00000002 cbc37b48 cbc37b44 00000000 cb203a2c d0859b10 
>Feb 16 15:22:10 rh5cluster2 kernel:        0063d888 00000000 0000000a cc891a90 c06ae440 f1c557c4 00000169 0001d411 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc891bb4 c9204400 00000000 cbeff740 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0524a91>] extract_buf+0x9d/0xce
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0499ce9>] proc_alloc_inode+0x3e/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047cff4>] iget_locked+0x8b/0xfb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0499d9c>] proc_get_inode+0x8e/0x116
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c049cf16>] proc_lookup+0x72/0xa4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0499f7f>] proc_root_lookup+0xe/0x26
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089ebbe>] ext3_permission+0x0/0xa [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04df806>] sprintf+0x20/0x23
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c049e6a7>] loadavg_read_proc+0xce/0xfa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060e8e1>] lock_kernel+0x14/0x2e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0499eb3>] proc_delete_inode+0x0/0x41
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sendmail      S CA26FF98  2476  2707      1          2724  2699 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca26ffac 00200086 00000002 ca26ff98 ca26ff94 00000000 00000000 00000002 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000002 caf9a000 00000001 cc891030 c06ae440 ab35b889 0000001d 0000e0f1 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc891154 c9204400 00000000 cf9b5ac0 ca26ffb8 c040794d ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043157e>] sys_pause+0x11/0x17
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gpm           S CDBD39B0  2476  2724      1          2740  2707 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cafaeb5c 00000082 00000000 cdbd39b0 00000000 cdbd39b0 cb203a2c d0859b10 
>Feb 16 15:22:10 rh5cluster2 kernel:        0063a001 00000000 00000001 cc4f8560 cc4f8030 b193838a 0000001d 0001d5e7 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc4f8684 c9204400 00000000 cbeff580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042d3ac>] __capable+0xc/0x1f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052c256>] vt_ioctl+0x555/0x148f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052bd01>] vt_ioctl+0x0/0x148f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047652a>] fasync_helper+0x3c/0xae
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046dbf7>] file_kill+0x11/0x2a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046bc23>] do_filp_open+0x32/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: crond         S CA6D8F24  2128  2740      1          2778  2724 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca6d8f38 00000086 00000002 ca6d8f24 ca6d8f20 00000000 ca6d8efc c920554c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c0439f90 00000007 cc891560 c06ae440 3de6c644 00000169 0001c2db 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc891684 c9204400 00000000 cbeff200 7c5f7fa7 0000016d ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: xfs           S CA8F5B48  1936  2778      1          2809  2740 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca8f5b5c 00000082 00000002 ca8f5b48 ca8f5b44 00000000 005d8030 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cf4606c0 00000007 cb2b6a90 c06ae440 b481b55d 00000134 0000407d 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb2b6bb4 c9204400 00000000 cc818740 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488721>] ll_rw_block+0x84/0x93
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04555cd>] free_pages_bulk+0x100/0x16e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04557f4>] __pagevec_free+0x14/0x1a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0457dc8>] __pagevec_release_nonlru+0x44/0x4f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0458a9f>] remove_mapping+0x84/0xa7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04590b7>] shrink_inactive_list+0x5f5/0x6c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04585e7>] isolate_lru_pages+0x64/0x7d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0457d7c>] __pagevec_release+0x15/0x1d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0458986>] shrink_active_list+0x386/0x38e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0475e9c>] open_namei+0x6e/0x58d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046bc16>] do_filp_open+0x25/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: atd           S CF934F24  2476  2809      1          2828  2778 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cf934f38 00000082 00000002 cf934f24 cf934f20 00000000 cf934efc c920554c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c0439f90 00000007 cb2b6030 c06ae440 df6b0d4e 00000134 00009134 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb2b6154 c9204400 00000000 cc818c80 e7fb5841 00000170 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: yum-updatesd  S CA666BE8  1640  2828      1          2845  2809 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca666bfc 00200086 00000002 ca666be8 ca666be4 00000000 ca666fac 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        c04057e7 ca666c0c 0000000a cb2b6560 c06ae440 0565df23 0000016b 0000535e 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb2b6684 c9204400 00000000 cc818ac0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048b4a7>] bio_add_page+0x31/0x3a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045eba5>] find_extend_vma+0x12/0x49
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043007b>] __group_complete_signal+0x111/0x21d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043c05e>] do_futex+0x202/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: avahi-daemon  S CAFB3BE8  2072  2845      1  2846    2862  2828 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cafb3bfc 00000082 00000002 cafb3be8 cafb3be4 00000000 cfa44480 cafb3c30 
>Feb 16 15:22:10 rh5cluster2 kernel:        c05c71fc ccea0000 0000000a cc4f8a90 c06ae440 a40ceca6 00000079 0000d2fe 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc4f8bb4 c9204400 00000000 cc818e40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05c71fc>] nf_iterate+0x38/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 8 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a965d>] sock_common_recvmsg+0x3e/0x54
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05e95fe>] udp_ioctl+0x5a/0x5e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05e95a4>] udp_ioctl+0x0/0x5e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: avahi-daemon  S CE1F2DB8  3204  2846   2845                     (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ce1f2dcc 00000082 00000002 ce1f2db8 ce1f2db4 00000000 00000001 cf4987dc 
>Feb 16 15:22:10 rh5cluster2 kernel:        cf9adc9c 00000000 00000001 ccf45560 c06ae440 8ba94daf 0000001e 000007a2 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccf45684 c9204400 00000000 cbeff040 c0472eca ccf45560 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald          S CBA74BE8  1632  2862      1  2863    3072  2845 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cba74bfc 00000086 00000002 cba74be8 cba74be4 00000000 00000000 00000801 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfbe8318 00000000 00000006 cc4f8030 c06ae440 81581056 00000166 0002fa6f 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc4f8154 c9204400 00000000 cc818900 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 11 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d089e901>] ext3_get_acl+0x51/0x2ae [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0475069>] __link_path_walk+0xc02/0xc2c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-runner   S CBEE3BE8  2068  2863   2862  2869               (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbee3bfc 00000086 00000002 cbee3be8 cbee3be4 00000000 00000001 cbc776e8 
>Feb 16 15:22:10 rh5cluster2 kernel:        cff76000 cff76000 00000003 ccbb9a90 cbac9560 50819f84 00000020 0000e820 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccbb9bb4 c9204400 00000000 cc818200 cbee3c0c ffffffff 00000286 c042ee65 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ee65>] try_to_del_timer_sync+0x44/0x4a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-addon-ac S CCFA63EC  2172  2869   2863          2873       (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ce1d5dcc 00000082 cc130c68 ccfa63ec d085ab14 cf406600 000000c7 d0891698 
>Feb 16 15:22:10 rh5cluster2 kernel:        00001000 00000000 00000004 ca6a4560 cc4f8030 e17fea4b 0000001e 00004d04 
>Feb 16 15:22:10 rh5cluster2 kernel:        ca6a4684 c9204400 00000000 cc818040 00000000 00000003 00000282 00000246 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0460254>] do_mmap_pgoff+0x57b/0x6db
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-addon-ke S CBF1BF3C  2172  2873   2863          2881  2869 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbf1bf50 00000082 00000002 cbf1bf3c cbf1bf38 00000000 ca6a4030 cbf1bf24 
>Feb 16 15:22:10 rh5cluster2 kernel:        c0420d86 ca6a4030 0000000a ca6a4030 c06ae440 cfbdb7a0 000000c1 00003a31 
>Feb 16 15:22:10 rh5cluster2 kernel:        ca6a4154 c9204400 00000000 cbe91e40 00000046 00000000 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420d86>] task_running_tick+0x22a/0x234
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-addon-ke S 00000000  2868  2881   2863          2884  2873 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ce9a0f50 00000086 cc8183c0 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 ceb4c710 00000001 ce99fa90 ccbb9a90 136f0b55 0000001f 00089ea0 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce99fbb4 c9204400 00000000 cc8183c0 c06bc760 00100073 ce9a0f58 cf8a4c00 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-addon-ke S CBF0AF3C  2868  2884   2863          2889  2881 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbf0af50 00000086 00000002 cbf0af3c cbf0af38 00000000 00000000 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cbda8e48 00000001 ce99f030 c06ae440 15e8bd1a 0000001f 0008d3b7 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce99f154 c9204400 00000000 cbe91ac0 c06bc760 00100073 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: hald-addon-st S CF395F24  2172  2889   2863                2884 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cf395f38 00000086 00000002 cf395f24 cf395f20 00000000 cf395efc c920554c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c0439f90 00000007 ce414a90 c06ae440 d831ab25 0000016a 00002c76 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce414bb4 c9204400 00000000 cbe91900 964d0078 00000161 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: scsi_eh_1     S C9279A90  3752  2941      7          2947  1294 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cff64f6c 00000046 c92048c0 c9279a90 c0420b4c c9204400 c9279a90 cff64f28 
>Feb 16 15:22:10 rh5cluster2 kernel:        c0420dac 00000000 00000005 cfd9a030 cfd9fa90 77a24451 0000001f 00000456 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd9a154 c9204400 00000000 cbeffac0 c92048c0 c9fc8c84 c060d307 cff64fc8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d086c867>] scsi_error_handler+0x53/0x462 [scsi_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: qla2xxx_1_dpc S CFF62F8C  2920  2947      7          2951  2941 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cff62fa0 00000046 00000002 cff62f8c cff62f88 00000000 fffcd8eb cff62f4c 
>Feb 16 15:22:10 rh5cluster2 kernel:        cff62f4c 000003e8 00000009 cfd9a560 c9279030 4def62b3 00000054 000024bd 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd9a684 c9204400 00000000 cfdaf580 00000100 00000000 00000000 d08de28d 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08de28d>] qla2x00_loop_resync+0x5d/0xae [qla2xxx]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08d9488>] qla2x00_do_dpc+0x31/0x442 [qla2xxx]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d08d9457>] qla2x00_do_dpc+0x0/0x442 [qla2xxx]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: scsi_wq_1     S 00000003  1832  2951      7          2952  2947 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        c9c4df8c 00000046 cb838848 00000003 cfbea8e0 00000001 cfbea8e0 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 c9c4df64 00000009 ccf45030 cc4f8030 10604a58 00000020 000084d3 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccf45154 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 c9c4dfa8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: fc_wq_1       S 00000003  3436  2952      7          2953  2951 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cff67f8c 00000046 c0546f9e 00000003 cdb09ce0 cbfa88c0 cdb09ce0 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 cff67f64 00000009 ccf45a90 c9279030 46d4bb79 00000054 00005a11 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccf45bb4 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 cff67fa8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0546f9e>] device_release+0x37/0x7c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: fc_dl_1       S 00000086  3784  2953      7          3303  2952 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd1af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd9fc60 
>Feb 16 15:22:10 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd15030 cfd9fa90 887ebba7 0000001f 000004e4 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd15154 c9204400 00000000 cbeffac0 cfd06184 00000000 c042f8b3 cfd1afa8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: smartd        S C042E00A  3136  3072      1          3076  2862 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        c9fc8f38 00000082 c0453f95 c042e00a c9fc8efc 0002ec18 c9fc8efc c920554c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c0439f90 00000005 ce414560 ce414030 940c2758 00000020 00120009 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce414684 c9204400 00000000 cd46ce40 621cda46 000001b9 49c08a46 00000016 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: login         S C0403D2D  2120  3076      1  3217    3077  3072 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd99f54 00000086 c9d55fb8 c0403d2d ce99f560 00000000 c91f1a2c bffb30e0 
>Feb 16 15:22:10 rh5cluster2 kernel:        09d13065 cf8d1ecc 00000008 cfd9f030 ccfe8030 8bb2a884 000000b6 00007638 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd9f154 c9204400 00000000 cd46c200 cd46c200 00000ecc cb4abbfc cfd99f84 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0403d2d>] copy_thread+0x2f/0x1bd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: mingetty      S C05337DB  2952  3077      1          3078  3076 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ce73fed4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000013 caf9a400 00000006 ca6a4a90 ce414030 a07bc494 00000020 000129cf 
>Feb 16 15:22:10 rh5cluster2 kernel:        ca6a4bb4 c9204400 00000000 cd46cac0 00000000 00000065 cfc40800 cf041b40 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: mingetty      S C05337DB  2952  3078      1          3079  3077 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        c9eeaed4 00000086 00000020 c05337db 00000000 00000000 00001000 cfaa7440 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000013 cb838400 00000005 ce414030 cbac9030 a088996b 00000020 00026877 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce414154 c9204400 00000000 cbeff3c0 00000000 00000065 cfc40800 cf002c80 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: mingetty      S C05337DB  2952  3079      1          3080  3078 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd1ced4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000013 cdbfa400 00000004 cb4fa560 cd922030 a0f55035 00000020 0002879a 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb4fa684 c9204400 00000000 cd46c900 00000000 00000065 cfc40800 cbdce680 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: mingetty      S C05337DB  2952  3080      1          3081  3079 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca59fed4 00000082 00000020 c05337db 00000000 00000000 00001000 cec9e0c0 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000013 ce332000 00000003 cfd9fa90 cb4faa90 a1359f74 00000020 00028c54 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfd9fbb4 c9204400 00000000 cd46cc80 00000000 00000065 cfc40800 cfac5e40 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: mingetty      S CF4D7EC0  2952  3081      1          3311  3080 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cf4d7ed4 00000086 00000002 cf4d7ec0 cf4d7ebc 00000000 00001000 cf364600 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000013 cf68e400 00000002 cbac9560 c06ae440 a41b4d7f 00000020 00008d48 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbac9684 c9204400 00000000 cbeffac0 00000000 00000065 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sshd          S CB809B48  1772  3166   2673  3168    3373       (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cb809b5c 00000082 00000002 cb809b48 cb809b44 00000000 c0423f8f 00000004 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000046 00000000 0000000a cbac9a90 cbac9030 72ca7814 00000159 0000bc5d 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbac9bb4 c9204400 00000000 cd46c740 c0420985 00000000 00000000 00000003 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c045007b>] relay_reset+0x5f/0x68
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060e79d>] _spin_lock_irqsave+0x0/0x27
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: bash          S 00000159  1772  3168   3166  3371               (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cb8dff40 00000082 72d4975e 00000159 cd94d128 00000086 ca11be64 c9d4fa04 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 cb8dff28 00000009 cbac9030 cbc3d560 72d4daac 00000159 00029ce2 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbac9154 c9204400 00000000 cd46c580 00000000 00000001 cb8dff4c cb8dff70 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04294f4>] sys_waitpid+0x27/0x2b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: bash          S C9D55EC0  2844  3217   3076                     (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        c9d55ed4 00000082 00000002 c9d55ec0 c9d55ebc 00000000 c0525b0b 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000016 cb112000 00000009 ce99f560 c06ae440 cae63a06 000000c1 0005d72c 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce99f684 c9204400 00000000 cd46c040 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b0b>] tty_ldisc_deref+0x55/0x64
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: clvmd         S CEACBB48  2580  3302      1          3318  3312 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ceacbb5c 00000082 00000002 ceacbb48 ceacbb44 00000000 ce487a90 c9204400 
>Feb 16 15:22:10 rh5cluster2 kernel:        ceacbb24 c0423f8f 00000007 ce487a90 c06ae440 efb096d7 0000016a 00001576 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce487bb4 c9204400 00000000 cce33c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c05a8e0a>] sys_recvfrom+0x11e/0x12b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: clvmd         S CE8EFEDC  3484  3318      1          3319  3302 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ce8eff18 00000082 c043c5c4 ce8efedc c0663bcf 00000003 cc140584 c07a8468 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc4c5400 00000001 0000000a cb4faa90 cc6d6030 2a8f1852 00000133 000033bf 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb4fabb4 c9204400 00000000 cce33c80 d0b7f1b4 0000007b ccc200d8 ce8eff58 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b7f1b4>] __put_lkb+0xba/0xd5 [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b855eb>] device_read+0xa2/0x22b [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b85549>] device_read+0x0/0x22b [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: clvmd         S 00000133  1780  3319      1                3318 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cb603e80 00000082 2a9bdea7 00000133 c047ef4e 00000082 cc248f34 cc248f0c 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cb603e58 00000007 ccc20030 cfd9f560 2abc8320 00000133 00003158 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccc20154 c9204400 00000000 cce33c80 00000000 00000000 00000003 00000286 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046fb37>] vfs_stat_fd+0x19/0x40
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: dlm_astd      S C041FA15  3724  3303      7          3304  2953 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc0f2fb0 00000046 cc0f2f6c c041fa15 00000000 00000003 cfa509b4 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        cfa509b4 00000000 00000009 ce487560 ccde1030 defb2b02 00000147 000001b4 
>Feb 16 15:22:10 rh5cluster2 kernel:        ce487684 c9204400 00000000 cfdaf580 d0b7f125 d0a23923 00000000 d0a238a0 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b7f125>] __put_lkb+0x2b/0xd5 [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a23923>] gdlm_bast+0x83/0x8c [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b7b151>] dlm_astd+0x27/0x11e [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b7b12a>] dlm_astd+0x0/0x11e [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: dlm_scand     S CC508F80  3660  3304      7          3305  3303 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc508f94 00000046 00000002 cc508f80 cc508f7c 00000000 cc541694 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc1cfd80 cc0730a8 0000000a cafcc560 c06ae440 eb0c29d0 0000016a 00002fd2 
>Feb 16 15:22:10 rh5cluster2 kernel:        cafcc684 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b8223c>] dlm_scand+0x0/0x3e [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b8226d>] dlm_scand+0x31/0x3e [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: dlm_recv/0    S 00000003  2576  3305      7          3306  3304 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc572f8c 00000046 00000000 00000003 c08c8960 00000002 c08c8960 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 cc572f64 00000009 cc24ea90 ccde1030 1aabe186 00000148 00001494 
>Feb 16 15:22:10 rh5cluster2 kernel:        cc24ebb4 c9204400 00000000 cce333c0 00000000 00000000 ffffffff cc572fa8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: dlm_send      S 00000003  2944  3306      7          3307  3305 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ccaa0f8c 00000046 00000000 00000003 c08c8760 c91b1720 c08c8760 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 ccaa0f64 0000000a ccde1030 c9279030 1aac2683 00000148 000044fd 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccde1154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff ccaa0fa8 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: dlm_recoverd  S CC5A5F80  3540  3307      7          3333  3306 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc5a5f94 00000046 00000002 cc5a5f80 cc5a5f7c 00000000 d0b82ca2 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        d0b8370a 00000000 0000000a ccc20560 c06ae440 dc2babe1 00000130 00001c92 
>Feb 16 15:22:10 rh5cluster2 kernel:        ccc20684 c9204400 00000000 cc818ac0 00000001 cc4c5400 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b82ca2>] disallow_sync_reply+0xe/0x24 [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b8370a>] dlm_rcom_status+0xb8/0x18e [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b845f0>] dlm_recoverd+0x49/0x4aa [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0b845a7>] dlm_recoverd+0x0/0x4aa [dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: lock_dlm2     D C9204448  2744  3333      7          3338  3307 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cdbe0e6c 00000046 cc834a5c c9204448 cf78d030 c0420b4c c9204400 cf78d030 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000009 c04209fd 00000009 cb3d6a90 c9279030 1ac5aacf 00000148 00000576 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb3d6bb4 c9204400 00000000 cfdaf580 00000001 c9204400 00000000 c9204400 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04209fd>] task_rq_lock+0x31/0x58
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d45f>] wait_for_completion+0x63/0x87
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437485>] kthread_stop+0x4e/0x6c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a23e25>] gdlm_withdraw+0x96/0xad [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a4a492>] gfs2_withdraw_lockproto+0x16/0x30 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a47ada>] gfs2_lm_withdraw+0x6d/0x8d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a58eff>] gfs2_assert_withdraw_i+0x3b/0x4e [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a4ab0a>] gfs2_meta_inval+0x41/0x86 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a45658>] meta_go_inval+0xd/0x17 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a44dfe>] drop_bh+0xab/0x12a [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a44692>] gfs2_glock_cb+0xae/0x11c [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a24769>] gdlm_thread+0x5ac/0x5f9 [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a241bd>] gdlm_thread+0x0/0x5f9 [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gfs2_scand    S CBBACF80  3660  3338      7          3339  3333 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cbbacf94 00000046 00000002 cbbacf80 cbbacf7c 00000000 00000000 cac07fb0 
>Feb 16 15:22:10 rh5cluster2 kernel:        cf7fa360 c0406fb4 0000000a cbc3d030 c06ae440 e5703ff8 00000168 000d125a 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc3d154 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d464>] gfs2_scand+0x0/0x3d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a43b46>] examine_bucket+0x25/0x7b [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d494>] gfs2_scand+0x30/0x3d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gfs2_glockd   S CC68F400  3492  3339      7          3342  3338 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cac07fa8 00000046 ca67f93c cc68f400 00000003 c08c8540 00000000 d0a232b9 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 c08c8540 0000000a cbf9aa90 ccde1030 11281626 0000013f 00005a8e 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbf9abb4 c9204400 00000000 cd46c740 cb49875c cb49875c cac07fb0 cf7fa35c 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a232b9>] gdlm_do_unlock+0x3a/0x4b [lock_dlm]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d440>] gfs2_glockd+0x84/0xa8 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d3bc>] gfs2_glockd+0x0/0xa8 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gfs2_recoverd S CC7DEF80  3660  3342      7          3343  3339 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cc7def94 00000046 00000002 cc7def80 cc7def7c 00000000 c9201700 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000000 cff72030 00000001 cebbba90 c06ae440 6d856cbc 00000165 00000c5d 
>Feb 16 15:22:10 rh5cluster2 kernel:        cebbbbb4 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d4a1>] gfs2_recoverd+0x0/0x3d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d4d1>] gfs2_recoverd+0x30/0x3d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gfs2_logd     D CEA6CE88  3412  3343      7          3344  3342 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cea6ce9c 00000046 00000002 cea6ce88 cea6ce84 00000000 0001c968 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000400 cc4af0ac 0000000a cebbb560 c06ae440 8bad5447 00000149 0000a297 
>Feb 16 15:22:10 rh5cluster2 kernel:        cebbb684 c9204400 00000000 cce333c0 c042e00a cff7775c ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488586>] sync_buffer+0x30/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060dae7>] __wait_on_bit+0x33/0x58
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060db6f>] out_of_line_wait_on_bit+0x63/0x6b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488504>] __wait_on_buffer+0x24/0x27
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0488f22>] sync_dirty_buffer+0x89/0xbe
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a48493>] log_write_header+0x138/0x3b6 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a48a7c>] gfs2_log_flush+0x1d0/0x27c [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d570>] gfs2_logd+0x92/0x11d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d4de>] gfs2_logd+0x0/0x11d [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: gfs2_quotad   S CAD74F70  2540  3344      7                3343 (L-TLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cad74f84 00000046 00000002 cad74f70 cad74f6c 00000000 cffdd240 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel:        d0a4476f cf7fa000 0000000a cebbb030 c06ae440 162a822f 0000016a 00000d29 
>Feb 16 15:22:10 rh5cluster2 kernel:        cebbb154 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a4476f>] gfs2_glock_dq+0x6f/0x79 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d5fb>] gfs2_quotad+0x0/0x129 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a51d15>] gfs2_quota_scan+0x20/0x16a [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:10 rh5cluster2 kernel:  [<d0a3d70e>] gfs2_quotad+0x113/0x129 [gfs2]
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: umount        S CA11BF40  2116  3371   3168  3372               (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ca11bf54 00000086 00000002 ca11bf40 ca11bf3c 00000000 c915804c 485f0018 
>Feb 16 15:22:10 rh5cluster2 kernel:        0e84b065 cac027c0 00000007 cbc3d560 c06ae440 74b2a0b7 00000159 00003ef8 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc3d684 c9204400 00000000 cbe91200 cbe91200 000007c0 ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: umount.gfs2   D CF279DA0  1980  3372   3371                     (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        cf279db4 00000082 00000002 cf279da0 cf279d9c 00000000 c04573ec 000000d0 
>Feb 16 15:22:10 rh5cluster2 kernel:        cf1ba540 cc4af458 00000007 cbc3da90 c06ae440 76852d96 00000159 000b1f66 
>Feb 16 15:22:10 rh5cluster2 kernel:        cbc3dbb4 c9204400 00000000 cbe91740 c042e00a cff7718c ffffffff 00000000 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c04573ec>] __do_page_cache_readahead+0x1a3/0x1bd
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d86>] sync_page+0x0/0x3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451dbe>] sync_page+0x38/0x3b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c060da21>] __wait_on_bit_lock+0x2a/0x52
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451d78>] __lock_page+0x58/0x5e
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0452456>] do_generic_mapping_read+0x1d1/0x43b
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0454274>] generic_file_aio_read+0x173/0x1a3
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0451ba0>] file_read_actor+0x0/0xd1
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046cd64>] do_sync_read+0x0/0x10a
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:10 rh5cluster2 kernel:  =======================
>Feb 16 15:22:10 rh5cluster2 kernel: sshd          S CEB60B48  1772  3373   2673  3375          3166 (NOTLB)
>Feb 16 15:22:10 rh5cluster2 kernel:        ceb60b5c 00000082 00000002 ceb60b48 ceb60b44 00000000 c041ff8c cef49480 
>Feb 16 15:22:10 rh5cluster2 kernel:        00000001 c0406fb4 0000000a cb3d6560 cb3d6030 07e7159e 0000016b 0000b91f 
>Feb 16 15:22:10 rh5cluster2 kernel:        cb3d6684 c9204400 00000000 cbe91580 c0420985 00000000 00000000 00000003 
>Feb 16 15:22:10 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:10 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:11 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:11 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:11 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel:   c9274f8c 00000046 00000002 c9274f78 c9274f74 00000000 c925b5e0 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000001 c9274f64 0000000a c9272560 c06ae440 740b20d4 0000016d 00010bf6 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9272684 c9204400 00000000 cf9b53c0 00000000 c925b5c0 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: khelper       S 00000003  3332     6      1             7     5 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        c9276f8c 00000046 00000000 00000003 c925b4e0 000000d8 c925b4e0 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000001 c9276f64 0000000a c9272030 cafcca90 eba6a1e6 0000001c 000073b1 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9272154 c9204400 00000000 cf9b5040 00000000 00000000 c042f8b3 c9276fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kthread       S 00000003  3328     7      1    44     434     6 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        c927af8c 00000046 cc248ce8 00000003 c92781e0 00000003 c92781e0 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000001 c927af64 00000009 c9279a90 cfd9f560 9a7e0b62 0000013b 00000588 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9279bb4 c9204400 00000000 cbe91740 00000000 00000000 c042f8b3 c927afa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kblockd/0     S 00000003  3412    44      7            45       (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd5af8c 00000046 00000002 00000003 cfd30660 00000000 cfd30660 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000001 cfd5af64 0000000a cfd47560 ce487030 ce18380a 0000015f 00009403 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd47684 c9204400 00000000 cbe913c0 00000000 00000000 ffffffff cfd5afa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kacpid        S C0406FB4  3784    45      7           137    44 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd5ef8c 00000046 00000000 c0406fb4 c0422130 00000000 00000000 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd5efbc 00000000 00000001 cfd47030 c9272030 5926fa42 0000000a 00001ae0 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd47154 c9204400 00000000 c06ae280 cfd056c4 00000000 c042f8b3 cfd5efa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: cqueue/0      S 00000086  3784   137      7           138    45 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd25f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24560 c9279a90 5aa26eca 0000000a 000008de 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd24684 c9204400 00000000 c06ae280 cfd05184 00000000 c042f8b3 cfd25fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: ksuspend_usbd S 00000086  3784   138      7           141   137 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd28f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24030 c9272030 5aa310e1 0000000a 00000567 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd24154 c9204400 00000000 c06ae280 c92bac04 00000000 c042f8b3 cfd28fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: khubd         S 00000004  3456   141      7           143   138 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd31f3c 00000046 cfde2660 00000004 c05788f0 00049544 000000a3 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        80000180 cfdf9400 0000000a cfd32a90 cfd15030 fe28ddce 0000000a 00001d24 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd32bb4 c9204400 00000000 cfdafac0 00000004 000003e8 cfd31f9c c06cf940 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05788f0>] usb_control_msg+0xc5/0xcf
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c057676a>] hub_thread+0x940/0x9a7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0575e2a>] hub_thread+0x0/0x9a7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kseriod       S CFD46F7C  3176   143      7           166   141 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd46f90 00000046 00000002 cfd46f7c cfd46f78 00000000 00000000 c054941b 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000004 c06d0a4c 0000000a cfd47a90 c06ae440 eaa67105 0000000a 00000d93 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd47bb4 c9204400 00000000 cfdafc80 c06d089c c06d08a0 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c054941b>] driver_probe_device+0x44/0xa5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0587f66>] serio_thread+0x25b/0x2a9
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0587d0b>] serio_thread+0x0/0x2a9
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: pdflush       S CFD71F88  3668   166      7           167   143 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd71f9c 00000046 00000002 cfd71f88 cfd71f84 00000000 c07a5c00 00000286 
>Feb 16 15:22:20 rh5cluster2 kernel:        c042ef10 00000000 0000000a cfd6e560 c06ae440 19e968fa 00000133 00001974 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd6e684 c9204400 00000000 cf9b5740 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04570ae>] pdflush+0xb0/0x19f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: pdflush       D CFD74EEC  2420   167      7           168   166 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd74f00 00000046 00000002 cfd74eec cfd74ee8 00000000 c07a5c00 c04058a4 
>Feb 16 15:22:20 rh5cluster2 kernel:        c07a5c00 00000286 0000000a cfd75a90 c06ae440 a199fe94 0000016d 00000d26 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd75bb4 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d504>] io_schedule_timeout+0x3f/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045a78e>] congestion_wait+0x50/0x64
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0456d64>] wb_kupdate+0x98/0xde
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457107>] pdflush+0x109/0x19f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0456ccc>] wb_kupdate+0x0/0xde
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kswapd0       S C9258D34  2512   168      7           169   167 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd2af48 00000046 c0439f90 c9258d34 00000246 c0450a41 00000000 00039957 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9258d4c 00000064 0000000a cfd2ba90 cfd75a90 771b5166 0000016b 0000079a 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd2bbb4 c9204400 00000000 cf9b5200 c045a799 00000000 cfd2af98 c06b87d8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045a799>] congestion_wait+0x5b/0x64
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04594e2>] kswapd+0xb4/0x3d5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045942e>] kswapd+0x0/0x3d5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aio/0         S 00000086  3784   169      7           329   168 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd33f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd32560 c9272030 6ae0b51c 0000000a 00000d4e 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd32684 c9204400 00000000 c06ae280 cffc7c04 00000000 c042f8b3 cfd33fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kpsmoused     S 00000086  3784   329      7           362   169 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd18f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000009 cfd15560 cfb73a90 e3191df7 0000000a 00000b35 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd15684 c9204400 00000000 c06ae280 cffc7184 00000000 c042f8b3 cfd18fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kmirrord      S 00000086  3784   362      7           367   329 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cff9af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cff7b730 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd6e030 cff7b560 8757d833 0000000e 0000097a 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd6e154 c9204400 00000000 cfdaf900 cfc3d184 00000000 c042f8b3 cff9afa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: ksnapd        S 00000086  3784   367      7           370   362 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cff73f8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd75200 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9201700 00000000 00000005 cff7b030 cfd75030 877b12a3 0000000e 0000095c 
>Feb 16 15:22:20 rh5cluster2 kernel:        cff7b154 c9204400 00000000 cfdaf900 cfc3d6c4 00000000 c042f8b3 cff73fa8 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kjournald     S 00000004  2364   370      7           400   367 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd93f9c 00000046 c0423f8f 00000004 00000046 cfdf1340 00000000 cff98800 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000000 0000000a cff72030 cfd75a90 786e3992 0000016b 00000cec 
>Feb 16 15:22:20 rh5cluster2 kernel:        cff72154 c9204400 00000000 cf9b5200 c0420985 00000000 cfd93fac cf406a68 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kauditd       S CFF70F60  3700   400      7          1294   370 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cff70fa0 00000046 00000001 cff70f60 c041fa15 00000000 00000001 cb71d218 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 cb71d218 0000000a cfb73a90 cfd15a90 8ddbdcb4 0000015e 00001510 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfb73bb4 c9204400 00000000 cbe913c0 c0420985 00000000 00000000 cff70fb0 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044986d>] kauditd_thread+0x103/0x132
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044976a>] kauditd_thread+0x0/0x132
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: udevd         S CFF78B48  2580   434      1          1977     7 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cff78b5c 00000086 00000002 cff78b48 cff78b44 00000000 00000004 cff78b18 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000011 c044f208 00000004 c9279030 c06ae440 1b53b7b6 00000148 0001b087 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9279154 c9204400 00000000 cfdaf3c0 00000203 00000200 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c048fb82>] inotify_poll+0x3a/0x40
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04dedbb>] number+0x105/0x24d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047007b>] copy_strings+0xe7/0x1b9
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04376c6>] bit_waitqueue+0x16/0x48
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04824fc>] seq_printf+0x2e/0x4b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c049f22a>] show_stat+0x9ec/0x9f9
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0483a12>] simple_empty+0xe/0x50
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04293f5>] do_wait+0x8ba/0x95e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: kjournald     S CFD16F88  3184  1294      7          2941   400 (L-TLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd16f9c 00000046 00000002 cfd16f88 cfd16f84 00000000 00000000 cfa56000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000000 00000002 c9279560 c06ae440 4b25b168 0000001a 0000132b 
>Feb 16 15:22:20 rh5cluster2 kernel:        c9279684 c9204400 00000000 cce333c0 c0420985 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: dhclient      S CFD63B48  2476  1977      1          2106   434 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd63b5c 00000082 00000002 cfd63b48 cfd63b44 00000000 cc130e08 d0859b10 
>Feb 16 15:22:20 rh5cluster2 kernel:        0063c853 00000000 0000000a ccea3030 c06ae440 946642af 00000018 00007ef7 
>Feb 16 15:22:20 rh5cluster2 kernel:        ccea3154 c9204400 00000000 cf9b5c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457b8d>] activate_page+0x61/0x84
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457d23>] mark_page_accessed+0x1c/0x30
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: auditd        S CCAC9B48  2476  2106      1  2108    2107  1977 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        ccac9b5c 00000086 00000002 ccac9b48 ccac9b44 00000000 c041ff8c d0859b10 
>Feb 16 15:22:20 rh5cluster2 kernel:        0063d05f ccac9bec 00000007 cfd15a90 c06ae440 8e90601b 0000016c 000030d3 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd15bb4 c9204400 00000000 cfdaf200 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477612>] poll_freewait+0x18/0x4c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477948>] do_sys_poll+0x302/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04224cb>] __cond_resched+0x16/0x34
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05c4e5b>] netlink_unicast+0x171/0x1a9
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05c45d2>] netlink_sendskb+0x19/0x30
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05c5012>] netlink_ack+0x17f/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044973e>] audit_receive+0x944/0x970
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043b209>] wake_futex+0x42/0x4c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046d3c6>] do_readv_writev+0x17d/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a75c7>] sock_aio_write+0x0/0x102
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: auditd        S CFC562BC  2176  2107      1          2127  2106 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cc8a0e80 00000086 000000cf cfc562bc cfd2b030 0000000a cc8a0000 cc8a0f64 
>Feb 16 15:22:20 rh5cluster2 kernel:        cc8a0f64 cc8a0ee0 00000007 cfd2b030 cb3d6030 8dddc5e1 0000015e 00016596 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd2b154 c9204400 00000000 cfdaf200 cfdaf200 c043b07d cc8a0f48 c0419ccd 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e3ca3>] percpu_counter_sum+0x37/0x52
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046b6df>] vfs_statfs+0x47/0x5f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046c5ce>] sys_fstatfs+0x4b/0x54
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: python        S CFF60B48  1820  2108   2106                     (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cff60b5c 00200082 00000002 cff60b48 cff60b44 00000000 0000019a 02cf6a5d 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000075 00000002 cfd6ea90 c06ae440 8e91f09a 0000015e 000381b1 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd6ebb4 c9204400 00000000 cf9b5e40 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c06074c2>] unix_write_space+0x12/0x71
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: syslogd       S CCFF0B48  1732  2127      1          2130  2107 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        ccff0b5c 00000086 00000002 ccff0b48 ccff0b44 00000000 00000000 cff586a4 
>Feb 16 15:22:20 rh5cluster2 kernel:        c04d96ce 00000001 00000007 ccfe8030 c06ae440 786e5bfc 0000016b 000017c7 
>Feb 16 15:22:20 rh5cluster2 kernel:        ccfe8154 c9204400 00000000 cf9b5200 c04d021f cff4cb30 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04d96ce>] cfq_insert_request+0xa5/0x391
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04d021f>] elv_insert+0x146/0x1ec
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ae8c5>] datagram_poll+0x14/0xb6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0606c21>] unix_dgram_recvmsg+0x222/0x239
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045440d>] mempool_alloc+0x28/0xc6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457888>] pagevec_lookup_tag+0x24/0x2b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04528d1>] wait_on_page_writeback_range+0xaf/0xf8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d08596cf>] journal_stop+0x1b6/0x1c2 [jbd]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04563c7>] mapping_tagged+0x2b/0x32
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0451ca9>] find_get_pages_tag+0x38/0x77
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0457888>] pagevec_lookup_tag+0x24/0x2b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04528d1>] wait_on_page_writeback_range+0xaf/0xf8
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: portmap       S 00000046  2488  2183      1          2222  2130 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cceeebfc 00000082 c9205900 00000046 00000046 c9205880 00000246 00000246 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfa57d80 c060e744 00000002 cfd75560 cfd75030 de9fcb2d 00000018 000087fb 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd75684 c9204400 00000000 cce33e40 cfa57d80 c06c9600 cceeec68 cb71d998 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a826f>] sys_sendmsg+0x192/0x1f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8c6a>] sys_recvmsg+0x14b/0x1cd
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b2666>] netdev_run_todo+0x1ec/0x1f3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05eedea>] inet_sock_destruct+0x16a/0x1b0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e862>] _write_lock_bh+0x8/0x1a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: rpc.statd     S CC908B48  1600  2222      1          2265  2183 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cc908b5c 00000082 00000002 cc908b48 cc908b44 00000000 00ef8300 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 cf4606c0 00000001 cfd75030 c06ae440 dea35352 00000018 00003bd7 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd75154 c9204400 00000000 cce33580 03a57d00 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: rpc.idmapd    S CC990F20  3064  2265      1          2359  2222 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cc990f34 00200086 00000002 cc990f20 cc990f1c 00000000 00000000 bf8329cc 
>Feb 16 15:22:20 rh5cluster2 kernel:        cc990f60 cf9806bc 0000000a cff7ba90 c06ae440 7078d954 0000016d 00006bcb 
>Feb 16 15:22:20 rh5cluster2 kernel:        cff7bbb4 c9204400 00000000 cf9b53c0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c049043f>] sys_epoll_wait+0x13e/0x37f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: ccsd          S CA943B48  2364  2359      1          2360  2265 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        ca943b5c 00000086 00000002 ca943b48 ca943b44 00000000 0000003e 0b19380a 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000365 00000007 cfd9aa90 cfd9f560 94ab951a 0000013b 00005f7a 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd9abb4 c9204400 00000000 cce33ac0 0000012d 00000000 ca943c0c caf5e218 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0ecaad3>] ip6_push_pending_frames+0x34c/0x3d6 [ipv6]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05acc76>] skb_queue_purge+0x11/0x17
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060666d>] unix_sock_destructor+0xe/0xd2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05aa1d3>] sk_free+0xb2/0xc7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: ccsd          S C9D7FB48  2260  2360      1          2365  2359 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        c9d7fb5c 00000086 00000002 c9d7fb48 c9d7fb44 00000000 00000086 c042ee07 
>Feb 16 15:22:20 rh5cluster2 kernel:        cff4cb9c 00000000 00000008 cfb73560 c06ae440 e564e91c 000000b2 0001c961 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfb73684 c9204400 00000000 cce33ac0 00000003 c04d2cce ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04d2cce>] __generic_unplug_device+0x1d/0x1f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0908098>] tg3_start_xmit_dma_bug+0x581/0x64f [tg3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ac932>] skb_clone+0x3d/0x1f1
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060a167>] packet_rcv_spkt+0x0/0x139
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b127c>] dev_hard_start_xmit+0x1bc/0x21b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05e36f7>] tcp_v4_send_check+0x74/0xaa
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05d6e05>] tcp_close+0x504/0x523
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S CC991BE8  2104  2365      1          2366  2360 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cc991bfc 00000086 00000002 cc991be8 cc991be4 00000000 00000000 00000036 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000001 00000000 00000001 cff72560 c06ae440 9b5eda52 0000016d 0000b12b 
>Feb 16 15:22:20 rh5cluster2 kernel:        cff72684 c9204400 00000000 cce333c0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 last message repeated 10 times
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8b07>] move_addr_to_user+0x50/0x68
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S CAFC7BE8  2740  2366      1          2367  2365 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cafc7bfc 00000086 00000002 cafc7be8 cafc7be4 00000000 00000000 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000000 00000001 cfb73030 c06ae440 4a55bda4 00000019 00005293 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfb73154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S C992BE6C  3024  2367      1          2374  2366 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        c992be80 00000086 00000002 c992be6c c992be68 00000000 cfd878c0 cff2b114 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd878c0 cfd87920 00000006 cfd32030 c06ae440 842192fd 00000057 00002982 
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd32154 c9204400 00000000 cce333c0 c047513c c992bd90 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S CFD9BBE8  2740  2374      1          2375  2367 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cfd9bbfc 00000086 00000002 cfd9bbe8 cfd9bbe4 00000000 00000000 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000000 00000001 ccea3560 c06ae440 ed7134ee 00000157 000004ff 
>Feb 16 15:22:20 rh5cluster2 kernel:        ccea3684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S CD78DBE8  2688  2375      1          2373  2374 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cd78dbfc 00000086 00000002 cd78dbe8 cd78dbe4 00000000 000001cc 00000001 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 cd78dbc0 00000001 ccfe8560 cc24ea90 1aabccf2 00000148 000004a7 
>Feb 16 15:22:20 rh5cluster2 kernel:        ccfe8684 c9204400 00000000 cce333c0 c04057e7 c0a3f680 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:20 rh5cluster2 kernel:  =======================
>Feb 16 15:22:20 rh5cluster2 kernel: aisexec       S CD073BE8  2740  2394      1          2395  2387 (NOTLB)
>Feb 16 15:22:20 rh5cluster2 kernel:        cd073bfc 00000086 00000002 cd073be8 cd073be4 00000000 00000000 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel:        00000000 00000000 00000001 cd117560 ce487030 1ae0cde0 00000148 000007ea 
>Feb 16 15:22:20 rh5cluster2 kernel:        cd117684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:20 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:20 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S CCDC2BE8  2740  2395      1          2393  2394 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ccdc2bfc 00000086 00000002 ccdc2be8 ccdc2be4 00000000 00000194 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1a90 c06ae440 1a7d9fb8 00000148 00000406 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccde1bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S CCEA0000  2784  2396      1          2397  2393 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ccdc4bfc 00000086 00000000 ccea0000 cfacba80 00000000 000000b1 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1560 cfd9f560 93ff7cdd 0000013b 00000590 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccde1684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S 00000000  2868  2397      1          2400  2396 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c08c7bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 ccfe8a90 cff72560 9d870693 00000019 00000a7a 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccfe8bb4 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S 00000000  2868  2400      1          2401  2397 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cd093bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 cd117030 cfd2b560 0348822d 00000058 00000802 
>Feb 16 15:22:21 rh5cluster2 kernel:        cd117154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S CCEA0000  2800  2401      1          2419  2400 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc4c4bfc 00000086 00000000 ccea0000 cfa51c80 00000000 000000c2 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 ccadb560 cfd2ba90 dc3cfdfa 00000019 000004e0 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccadb684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e007b>] __copy_to_user_ll+0x32/0xd6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S 00000000  2868  3311      1          3312  3081 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ccb6ebfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 cafcc030 c9279030 d3e6a84d 00000130 00000758 
>Feb 16 15:22:21 rh5cluster2 kernel:        cafcc154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: aisexec       S CCEA0000  2800  3312      1          3302  3311 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc59cbfc 00000086 00000000 ccea0000 cfa47980 00000000 00000019 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 ccc20a90 c9279030 d3e077b4 00000130 000003ca 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccc20bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: groupd        S CC8A9BE8  2476  2373      1          2381  2375 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc8a9bfc 00000082 00000002 cc8a9be8 cc8a9be4 00000000 cff4cb30 00000020 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000046 c0406fb4 00000002 cff7b560 ccea3560 ed712fef 00000157 00003fc5 
>Feb 16 15:22:21 rh5cluster2 kernel:        cff7b684 c9204400 00000000 cce33040 c062ad80 cc8a9ea4 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 7 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: fenced        S C0A5FBE8  2476  2381      1          2387  2373 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c0a5fbfc 00000086 00000002 c0a5fbe8 c0a5fbe4 00000000 cff4cb30 00000008 
>Feb 16 15:22:21 rh5cluster2 kernel:        00300000 00000000 00000009 cfd2b560 c06ae440 036ac9b9 00000058 00002bf0 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd2b684 c9204400 00000000 cce33200 00001000 00000001 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: dlm_controld  S CFF72A90  2476  2387      1          2394  2381 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        caef8bfc 00000082 00000117 cff72a90 c9204400 caef8bb8 c0423f8f 00000046 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000046 00000000 00000001 cff72a90 cb3d6a90 1ac52279 00000148 0000612f 
>Feb 16 15:22:21 rh5cluster2 kernel:        cff72bb4 c9204400 00000000 cfdaf580 0807b008 caef8e9c 00000018 c04057e7 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 2 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ae166>] memcpy_toiovec+0x27/0x4a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05c59dd>] netlink_recvmsg+0x285/0x2a4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: gfs_controld  S 00000012  2184  2393      1          2396  2395 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cd784bfc 00000082 00000000 00000012 00000000 00000100 cceb5b54 08262176 
>Feb 16 15:22:21 rh5cluster2 kernel:        cd117a90 00000003 00000001 cd117a90 cd117560 1ae0c5f6 00000148 000082a4 
>Feb 16 15:22:21 rh5cluster2 kernel:        cd117bb4 c9204400 00000000 cfdaf740 0000000e 0000000b 00000001 cd784ebc 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 5 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: dbus-daemon   S CCED6BE8  2308  2419      1          2474  2401 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cced6bfc 00000082 00000002 cced6be8 cced6be4 00000000 cff4cb30 00000008 
>Feb 16 15:22:21 rh5cluster2 kernel:        00300000 ccadba90 00000009 ccadba90 c06ae440 6b9635d1 00000020 0000c09e 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccadbbb4 c9204400 00000000 cfdafc80 00001000 00000001 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 5 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060da42>] __wait_on_bit_lock+0x4b/0x52
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hcid          S CA93ABE8  2128  2474      1          2487  2419 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca93abfc 00000086 00000002 ca93abe8 ca93abe4 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        cf4606c0 cbc75e00 00000001 cfd24a90 c06ae440 c8e71f0d 0000001c 0002c8d2 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd24bb4 c9204400 00000000 cfdafac0 ca93ac0c ffffffff ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: sdpd          S 00000000  2660  2487      1          2498  2474 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc21b28 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 cbc98a90 ccadb030 c5a36808 0000001c 0004ebc9 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc98bb4 c9204400 00000000 cf9b5580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478111>] sys_pselect7+0xdd/0x208
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782a9>] sys_pselect6+0x6d/0x79
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: krfcommd      S CBC38680  3480  2498      1          2547  2487 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc6aaf20 00000046 00000000 cbc38680 d0a2b75f 00000000 ffffffea d0a90079 
>Feb 16 15:22:21 rh5cluster2 kernel:        c924fa00 00000286 00000005 cc24e560 ccea3a90 cfe5f3ca 0000001c 00002ad9 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc24e684 c9204400 00000000 cf9b5740 00000000 00000246 00000000 c992d4c0 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0a2b75f>] l2cap_sock_create+0x64/0x6b [l2cap]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0a90079>] bt_sock_create+0x79/0x8f [bluetooth]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0b28dad>] rfcomm_run+0x184/0x1097 [rfcomm]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cfcc>] audit_syscall_exit+0x294/0x2b0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0407880>] do_syscall_trace+0x57/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0b28c29>] rfcomm_run+0x0/0x1097 [rfcomm]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: pcscd         S D084A4C7  2108  2547      1          2551  2498 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbcfdb5c 00000082 d0845080 d084a4c7 cd276e80 c048b0ae cff4c02c cd276e80 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000001 00001000 00000001 cc24e030 ccbb9560 e429912c 0000001c 0004fdce 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc24e154 c9204400 00000000 cf9b5740 d084aff0 00000000 00000001 00000008 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060db04>] __wait_on_bit+0x50/0x58
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d085d1f8>] find_revoke_record+0x42/0x6c [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d085d6c8>] journal_cancel_revoke+0x7a/0x9c [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d085dce1>] journal_add_journal_head+0x1c/0x136 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04572cd>] __do_page_cache_readahead+0x84/0x1bd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0452679>] do_generic_mapping_read+0x3f4/0x43b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04ddfdc>] rb_insert_color+0x8c/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045ecbd>] __vma_link+0x44/0x4f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04243cb>] sched_fork+0x73/0xaa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0425dbf>] copy_process+0xefa/0x10b1
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0424753>] wake_up_new_task+0x1af/0x1b7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: pcscd         S CFD0EF24  3188  2551      1          2572  2547 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd0ef38 00000082 00000002 cfd0ef24 cfd0ef20 00000000 cfd0eefc c920554c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 c0439f90 00000007 ccbb9560 c06ae440 88beba2b 0000016d 0001a912 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccbb9684 c9204400 00000000 cf9b5740 0c6df749 00000164 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hidd          S 00000000  2812  2572      1          2592  2551 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cf969bc0 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 00000001 cba58560 ccadb030 f010d4e3 0000001c 000085f6 
>Feb 16 15:22:21 rh5cluster2 kernel:        cba58684 c9204400 00000000 cbeffe40 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0a903a5>] bt_sock_poll+0x14/0xf4 [bluetooth]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047843a>] sys_ppoll+0xd9/0x1fc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: automount     S CFBAFC00  2104  2592      1          2593  2572 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc60ed8 00000082 00000000 cfbafc00 c92a6800 00000004 c9177220 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfbafc00 c0472500 00000001 cbc98030 cba58030 08c2edd5 0000001d 00007fd4 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc98154 c9204400 00000000 cfdafe40 c04ddf9c 00000004 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472500>] pipe_write+0x393/0x3ec
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04ddf9c>] rb_insert_color+0x4c/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0430d0d>] dequeue_signal+0x36/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04319fe>] sys_rt_sigtimedwait+0x1cd/0x2a8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: automount     S CC50AE6C  3384  2593      1          2594  2592 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc50ae80 00000082 00000002 cc50ae6c cc50ae68 00000000 cb81cf34 cb81cf0c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 cc50ae58 00000007 cafcca90 c06ae440 06e137f7 00000164 000014c4 
>Feb 16 15:22:21 rh5cluster2 kernel:        cafccbb4 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: automount     S CB81CE6C  2316  2594      1          2597  2593 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cb81ce80 00000082 00000002 cb81ce6c cb81ce68 00000000 cf764f34 cf764f0c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000254 c06b6280 0000000a ccbb9030 c06ae440 8e9182c2 0000016d 00003326 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccbb9154 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: automount     S 00000086  2376  2597      1          2600  2594 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cba67bfc 00000082 c0564b78 00000086 c042ee07 cfb76200 c07a5c00 00000086 
>Feb 16 15:22:21 rh5cluster2 kernel:        c042ef10 00000000 0000000a cba58a90 ccbb9030 8a40c87b 00000160 000020be 
>Feb 16 15:22:21 rh5cluster2 kernel:        cba58bb4 c9204400 00000000 cfdafe40 cba67c70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0564b78>] ide_build_sglist+0x2b/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05645da>] ide_dma_exec_cmd+0x1f/0x22
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: automount     S CFF4C02C  2184  2600      1          2616  2597 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cfc4bbfc 00000082 c048b0ae cff4c02c cbc75500 00000001 00001000 030c0020 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 00000000 0000000a cba58030 ccbb9030 06e0b73b 00000164 00002206 
>Feb 16 15:22:21 rh5cluster2 kernel:        cba58154 c9204400 00000000 cfdafe40 cfc4bc70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: acpid         S CC50BBE8  2108  2616      1          2632  2600 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc50bbfc 00000082 00000002 cc50bbe8 cc50bbe4 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        cf4606c0 cbc75480 00000007 ccadb030 ca6a4560 e17f9d47 0000001e 00020054 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccadb154 c9204400 00000000 cf9b5040 00001000 00000000 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08596cf>] journal_stop+0x1b6/0x1c2 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891bde>] ext3_journal_dirty_data+0x0/0x32 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453322>] generic_file_buffered_write+0x4ca/0x5f8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453a08>] generic_file_aio_write+0x5e/0xb3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d088ffdc>] ext3_file_write+0x24/0x8f [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hpiod         S 3537FFFF  2476  2632      1          2637  2616 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cadf9e3c 00200086 00000000 3537ffff 00003736 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 cbc98560 00000001 cbc98560 ccbb9a90 186c89d0 0000001d 00005699 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc98684 c9204400 00000000 cf9b5900 cadf9e94 cadf9e95 cadf9e94 ffffffff 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9d7f>] lock_sock_nested+0x8a/0x92
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d4088>] inet_csk_accept+0x9b/0x200
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b7de>] d_instantiate+0x5c/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05eebea>] inet_accept+0x1f/0xb5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a90a0>] sys_accept+0xee/0x1c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a89ec>] sys_connect+0x82/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9248>] sys_socketcall+0xd6/0x261
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: python        S CA721B48  2476  2637      1          2654  2632 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca721b5c 00200086 00000002 ca721b48 ca721b44 00000000 0003ea1c c04057e7 
>Feb 16 15:22:21 rh5cluster2 kernel:        c0406fb4 00200286 0000000a cbea7560 c06ae440 1e4616d5 0000016d 00016990 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbea7684 c9204400 00000000 cbeff900 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: cupsd         S CC31EB48  2040  2654      1          2673  2637 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cc31eb5c 00200082 00000002 cc31eb48 cc31eb44 00000000 cc130c68 d0859b10 
>Feb 16 15:22:21 rh5cluster2 kernel:        0063f04f 00000000 00000007 cbea7030 c06ae440 9a21bfcf 0000001d 00008dde 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbea7154 c9204400 00000000 cce33740 cc31eb6c ffffffff ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ff5c>] signal_wake_up+0x1e/0x2c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04301fb>] __group_send_sig_info+0x74/0x7f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04313fc>] sys_kill+0xee/0xfa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: sshd          S CB443B48  2476  2673      1  3166    2699  2654 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cb443b5c 00200082 00000002 cb443b48 cb443b44 00000000 cb203a2c d0859b10 
>Feb 16 15:22:21 rh5cluster2 kernel:        00638802 00000000 00000007 cbea7a90 c06ae440 8c927a69 0000015e 000067b3 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbea7bb4 c9204400 00000000 cbeffc80 d0891698 00001000 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437720>] __wake_up_bit+0x28/0x33
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0461d88>] anon_vma_prepare+0x11/0xa5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: sendmail      S CBC37B48  2476  2699      1          2707  2673 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbc37b5c 00200082 00000002 cbc37b48 cbc37b44 00000000 cb203a2c d0859b10 
>Feb 16 15:22:21 rh5cluster2 kernel:        0063d888 00000000 0000000a cc891a90 c06ae440 7d9f6873 0000016d 0001cf79 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc891bb4 c9204400 00000000 cbeff740 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0524a91>] extract_buf+0x9d/0xce
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0499ce9>] proc_alloc_inode+0x3e/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047cff4>] iget_locked+0x8b/0xfb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0499d9c>] proc_get_inode+0x8e/0x116
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c049cf16>] proc_lookup+0x72/0xa4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0499f7f>] proc_root_lookup+0xe/0x26
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089ebbe>] ext3_permission+0x0/0xa [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04df806>] sprintf+0x20/0x23
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c049e6a7>] loadavg_read_proc+0xce/0xfa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e8e1>] lock_kernel+0x14/0x2e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0499eb3>] proc_delete_inode+0x0/0x41
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: sendmail      S CA26FF98  2476  2707      1          2724  2699 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca26ffac 00200086 00000002 ca26ff98 ca26ff94 00000000 00000000 00000002 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000002 caf9a000 00000001 cc891030 c06ae440 ab35b889 0000001d 0000e0f1 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc891154 c9204400 00000000 cf9b5ac0 ca26ffb8 c040794d ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043157e>] sys_pause+0x11/0x17
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: gpm           S CDBD39B0  2476  2724      1          2740  2707 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cafaeb5c 00000082 00000000 cdbd39b0 00000000 cdbd39b0 cb203a2c d0859b10 
>Feb 16 15:22:21 rh5cluster2 kernel:        0063a001 00000000 00000001 cc4f8560 cc4f8030 b193838a 0000001d 0001d5e7 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc4f8684 c9204400 00000000 cbeff580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042d3ac>] __capable+0xc/0x1f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052c256>] vt_ioctl+0x555/0x148f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052bd01>] vt_ioctl+0x0/0x148f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047652a>] fasync_helper+0x3c/0xae
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046dbf7>] file_kill+0x11/0x2a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046bc23>] do_filp_open+0x32/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: crond         S CA6D8F24  2128  2740      1          2778  2724 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca6d8f38 00000086 00000002 ca6d8f24 ca6d8f20 00000000 ca6d8efc c920554c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 c0439f90 00000007 cc891560 c06ae440 3de6c644 00000169 0001c2db 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc891684 c9204400 00000000 cbeff200 7c5f7fa7 0000016d ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: xfs           S CA8F5B48  1936  2778      1          2809  2740 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca8f5b5c 00000082 00000002 ca8f5b48 ca8f5b44 00000000 005d8030 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 cf4606c0 00000007 cb2b6a90 c06ae440 b481b55d 00000134 0000407d 
>Feb 16 15:22:21 rh5cluster2 kernel:        cb2b6bb4 c9204400 00000000 cc818740 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0488721>] ll_rw_block+0x84/0x93
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04555cd>] free_pages_bulk+0x100/0x16e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04557f4>] __pagevec_free+0x14/0x1a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0457dc8>] __pagevec_release_nonlru+0x44/0x4f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0458a9f>] remove_mapping+0x84/0xa7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04590b7>] shrink_inactive_list+0x5f5/0x6c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04585e7>] isolate_lru_pages+0x64/0x7d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0457d7c>] __pagevec_release+0x15/0x1d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0458986>] shrink_active_list+0x386/0x38e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0475e9c>] open_namei+0x6e/0x58d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046bc16>] do_filp_open+0x25/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: atd           S CF934F24  2476  2809      1          2828  2778 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cf934f38 00000082 00000002 cf934f24 cf934f20 00000000 cf934efc c920554c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 c0439f90 00000007 cb2b6030 c06ae440 df6b0d4e 00000134 00009134 
>Feb 16 15:22:21 rh5cluster2 kernel:        cb2b6154 c9204400 00000000 cc818c80 e7fb5841 00000170 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: yum-updatesd  S CA666BE8  1640  2828      1          2845  2809 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca666bfc 00200086 00000002 ca666be8 ca666be4 00000000 ca666fac 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        c04057e7 ca666c0c 0000000a cb2b6560 c06ae440 9dbc80a7 0000016d 000050bd 
>Feb 16 15:22:21 rh5cluster2 kernel:        cb2b6684 c9204400 00000000 cc818ac0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048b4a7>] bio_add_page+0x31/0x3a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045eba5>] find_extend_vma+0x12/0x49
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043007b>] __group_complete_signal+0x111/0x21d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043c05e>] do_futex+0x202/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: avahi-daemon  S CAFB3BE8  2072  2845      1  2846    2862  2828 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cafb3bfc 00000082 00000002 cafb3be8 cafb3be4 00000000 cfa44480 cafb3c30 
>Feb 16 15:22:21 rh5cluster2 kernel:        c05c71fc ccea0000 0000000a cc4f8a90 c06ae440 a40ceca6 00000079 0000d2fe 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc4f8bb4 c9204400 00000000 cc818e40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05c71fc>] nf_iterate+0x38/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 8 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a965d>] sock_common_recvmsg+0x3e/0x54
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e95fe>] udp_ioctl+0x5a/0x5e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e95a4>] udp_ioctl+0x0/0x5e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: avahi-daemon  S CE1F2DB8  3204  2846   2845                     (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ce1f2dcc 00000082 00000002 ce1f2db8 ce1f2db4 00000000 00000001 cf4987dc 
>Feb 16 15:22:21 rh5cluster2 kernel:        cf9adc9c 00000000 00000001 ccf45560 c06ae440 8ba94daf 0000001e 000007a2 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccf45684 c9204400 00000000 cbeff040 c0472eca ccf45560 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald          S CBA74BE8  1632  2862      1  2863    3072  2845 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cba74bfc 00000086 00000002 cba74be8 cba74be4 00000000 00000000 00000801 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfbe8318 00000000 00000008 cc4f8030 c06ae440 8977206d 0000016d 00035954 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc4f8154 c9204400 00000000 cc818900 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 11 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d089e901>] ext3_get_acl+0x51/0x2ae [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0475069>] __link_path_walk+0xc02/0xc2c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-runner   S CBEE3BE8  2068  2863   2862  2869               (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbee3bfc 00000086 00000002 cbee3be8 cbee3be4 00000000 00000001 cbc776e8 
>Feb 16 15:22:21 rh5cluster2 kernel:        cff76000 cff76000 00000003 ccbb9a90 cbac9560 50819f84 00000020 0000e820 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccbb9bb4 c9204400 00000000 cc818200 cbee3c0c ffffffff 00000286 c042ee65 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ee65>] try_to_del_timer_sync+0x44/0x4a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-addon-ac S CCFA63EC  2172  2869   2863          2873       (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ce1d5dcc 00000082 cc130c68 ccfa63ec d085ab14 cf406600 000000c7 d0891698 
>Feb 16 15:22:21 rh5cluster2 kernel:        00001000 00000000 00000004 ca6a4560 cc4f8030 e17fea4b 0000001e 00004d04 
>Feb 16 15:22:21 rh5cluster2 kernel:        ca6a4684 c9204400 00000000 cc818040 00000000 00000003 00000282 00000246 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0460254>] do_mmap_pgoff+0x57b/0x6db
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-addon-ke S CBF1BF3C  2172  2873   2863          2881  2869 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbf1bf50 00000082 00000002 cbf1bf3c cbf1bf38 00000000 ca6a4030 cbf1bf24 
>Feb 16 15:22:21 rh5cluster2 kernel:        c0420d86 ca6a4030 0000000a ca6a4030 c06ae440 cfbdb7a0 000000c1 00003a31 
>Feb 16 15:22:21 rh5cluster2 kernel:        ca6a4154 c9204400 00000000 cbe91e40 00000046 00000000 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420d86>] task_running_tick+0x22a/0x234
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-addon-ke S 00000000  2868  2881   2863          2884  2873 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ce9a0f50 00000086 cc8183c0 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 ceb4c710 00000001 ce99fa90 ccbb9a90 136f0b55 0000001f 00089ea0 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce99fbb4 c9204400 00000000 cc8183c0 c06bc760 00100073 ce9a0f58 cf8a4c00 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-addon-ke S CBF0AF3C  2868  2884   2863          2889  2881 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cbf0af50 00000086 00000002 cbf0af3c cbf0af38 00000000 00000000 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 cbda8e48 00000001 ce99f030 c06ae440 15e8bd1a 0000001f 0008d3b7 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce99f154 c9204400 00000000 cbe91ac0 c06bc760 00100073 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: hald-addon-st S CF395F24  2172  2889   2863                2884 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cf395f38 00000086 00000002 cf395f24 cf395f20 00000000 cf395efc c920554c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 c0439f90 00000007 ce414a90 c06ae440 2d41b1b2 0000016d 00002b82 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce414bb4 c9204400 00000000 cbe91900 ec63814d 00000163 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: scsi_eh_1     S C9279A90  3752  2941      7          2947  1294 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cff64f6c 00000046 c92048c0 c9279a90 c0420b4c c9204400 c9279a90 cff64f28 
>Feb 16 15:22:21 rh5cluster2 kernel:        c0420dac 00000000 00000005 cfd9a030 cfd9fa90 77a24451 0000001f 00000456 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd9a154 c9204400 00000000 cbeffac0 c92048c0 c9fc8c84 c060d307 cff64fc8 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d086c867>] scsi_error_handler+0x53/0x462 [scsi_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: qla2xxx_1_dpc S CFF62F8C  2920  2947      7          2951  2941 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cff62fa0 00000046 00000002 cff62f8c cff62f88 00000000 fffcd8eb cff62f4c 
>Feb 16 15:22:21 rh5cluster2 kernel:        cff62f4c 000003e8 00000009 cfd9a560 c9279030 4def62b3 00000054 000024bd 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd9a684 c9204400 00000000 cfdaf580 00000100 00000000 00000000 d08de28d 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08de28d>] qla2x00_loop_resync+0x5d/0xae [qla2xxx]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08d9488>] qla2x00_do_dpc+0x31/0x442 [qla2xxx]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d08d9457>] qla2x00_do_dpc+0x0/0x442 [qla2xxx]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: scsi_wq_1     S 00000003  1832  2951      7          2952  2947 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c9c4df8c 00000046 cb838848 00000003 cfbea8e0 00000001 cfbea8e0 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000001 c9c4df64 00000009 ccf45030 cc4f8030 10604a58 00000020 000084d3 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccf45154 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 c9c4dfa8 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: fc_wq_1       S 00000003  3436  2952      7          2953  2951 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cff67f8c 00000046 c0546f9e 00000003 cdb09ce0 cbfa88c0 cdb09ce0 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000001 cff67f64 00000009 ccf45a90 c9279030 46d4bb79 00000054 00005a11 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccf45bb4 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 cff67fa8 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0546f9e>] device_release+0x37/0x7c
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: fc_dl_1       S 00000086  3784  2953      7          3303  2952 (L-TLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd1af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd9fc60 
>Feb 16 15:22:21 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd15030 cfd9fa90 887ebba7 0000001f 000004e4 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd15154 c9204400 00000000 cbeffac0 cfd06184 00000000 c042f8b3 cfd1afa8 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: smartd        S C042E00A  3136  3072      1          3076  2862 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c9fc8f38 00000082 c0453f95 c042e00a c9fc8efc 0002ec18 c9fc8efc c920554c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 c0439f90 00000005 ce414560 ce414030 940c2758 00000020 00120009 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce414684 c9204400 00000000 cd46ce40 621cda46 000001b9 49c08a46 00000016 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: login         S C0403D2D  2120  3076      1  3217    3077  3072 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd99f54 00000086 c9d55fb8 c0403d2d ce99f560 00000000 c91f1a2c bffb30e0 
>Feb 16 15:22:21 rh5cluster2 kernel:        09d13065 cf8d1ecc 00000008 cfd9f030 ccfe8030 8bb2a884 000000b6 00007638 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd9f154 c9204400 00000000 cd46c200 cd46c200 00000ecc cb4abbfc cfd99f84 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0403d2d>] copy_thread+0x2f/0x1bd
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: mingetty      S C05337DB  2952  3077      1          3078  3076 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ce73fed4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000013 caf9a400 00000006 ca6a4a90 ce414030 a07bc494 00000020 000129cf 
>Feb 16 15:22:21 rh5cluster2 kernel:        ca6a4bb4 c9204400 00000000 cd46cac0 00000000 00000065 cfc40800 cf041b40 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: mingetty      S C05337DB  2952  3078      1          3079  3077 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c9eeaed4 00000086 00000020 c05337db 00000000 00000000 00001000 cfaa7440 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000013 cb838400 00000005 ce414030 cbac9030 a088996b 00000020 00026877 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce414154 c9204400 00000000 cbeff3c0 00000000 00000065 cfc40800 cf002c80 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: mingetty      S C05337DB  2952  3079      1          3080  3078 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd1ced4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000013 cdbfa400 00000004 cb4fa560 cd922030 a0f55035 00000020 0002879a 
>Feb 16 15:22:21 rh5cluster2 kernel:        cb4fa684 c9204400 00000000 cd46c900 00000000 00000065 cfc40800 cbdce680 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: mingetty      S C05337DB  2952  3080      1          3081  3079 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ca59fed4 00000082 00000020 c05337db 00000000 00000000 00001000 cec9e0c0 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000013 ce332000 00000003 cfd9fa90 cb4faa90 a1359f74 00000020 00028c54 
>Feb 16 15:22:21 rh5cluster2 kernel:        cfd9fbb4 c9204400 00000000 cd46cc80 00000000 00000065 cfc40800 cfac5e40 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: mingetty      S CF4D7EC0  2952  3081      1          3311  3080 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cf4d7ed4 00000086 00000002 cf4d7ec0 cf4d7ebc 00000000 00001000 cf364600 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000013 cf68e400 00000002 cbac9560 c06ae440 a41b4d7f 00000020 00008d48 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbac9684 c9204400 00000000 cbeffac0 00000000 00000065 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: sshd          S CB809B48  1772  3166   2673  3168    3373       (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cb809b5c 00000082 00000002 cb809b48 cb809b44 00000000 c0423f8f 00000004 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000046 00000000 0000000a cbac9a90 cbac9030 72ca7814 00000159 0000bc5d 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbac9bb4 c9204400 00000000 cd46c740 c0420985 00000000 00000000 00000003 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c045007b>] relay_reset+0x5f/0x68
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e79d>] _spin_lock_irqsave+0x0/0x27
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: bash          S 00000159  1772  3168   3166  3371               (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cb8dff40 00000082 72d4975e 00000159 cd94d128 00000086 ca11be64 c9d4fa04 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000001 cb8dff28 00000009 cbac9030 cbc3d560 72d4daac 00000159 00029ce2 
>Feb 16 15:22:21 rh5cluster2 kernel:        cbac9154 c9204400 00000000 cd46c580 00000000 00000001 cb8dff4c cb8dff70 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04294f4>] sys_waitpid+0x27/0x2b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: bash          S C9D55EC0  2844  3217   3076                     (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        c9d55ed4 00000082 00000002 c9d55ec0 c9d55ebc 00000000 c0525b0b 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000016 cb112000 00000009 ce99f560 c06ae440 cae63a06 000000c1 0005d72c 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce99f684 c9204400 00000000 cd46c040 00000000 00000000 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b0b>] tty_ldisc_deref+0x55/0x64
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: clvmd         S CEACBB48  2580  3302      1          3318  3312 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ceacbb5c 00000082 00000002 ceacbb48 ceacbb44 00000000 ce487a90 c9204400 
>Feb 16 15:22:21 rh5cluster2 kernel:        ceacbb24 c0423f8f 00000007 ce487a90 c06ae440 efb096d7 0000016a 00001576 
>Feb 16 15:22:21 rh5cluster2 kernel:        ce487bb4 c9204400 00000000 cce33c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c05a8e0a>] sys_recvfrom+0x11e/0x12b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: clvmd         S CE8EFEDC  3484  3318      1          3319  3302 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        ce8eff18 00000082 c043c5c4 ce8efedc c0663bcf 00000003 cc140584 c07a8468 
>Feb 16 15:22:21 rh5cluster2 kernel:        cc4c5400 00000001 0000000a cb4faa90 cc6d6030 2a8f1852 00000133 000033bf 
>Feb 16 15:22:21 rh5cluster2 kernel:        cb4fabb4 c9204400 00000000 cce33c80 d0b7f1b4 0000007b ccc200d8 ce8eff58 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0b7f1b4>] __put_lkb+0xba/0xd5 [dlm]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0b855eb>] device_read+0xa2/0x22b [dlm]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:21 rh5cluster2 kernel:  [<d0b85549>] device_read+0x0/0x22b [dlm]
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:21 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:21 rh5cluster2 kernel:  =======================
>Feb 16 15:22:21 rh5cluster2 kernel: clvmd         S 00000133  1780  3319      1                3318 (NOTLB)
>Feb 16 15:22:21 rh5cluster2 kernel:        cb603e80 00000082 2a9bdea7 00000133 c047ef4e 00000082 cc248f34 cc248f0c 
>Feb 16 15:22:21 rh5cluster2 kernel:        00000000 cb603e58 00000007 ccc20030 cfd9f560 2abc8320 00000133 00003158 
>Feb 16 15:22:21 rh5cluster2 kernel:        ccc20154 c9204400 00000000 cce33c80 00000000 00000000 00000003 00000286 
>Feb 16 15:22:21 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046fb37>] vfs_stat_fd+0x19/0x40
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: dlm_astd      S C041FA15  3724  3303      7          3304  2953 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cc0f2fb0 00000046 cc0f2f6c c041fa15 00000000 00000003 cfa509b4 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        cfa509b4 00000000 00000009 ce487560 ccde1030 defb2b02 00000147 000001b4 
>Feb 16 15:22:22 rh5cluster2 kernel:        ce487684 c9204400 00000000 cfdaf580 d0b7f125 d0a23923 00000000 d0a238a0 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b7f125>] __put_lkb+0x2b/0xd5 [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a23923>] gdlm_bast+0x83/0x8c [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b7b151>] dlm_astd+0x27/0x11e [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b7b12a>] dlm_astd+0x0/0x11e [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: dlm_scand     S CC508F80  3660  3304      7          3305  3303 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cc508f94 00000046 00000002 cc508f80 cc508f7c 00000000 cc541694 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        cc1cfd80 cc0730a8 0000000a cafcc560 c06ae440 4d7f15f2 0000016d 00002669 
>Feb 16 15:22:22 rh5cluster2 kernel:        cafcc684 c9204400 00000000 cf9b5740 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b8223c>] dlm_scand+0x0/0x3e [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b8226d>] dlm_scand+0x31/0x3e [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: dlm_recv/0    S 00000003  2576  3305      7          3306  3304 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cc572f8c 00000046 00000000 00000003 c08c8960 00000002 c08c8960 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000001 cc572f64 00000009 cc24ea90 ccde1030 1aabe186 00000148 00001494 
>Feb 16 15:22:22 rh5cluster2 kernel:        cc24ebb4 c9204400 00000000 cce333c0 00000000 00000000 ffffffff cc572fa8 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: dlm_send      S 00000003  2944  3306      7          3307  3305 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        ccaa0f8c 00000046 00000000 00000003 c08c8760 c91b1720 c08c8760 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000001 ccaa0f64 0000000a ccde1030 c9279030 1aac2683 00000148 000044fd 
>Feb 16 15:22:22 rh5cluster2 kernel:        ccde1154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff ccaa0fa8 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: dlm_recoverd  S CC5A5F80  3540  3307      7          3333  3306 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cc5a5f94 00000046 00000002 cc5a5f80 cc5a5f7c 00000000 d0b82ca2 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        d0b8370a 00000000 0000000a ccc20560 c06ae440 dc2babe1 00000130 00001c92 
>Feb 16 15:22:22 rh5cluster2 kernel:        ccc20684 c9204400 00000000 cc818ac0 00000001 cc4c5400 ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b82ca2>] disallow_sync_reply+0xe/0x24 [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b8370a>] dlm_rcom_status+0xb8/0x18e [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b845f0>] dlm_recoverd+0x49/0x4aa [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0b845a7>] dlm_recoverd+0x0/0x4aa [dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: lock_dlm2     D C9204448  2744  3333      7          3338  3307 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cdbe0e6c 00000046 cc834a5c c9204448 cf78d030 c0420b4c c9204400 cf78d030 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000009 c04209fd 00000009 cb3d6a90 c9279030 1ac5aacf 00000148 00000576 
>Feb 16 15:22:22 rh5cluster2 kernel:        cb3d6bb4 c9204400 00000000 cfdaf580 00000001 c9204400 00000000 c9204400 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04209fd>] task_rq_lock+0x31/0x58
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d45f>] wait_for_completion+0x63/0x87
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437485>] kthread_stop+0x4e/0x6c
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a23e25>] gdlm_withdraw+0x96/0xad [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a4a492>] gfs2_withdraw_lockproto+0x16/0x30 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a47ada>] gfs2_lm_withdraw+0x6d/0x8d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a58eff>] gfs2_assert_withdraw_i+0x3b/0x4e [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a4ab0a>] gfs2_meta_inval+0x41/0x86 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a45658>] meta_go_inval+0xd/0x17 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a44dfe>] drop_bh+0xab/0x12a [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a44692>] gfs2_glock_cb+0xae/0x11c [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a24769>] gdlm_thread+0x5ac/0x5f9 [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a241bd>] gdlm_thread+0x0/0x5f9 [lock_dlm]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: gfs2_scand    S 00000001  3660  3338      7          3339  3333 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cbbacf94 00000046 cf7fa360 00000001 cbbacf68 c0437756 00000000 cac07fb0 
>Feb 16 15:22:22 rh5cluster2 kernel:        cf7fa360 c041fa15 0000000a cbc3d030 cbf9aa90 715991f2 0000016c 000d14e2 
>Feb 16 15:22:22 rh5cluster2 kernel:        cbc3d154 c9204400 00000000 cce333c0 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d464>] gfs2_scand+0x0/0x3d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a43b46>] examine_bucket+0x25/0x7b [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d494>] gfs2_scand+0x30/0x3d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: gfs2_glockd   S CAC07F94  3492  3339      7          3342  3338 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cac07fa8 00000046 00000002 cac07f94 cac07f90 00000000 cb96a640 d0a44e63 
>Feb 16 15:22:22 rh5cluster2 kernel:        d0a43a41 cb96a640 0000000a cbf9aa90 c06ae440 7159bf93 0000016c 00002da1 
>Feb 16 15:22:22 rh5cluster2 kernel:        cbf9abb4 c9204400 00000000 cce333c0 ca0a9914 c047cb1a ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a44e63>] drop_bh+0x110/0x12a [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a43a41>] gfs2_glock_put+0x1e/0xfe [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c047cb1a>] iput+0x60/0x62
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d440>] gfs2_glockd+0x84/0xa8 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d3bc>] gfs2_glockd+0x0/0xa8 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: gfs2_recoverd S CC7DEF80  3660  3342      7          3343  3339 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cc7def94 00000046 00000002 cc7def80 cc7def7c 00000000 c9201700 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000000 cff72030 00000001 cebbba90 c06ae440 6d856cbc 00000165 00000c5d 
>Feb 16 15:22:22 rh5cluster2 kernel:        cebbbbb4 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d4a1>] gfs2_recoverd+0x0/0x3d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d4d1>] gfs2_recoverd+0x30/0x3d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: gfs2_logd     D CEA6CE88  3412  3343      7          3344  3342 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cea6ce9c 00000046 00000002 cea6ce88 cea6ce84 00000000 0001c968 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000400 cc4af0ac 0000000a cebbb560 c06ae440 8bad5447 00000149 0000a297 
>Feb 16 15:22:22 rh5cluster2 kernel:        cebbb684 c9204400 00000000 cce333c0 c042e00a cff7775c ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0488586>] sync_buffer+0x30/0x33
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060dae7>] __wait_on_bit+0x33/0x58
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060db6f>] out_of_line_wait_on_bit+0x63/0x6b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0488504>] __wait_on_buffer+0x24/0x27
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0488f22>] sync_dirty_buffer+0x89/0xbe
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a48493>] log_write_header+0x138/0x3b6 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a48a7c>] gfs2_log_flush+0x1d0/0x27c [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d570>] gfs2_logd+0x92/0x11d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d4de>] gfs2_logd+0x0/0x11d [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: gfs2_quotad   S CAD74F70  2540  3344      7                3343 (L-TLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cad74f84 00000046 00000002 cad74f70 cad74f6c 00000000 cffdd240 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel:        d0a4476f cf7fa000 0000000a cebbb030 c06ae440 a2049963 0000016d 00000ec0 
>Feb 16 15:22:22 rh5cluster2 kernel:        cebbb154 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a4476f>] gfs2_glock_dq+0x6f/0x79 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d5fb>] gfs2_quotad+0x0/0x129 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a51d15>] gfs2_quota_scan+0x20/0x16a [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:22:22 rh5cluster2 kernel:  [<d0a3d70e>] gfs2_quotad+0x113/0x129 [gfs2]
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: umount        S CA11BF40  2116  3371   3168  3372               (NOTLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        ca11bf54 00000086 00000002 ca11bf40 ca11bf3c 00000000 c915804c 485f0018 
>Feb 16 15:22:22 rh5cluster2 kernel:        0e84b065 cac027c0 00000007 cbc3d560 c06ae440 74b2a0b7 00000159 00003ef8 
>Feb 16 15:22:22 rh5cluster2 kernel:        cbc3d684 c9204400 00000000 cbe91200 cbe91200 000007c0 ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: umount.gfs2   D CF279DA0  1980  3372   3371                     (NOTLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        cf279db4 00000082 00000002 cf279da0 cf279d9c 00000000 c04573ec 000000d0 
>Feb 16 15:22:22 rh5cluster2 kernel:        cf1ba540 cc4af458 00000007 cbc3da90 c06ae440 76852d96 00000159 000b1f66 
>Feb 16 15:22:22 rh5cluster2 kernel:        cbc3dbb4 c9204400 00000000 cbe91740 c042e00a cff7718c ffffffff 00000000 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04573ec>] __do_page_cache_readahead+0x1a3/0x1bd
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0451d86>] sync_page+0x0/0x3b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0451dbe>] sync_page+0x38/0x3b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060da21>] __wait_on_bit_lock+0x2a/0x52
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0451d78>] __lock_page+0x58/0x5e
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0452456>] do_generic_mapping_read+0x1d1/0x43b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0454274>] generic_file_aio_read+0x173/0x1a3
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0451ba0>] file_read_actor+0x0/0xd1
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046cd64>] do_sync_read+0x0/0x10a
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:22:22 rh5cluster2 kernel: sshd          S CEB60B48  1772  3373   2673  3375          3166 (NOTLB)
>Feb 16 15:22:22 rh5cluster2 kernel:        ceb60b5c 00000082 00000002 ceb60b48 ceb60b44 00000000 c041ff8c cef49480 
>Feb 16 15:22:22 rh5cluster2 kernel:        00000001 c0406fb4 0000000a cb3d6560 cb3d6030 a2379279 0000016d 0000de61 
>Feb 16 15:22:22 rh5cluster2 kernel:        cb3d6684 c9204400 00000000 cbe91580 c0420985 00000000 00000000 00000003 
>Feb 16 15:22:22 rh5cluster2 kernel: Call Trace:
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:22:22 rh5cluster2 last message repeated 3 times
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:22:22 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:22:22 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: read+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kthread       S 00000003  3328     7      1    44     434     6 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        c927af8c 00000046 cfd71f5c 00000003 c92781e0 00000003 c92781e0 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000001 c927af64 0000000a c9279a90 cfd6e560 18711b25 00000175 000004b6 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9279bb4 c9204400 00000000 cc818ac0 00000000 00000000 c042f8b3 c927afa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kblockd/0     S CFD5AF78  3412    44      7            45       (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd5af8c 00000046 00000002 cfd5af78 cfd5af74 00000000 cfd30660 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000001 cfd5af64 0000000a cfd47560 c06ae440 6fddb3e1 00000178 00007760 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd47684 c9204400 00000000 cc818ac0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kacpid        S C0406FB4  3784    45      7           137    44 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd5ef8c 00000046 00000000 c0406fb4 c0422130 00000000 00000000 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd5efbc 00000000 00000001 cfd47030 c9272030 5926fa42 0000000a 00001ae0 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd47154 c9204400 00000000 c06ae280 cfd056c4 00000000 c042f8b3 cfd5efa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: cqueue/0      S 00000086  3784   137      7           138    45 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd25f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24560 c9279a90 5aa26eca 0000000a 000008de 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd24684 c9204400 00000000 c06ae280 cfd05184 00000000 c042f8b3 cfd25fa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: ksuspend_usbd S 00000086  3784   138      7           141   137 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd28f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd24030 c9272030 5aa310e1 0000000a 00000567 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd24154 c9204400 00000000 c06ae280 c92bac04 00000000 c042f8b3 cfd28fa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: khubd         S 00000004  3456   141      7           143   138 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd31f3c 00000046 cfde2660 00000004 c05788f0 00049544 000000a3 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel:        80000180 cfdf9400 0000000a cfd32a90 cfd15030 fe28ddce 0000000a 00001d24 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd32bb4 c9204400 00000000 cfdafac0 00000004 000003e8 cfd31f9c c06cf940 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c05788f0>] usb_control_msg+0xc5/0xcf
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c057676a>] hub_thread+0x940/0x9a7
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0575e2a>] hub_thread+0x0/0x9a7
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kseriod       S CFD46F7C  3176   143      7           166   141 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd46f90 00000046 00000002 cfd46f7c cfd46f78 00000000 00000000 c054941b 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000004 c06d0a4c 0000000a cfd47a90 c06ae440 eaa67105 0000000a 00000d93 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd47bb4 c9204400 00000000 cfdafc80 c06d089c c06d08a0 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c054941b>] driver_probe_device+0x44/0xa5
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0587f66>] serio_thread+0x25b/0x2a9
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0587d0b>] serio_thread+0x0/0x2a9
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: pdflush       S 00000000  2484   166      7           167   143 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd71f9c 00000046 00000000 00000000 00000292 c0422130 00000000 c04375c0 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000000 0000000f 00000009 cfd6e560 cfd9f560 187124df 00000175 000009ba 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd6e684 c9204400 00000000 cc818ac0 c9204fa0 1871222e 00000175 c04374cb 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c0>] kthread_create+0xa1/0xa9
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04374cb>] keventd_create_kthread+0x0/0x54
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04570ae>] pdflush+0xb0/0x19f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: pdflush       D CFD74EEC  2420   167      7           168   166 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd74f00 00000046 00000002 cfd74eec cfd74ee8 00000000 00000000 c0406fb4 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd74eac c0406fb4 0000000a cfd75a90 c06ae440 ef83f7fa 00000187 00001144 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd75bb4 c9204400 00000000 cce333c0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060d504>] io_schedule_timeout+0x3f/0x63
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c045a78e>] congestion_wait+0x50/0x64
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0456d64>] wb_kupdate+0x98/0xde
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0457107>] pdflush+0x109/0x19f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0456ccc>] wb_kupdate+0x0/0xde
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kswapd0       S C9258D34  2512   168      7           169   167 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd2af48 00000046 c0439f90 c9258d34 00000246 c0450a41 00000000 05f36aa8 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9258d4c 00000000 0000000a cfd2ba90 cfd75a90 2a2c7088 0000016f 00001302 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd2bbb4 c9204400 00000000 cf9b5740 c045a799 00000000 cfd2af98 c06b87d8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c045a799>] congestion_wait+0x5b/0x64
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04594e2>] kswapd+0xb4/0x3d5
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c045942e>] kswapd+0x0/0x3d5
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: aio/0         S 00000086  3784   169      7           329   168 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd33f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000001 cfd32560 c9272030 6ae0b51c 0000000a 00000d4e 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd32684 c9204400 00000000 c06ae280 cffc7c04 00000000 c042f8b3 cfd33fa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kpsmoused     S 00000086  3784   329      7           362   169 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd18f8c 00000046 00000000 00000086 c0422130 00000000 00000000 c926bc60 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000009 cfd15560 cfb73a90 e3191df7 0000000a 00000b35 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd15684 c9204400 00000000 c06ae280 cffc7184 00000000 c042f8b3 cfd18fa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kmirrord      S 00000086  3784   362      7           367   329 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cff9af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cff7b730 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd6e030 cff7b560 8757d833 0000000e 0000097a 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd6e154 c9204400 00000000 cfdaf900 cfc3d184 00000000 c042f8b3 cff9afa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: ksnapd        S 00000086  3784   367      7           370   362 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cff73f8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd75200 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9201700 00000000 00000005 cff7b030 cfd75030 877b12a3 0000000e 0000095c 
>Feb 16 15:24:13 rh5cluster2 kernel:        cff7b154 c9204400 00000000 cfdaf900 cfc3d6c4 00000000 c042f8b3 cff73fa8 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kjournald     S 00000004  2364   370      7           400   367 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd93f9c 00000046 c0423f8f 00000004 cf406aec cdb09740 00000000 cff98800 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000000 00000000 0000000a cff72030 cfd75a90 60e67f5c 00000186 0000176a 
>Feb 16 15:24:13 rh5cluster2 kernel:        cff72154 c9204400 00000000 cc818ac0 c0420985 00000000 cfd93fac cf406a68 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:13 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kauditd       S CFF70F60  3700   400      7          1294   370 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cff70fa0 00000046 00000001 cff70f60 c041fa15 00000000 00000001 cb71d218 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000000 cb71d218 0000000a cfb73a90 cfd15a90 8ddbdcb4 0000015e 00001510 
>Feb 16 15:24:13 rh5cluster2 kernel:        cfb73bb4 c9204400 00000000 cbe913c0 c0420985 00000000 00000000 cff70fb0 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c044986d>] kauditd_thread+0x103/0x132
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c044976a>] kauditd_thread+0x0/0x132
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: udevd         S CFF78B48  2580   434      1          1977     7 (NOTLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cff78b5c 00000086 00000002 cff78b48 cff78b44 00000000 00000004 cff78b18 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000011 c044f208 00000004 c9279030 c06ae440 1b53b7b6 00000148 0001b087 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9279154 c9204400 00000000 cfdaf3c0 00000203 00000200 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c048fb82>] inotify_poll+0x3a/0x40
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 last message repeated 3 times
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04dedbb>] number+0x105/0x24d
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c047007b>] copy_strings+0xe7/0x1b9
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04376c6>] bit_waitqueue+0x16/0x48
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04824fc>] seq_printf+0x2e/0x4b
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c049f22a>] show_stat+0x9ec/0x9f9
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0483a12>] simple_empty+0xe/0x50
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04293f5>] do_wait+0x8ba/0x95e
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: kjournald     S CFD16F88  3184  1294      7          2941   400 (L-TLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd16f9c 00000046 00000002 cfd16f88 cfd16f84 00000000 00000000 cfa56000 
>Feb 16 15:24:13 rh5cluster2 kernel:        00000000 00000000 00000002 c9279560 c06ae440 4b25b168 0000001a 0000132b 
>Feb 16 15:24:13 rh5cluster2 kernel:        c9279684 c9204400 00000000 cce333c0 c0420985 00000000 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:13 rh5cluster2 kernel:  [<d085efc2>] kjournald+0x15e/0x1d4 [jbd]
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:13 rh5cluster2 kernel:  [<d085ee64>] kjournald+0x0/0x1d4 [jbd]
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:13 rh5cluster2 kernel:  =======================
>Feb 16 15:24:13 rh5cluster2 kernel: dhclient      S CFD63B48  2476  1977      1          2106   434 (NOTLB)
>Feb 16 15:24:13 rh5cluster2 kernel:        cfd63b5c 00000082 00000002 cfd63b48 cfd63b44 00000000 cc130e08 d0859b10 
>Feb 16 15:24:13 rh5cluster2 kernel:        0063c853 00000000 0000000a ccea3030 c06ae440 946642af 00000018 00007ef7 
>Feb 16 15:24:13 rh5cluster2 kernel:        ccea3154 c9204400 00000000 cf9b5c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:13 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:13 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:13 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457b8d>] activate_page+0x61/0x84
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457d23>] mark_page_accessed+0x1c/0x30
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: auditd        S CCAC9B48  2476  2106      1  2108    2107  1977 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ccac9b5c 00000086 00000002 ccac9b48 ccac9b44 00000000 c041ff8c d0859b10 
>Feb 16 15:24:14 rh5cluster2 kernel:        0063d05f ccac9bec 00000006 cfd15a90 c06ae440 881cd625 00000181 0000310d 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd15bb4 c9204400 00000000 cfdaf200 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477612>] poll_freewait+0x18/0x4c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477948>] do_sys_poll+0x302/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04224cb>] __cond_resched+0x16/0x34
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05c4e5b>] netlink_unicast+0x171/0x1a9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05c45d2>] netlink_sendskb+0x19/0x30
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05c5012>] netlink_ack+0x17f/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044973e>] audit_receive+0x944/0x970
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b209>] wake_futex+0x42/0x4c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046d3c6>] do_readv_writev+0x17d/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a75c7>] sock_aio_write+0x0/0x102
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: auditd        S CFC562BC  2176  2107      1          2127  2106 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc8a0e80 00000086 000000cf cfc562bc cfd2b030 0000000a cc8a0000 cc8a0f64 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc8a0f64 cc8a0ee0 00000007 cfd2b030 cb3d6030 8dddc5e1 0000015e 00016596 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd2b154 c9204400 00000000 cfdaf200 cfdaf200 c043b07d cc8a0f48 c0419ccd 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e3ca3>] percpu_counter_sum+0x37/0x52
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046b6df>] vfs_statfs+0x47/0x5f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046c5ce>] sys_fstatfs+0x4b/0x54
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: python        S CFF60B48  1820  2108   2106                     (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cff60b5c 00200082 00000002 cff60b48 cff60b44 00000000 0000019a 02cf6a5d 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000075 00000002 cfd6ea90 c06ae440 8e91f09a 0000015e 000381b1 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd6ebb4 c9204400 00000000 cf9b5e40 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b86>] task_running_tick+0x2a/0x234
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c06074c2>] unix_write_space+0x12/0x71
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404dd6>] sysenter_past_esp+0x5f/0x85
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: syslogd       S CCFF0B48  1732  2127      1          2130  2107 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ccff0b5c 00000086 00000002 ccff0b48 ccff0b44 00000000 00000000 cff586a4 
>Feb 16 15:24:14 rh5cluster2 kernel:        c04d96ce 00000001 00000007 ccfe8030 c06ae440 466591a0 00000183 00007475 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccfe8154 c9204400 00000000 cf9b5200 c04d021f cff4cb30 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d96ce>] cfq_insert_request+0xa5/0x391
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d021f>] elv_insert+0x146/0x1ec
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ae8c5>] datagram_poll+0x14/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606c21>] unix_dgram_recvmsg+0x222/0x239
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045440d>] mempool_alloc+0x28/0xc6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040418b>] setup_sigcontext+0x10b/0x18b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04ddf9c>] rb_insert_color+0x4c/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439a0b>] enqueue_hrtimer+0x4b/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439e11>] hrtimer_start+0xb2/0xbc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042a28a>] do_setitimer+0x166/0x4d6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: portmap       S 00000046  2488  2183      1          2222  2130 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cceeebfc 00000082 c9205900 00000046 00000046 c9205880 00000246 00000246 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfa57d80 c060e744 00000002 cfd75560 cfd75030 de9fcb2d 00000018 000087fb 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd75684 c9204400 00000000 cce33e40 cfa57d80 c06c9600 cceeec68 cb71d998 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a826f>] sys_sendmsg+0x192/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8c6a>] sys_recvmsg+0x14b/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b2666>] netdev_run_todo+0x1ec/0x1f3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05eedea>] inet_sock_destruct+0x16a/0x1b0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e862>] _write_lock_bh+0x8/0x1a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: rpc.statd     S CC908B48  1600  2222      1          2265  2183 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc908b5c 00000082 00000002 cc908b48 cc908b44 00000000 00ef8300 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 cf4606c0 00000001 cfd75030 c06ae440 dea35352 00000018 00003bd7 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd75154 c9204400 00000000 cce33580 03a57d00 00000000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 2 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: rpc.idmapd    S CC990F20  3064  2265      1          2359  2222 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc990f34 00200086 00000002 cc990f20 cc990f1c 00000000 00000000 bf8329cc 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc990f60 cf9806bc 0000000a cff7ba90 c06ae440 0fc04501 00000187 00006e89 
>Feb 16 15:24:14 rh5cluster2 kernel:        cff7bbb4 c9204400 00000000 cf9b53c0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c049043f>] sys_epoll_wait+0x13e/0x37f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: ccsd          S CA943B48  2364  2359      1          2360  2265 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca943b5c 00000086 00000002 ca943b48 ca943b44 00000000 0000003e 0b19380a 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000365 00000007 cfd9aa90 cfd9f560 94ab951a 0000013b 00005f7a 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd9abb4 c9204400 00000000 cce33ac0 0000012d 00000000 ca943c0c caf5e218 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 2 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0ecaad3>] ip6_push_pending_frames+0x34c/0x3d6 [ipv6]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0ec9550>] dst_output+0x0/0x7 [ipv6]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05acc76>] skb_queue_purge+0x11/0x17
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060666d>] unix_sock_destructor+0xe/0xd2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05aa1d3>] sk_free+0xb2/0xc7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: ccsd          S C9D7FB48  2260  2360      1          2365  2359 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        c9d7fb5c 00000086 00000002 c9d7fb48 c9d7fb44 00000000 00000086 c042ee07 
>Feb 16 15:24:14 rh5cluster2 kernel:        cff4cb9c 00000000 00000008 cfb73560 c06ae440 e564e91c 000000b2 0001c961 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfb73684 c9204400 00000000 cce33ac0 00000003 c04d2cce ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d2cce>] __generic_unplug_device+0x1d/0x1f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0908098>] tg3_start_xmit_dma_bug+0x581/0x64f [tg3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ac932>] skb_clone+0x3d/0x1f1
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060a167>] packet_rcv_spkt+0x0/0x139
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b127c>] dev_hard_start_xmit+0x1bc/0x21b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e744>] _read_lock_bh+0x8/0x17
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05b6b47>] neigh_resolve_output+0x1dd/0x20f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e36f7>] tcp_v4_send_check+0x74/0xaa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d6e05>] tcp_close+0x504/0x523
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CC991BE8  2104  2365      1          2366  2360 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc991bfc 00000086 00000002 cc991be8 cc991be4 00000000 00000000 0000016c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000001 00000000 00000001 cff72560 c06ae440 ef4ab964 00000187 0000bba1 
>Feb 16 15:24:14 rh5cluster2 kernel:        cff72684 c9204400 00000000 cce333c0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 10 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8b07>] move_addr_to_user+0x50/0x68
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CAFC7BE8  2740  2366      1          2367  2365 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cafc7bfc 00000086 00000002 cafc7be8 cafc7be4 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cfb73030 c06ae440 4a55bda4 00000019 00005293 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfb73154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S C992BE6C  3024  2367      1          2374  2366 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        c992be80 00000086 00000002 c992be6c c992be68 00000000 cfd878c0 cff2b114 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd878c0 cfd87920 00000006 cfd32030 c06ae440 842192fd 00000057 00002982 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd32154 c9204400 00000000 cce333c0 c047513c c992bd90 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046f7fb>] cp_new_stat64+0xfc/0x10e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CFD9BBE8  2740  2374      1          2375  2367 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd9bbfc 00000086 00000002 cfd9bbe8 cfd9bbe4 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccea3560 c06ae440 ed7134ee 00000157 000004ff 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccea3684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CD78DBE8  2688  2375      1          2373  2374 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cd78dbfc 00000086 00000002 cd78dbe8 cd78dbe4 00000000 000001cc 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 cd78dbc0 00000001 ccfe8560 cc24ea90 1aabccf2 00000148 000004a7 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccfe8684 c9204400 00000000 cce333c0 c04057e7 c0a3f680 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CD073BE8  2740  2394      1          2395  2387 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cd073bfc 00000086 00000002 cd073be8 cd073be4 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cd117560 ce487030 1ae0cde0 00000148 000007ea 
>Feb 16 15:24:14 rh5cluster2 kernel:        cd117684 c9204400 00000000 cce333c0 00000000 00000000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CCDC2BE8  2740  2395      1          2393  2394 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ccdc2bfc 00000086 00000002 ccdc2be8 ccdc2be4 00000000 00000194 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1a90 c06ae440 1a7d9fb8 00000148 00000406 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccde1bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CCEA0000  2784  2396      1          2397  2393 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ccdc4bfc 00000086 00000000 ccea0000 cfacba80 00000000 000000b1 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccde1560 cfd9f560 93ff7cdd 0000013b 00000590 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccde1684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e933a>] udp_sendmsg+0x45f/0x570
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ee041>] inet_sendmsg+0x3b/0x45
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S 00000000  2868  2397      1          2400  2396 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        c08c7bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccfe8a90 cff72560 9d870693 00000019 00000a7a 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccfe8bb4 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S 00000000  2868  2400      1          2401  2397 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cd093bfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cd117030 cfd2b560 0348822d 00000058 00000802 
>Feb 16 15:24:14 rh5cluster2 kernel:        cd117154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CCEA0000  2800  2401      1          2419  2400 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc4c4bfc 00000086 00000000 ccea0000 cfa51c80 00000000 000000c2 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccadb560 cfd2ba90 dc3cfdfa 00000019 000004e0 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccadb684 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e007b>] __copy_to_user_ll+0x32/0xd6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S 00000000  2868  3311      1          3312  3081 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ccb6ebfc 00000086 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cafcc030 c9279030 d3e6a84d 00000130 00000758 
>Feb 16 15:24:14 rh5cluster2 kernel:        cafcc154 c9204400 00000000 cce333c0 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: aisexec       S CCEA0000  2800  3312      1          3302  3311 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc59cbfc 00000086 00000000 ccea0000 cfa47980 00000000 00000019 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 ccc20a90 c9279030 d3e077b4 00000130 000003ca 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccc20bb4 c9204400 00000000 cce333c0 d0a1798c ccea0000 00000246 00000001 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05e83b5>] udp_push_pending_frames+0x29a/0x2b9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a82c0>] sys_sendmsg+0x1e3/0x1f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8cd8>] sys_recvmsg+0x1b9/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab139>] sock_setsockopt+0x499/0x4a3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: groupd        S CC8A9BE8  2476  2373      1          2381  2375 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc8a9bfc 00000082 00000002 cc8a9be8 cc8a9be4 00000000 cff4cb30 00000020 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000046 c0406fb4 00000002 cff7b560 ccea3560 ed712fef 00000157 00003fc5 
>Feb 16 15:24:14 rh5cluster2 kernel:        cff7b684 c9204400 00000000 cce33040 c062ad80 cc8a9ea4 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 7 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: fenced        S C0A5FBE8  2476  2381      1          2387  2373 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        c0a5fbfc 00000086 00000002 c0a5fbe8 c0a5fbe4 00000000 cff4cb30 00000008 
>Feb 16 15:24:14 rh5cluster2 kernel:        00300000 00000000 00000009 cfd2b560 c06ae440 036ac9b9 00000058 00002bf0 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd2b684 c9204400 00000000 cce33200 00001000 00000001 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 2 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: dlm_controld  S CFF72A90  2476  2387      1          2394  2381 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        caef8bfc 00000082 00000117 cff72a90 c9204400 caef8bb8 c0423f8f 00000046 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000046 00000000 00000001 cff72a90 cb3d6a90 1ac52279 00000148 0000612f 
>Feb 16 15:24:14 rh5cluster2 kernel:        cff72bb4 c9204400 00000000 cfdaf580 0807b008 caef8e9c 00000018 c04057e7 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 2 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ae166>] memcpy_toiovec+0x27/0x4a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05c59dd>] netlink_recvmsg+0x285/0x2a4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: gfs_controld  S 00000012  2184  2393      1          2396  2395 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cd784bfc 00000082 00000000 00000012 00000000 00000100 cceb5b54 08262176 
>Feb 16 15:24:14 rh5cluster2 kernel:        cd117a90 00000003 00000001 cd117a90 cd117560 1ae0c5f6 00000148 000082a4 
>Feb 16 15:24:14 rh5cluster2 kernel:        cd117bb4 c9204400 00000000 cfdaf740 0000000e 0000000b 00000001 cd784ebc 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 5 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437756>] autoremove_wake_function+0x15/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: dbus-daemon   S CCED6BE8  2308  2419      1          2474  2401 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cced6bfc 00000082 00000002 cced6be8 cced6be4 00000000 cff4cb30 00000008 
>Feb 16 15:24:14 rh5cluster2 kernel:        00300000 ccadba90 00000009 ccadba90 c06ae440 6b9635d1 00000020 0000c09e 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccadbbb4 c9204400 00000000 cfdafc80 00001000 00000001 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 5 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060da42>] __wait_on_bit_lock+0x4b/0x52
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: hcid          S CA93ABE8  2128  2474      1          2487  2419 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca93abfc 00000086 00000002 ca93abe8 ca93abe4 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        cf4606c0 cbc75e00 00000001 cfd24a90 c06ae440 c8e71f0d 0000001c 0002c8d2 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd24bb4 c9204400 00000000 cfdafac0 ca93ac0c ffffffff ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: sdpd          S 00000000  2660  2487      1          2498  2474 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc21b28 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cbc98a90 ccadb030 c5a36808 0000001c 0004ebc9 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc98bb4 c9204400 00000000 cf9b5580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047513c>] link_path_walk+0xa9/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478111>] sys_pselect7+0xdd/0x208
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782a9>] sys_pselect6+0x6d/0x79
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: krfcommd      S CBC38680  3480  2498      1          2547  2487 (L-TLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc6aaf20 00000046 00000000 cbc38680 d0a2b75f 00000000 ffffffea d0a90079 
>Feb 16 15:24:14 rh5cluster2 kernel:        c924fa00 00000286 00000005 cc24e560 ccea3a90 cfe5f3ca 0000001c 00002ad9 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc24e684 c9204400 00000000 cf9b5740 00000000 00000246 00000000 c992d4c0 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0a2b75f>] l2cap_sock_create+0x64/0x6b [l2cap]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0a90079>] bt_sock_create+0x79/0x8f [bluetooth]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0b28dad>] rfcomm_run+0x184/0x1097 [rfcomm]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cfcc>] audit_syscall_exit+0x294/0x2b0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0407880>] do_syscall_trace+0x57/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0b28c29>] rfcomm_run+0x0/0x1097 [rfcomm]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: pcscd         S D084A4C7  2108  2547      1          2551  2498 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cbcfdb5c 00000082 d0845080 d084a4c7 cd276e80 c048b0ae cff4c02c cd276e80 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000001 00001000 00000001 cc24e030 ccbb9560 e429912c 0000001c 0004fdce 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc24e154 c9204400 00000000 cf9b5740 d084aff0 00000000 00000001 00000008 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084a4c7>] __map_bio+0xcf/0x11f [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084aff0>] __split_bio+0x413/0x423 [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060db04>] __wait_on_bit+0x50/0x58
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d085d1f8>] find_revoke_record+0x42/0x6c [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d085d6c8>] journal_cancel_revoke+0x7a/0x9c [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08592a1>] __journal_file_buffer+0x121/0x205 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d085dce1>] journal_add_journal_head+0x1c/0x136 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04572cd>] __do_page_cache_readahead+0x84/0x1bd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0452679>] do_generic_mapping_read+0x3f4/0x43b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04ddfdc>] rb_insert_color+0x8c/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045ecbd>] __vma_link+0x44/0x4f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04243cb>] sched_fork+0x73/0xaa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0425dbf>] copy_process+0xefa/0x10b1
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0424753>] wake_up_new_task+0x1af/0x1b7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: pcscd         S CFD0EF24  3188  2551      1          2572  2547 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cfd0ef38 00000082 00000002 cfd0ef24 cfd0ef20 00000000 cfd0eefc c920554c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 c0439f90 00000007 ccbb9560 c06ae440 defca27f 00000187 0001d44f 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccbb9684 c9204400 00000000 cf9b5740 6e3fee10 0000017e ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: hidd          S 00000000  2812  2572      1          2592  2551 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cf969bc0 00000082 00000000 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 00000001 cba58560 ccadb030 f010d4e3 0000001c 000085f6 
>Feb 16 15:24:14 rh5cluster2 kernel:        cba58684 c9204400 00000000 cbeffe40 00000000 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0a903a5>] bt_sock_poll+0x14/0xf4 [bluetooth]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0605b32>] unix_dgram_sendmsg+0x3ea/0x476
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060ff63>] do_page_fault+0x273/0x4d1
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a80c2>] sock_sendmsg+0xd0/0xeb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a890f>] sys_sendto+0x118/0x138
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047843a>] sys_ppoll+0xd9/0x1fc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: automount     S CFBAFC00  2104  2592      1          2593  2572 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc60ed8 00000082 00000000 cfbafc00 c92a6800 00000004 c9177220 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        cfbafc00 c0472500 00000001 cbc98030 cba58030 08c2edd5 0000001d 00007fd4 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc98154 c9204400 00000000 cfdafe40 c04ddf9c 00000004 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472500>] pipe_write+0x393/0x3ec
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04ddf9c>] rb_insert_color+0x4c/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0430d0d>] dequeue_signal+0x36/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04319fe>] sys_rt_sigtimedwait+0x1cd/0x2a8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: automount     S CC50AE6C  3384  2593      1          2594  2592 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc50ae80 00000082 00000002 cc50ae6c cc50ae68 00000000 cb81cf34 cb81cf0c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 cc50ae58 00000007 cafcca90 c06ae440 a8b262c1 00000186 0000133d 
>Feb 16 15:24:14 rh5cluster2 kernel:        cafccbb4 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: automount     S CB81CE6C  2316  2594      1          2597  2593 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cb81ce80 00000082 00000002 cb81ce6c cb81ce68 00000000 cf764f34 cf764f0c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000254 c06b6280 0000000a ccbb9030 c06ae440 d25ee041 00000187 000035f7 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccbb9154 c9204400 00000000 cfdafe40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0403dc8>] copy_thread+0xca/0x1bd
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: automount     S 00000086  2376  2597      1          2600  2594 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cba67bfc 00000082 c0564b78 00000086 c042ee07 cfb76200 c07a5c00 00000086 
>Feb 16 15:24:14 rh5cluster2 kernel:        c042ef10 00000000 0000000a cba58a90 ccbb9030 2bd4a531 00000183 0000211f 
>Feb 16 15:24:14 rh5cluster2 kernel:        cba58bb4 c9204400 00000000 cfdafe40 cba67c70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0564b78>] ide_build_sglist+0x2b/0xb8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05645da>] ide_dma_exec_cmd+0x1f/0x22
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: automount     S CFF4C02C  2184  2600      1          2616  2597 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cfc4bbfc 00000082 c048b0ae cff4c02c cbc75500 00000001 00001000 030c0020 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 00000000 0000000a cba58030 ccbb9030 a8b1e0fa 00000186 000020ce 
>Feb 16 15:24:14 rh5cluster2 kernel:        cba58154 c9204400 00000000 cfdafe40 cfc4bc70 c04dbc9c 00000000 c041ff8c 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048b0ae>] __bio_clone+0x6f/0x8a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e05ed>] match_number+0x79/0x83
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0a9f605>] autofs4_fill_super+0x31f/0x3c9 [autofs4]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: acpid         S CC50BBE8  2108  2616      1          2632  2600 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc50bbfc 00000082 00000002 cc50bbe8 cc50bbe4 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        cf4606c0 cbc75480 00000007 ccadb030 ca6a4560 e17f9d47 0000001e 00020054 
>Feb 16 15:24:14 rh5cluster2 kernel:        ccadb154 c9204400 00000000 cf9b5040 00001000 00000000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08596cf>] journal_stop+0x1b6/0x1c2 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891bde>] ext3_journal_dirty_data+0x0/0x32 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453322>] generic_file_buffered_write+0x4ca/0x5f8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453a08>] generic_file_aio_write+0x5e/0xb3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d088ffdc>] ext3_file_write+0x24/0x8f [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: hpiod         S 3537FFFF  2476  2632      1          2637  2616 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cadf9e3c 00200086 00000000 3537ffff 00003736 00000000 00000000 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 cbc98560 00000001 cbc98560 ccbb9a90 186c89d0 0000001d 00005699 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc98684 c9204400 00000000 cf9b5900 cadf9e94 cadf9e95 cadf9e94 ffffffff 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9d7f>] lock_sock_nested+0x8a/0x92
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d4088>] inet_csk_accept+0x9b/0x200
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b7de>] d_instantiate+0x5c/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05eebea>] inet_accept+0x1f/0xb5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a90a0>] sys_accept+0xee/0x1c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a89ec>] sys_connect+0x82/0xad
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9248>] sys_socketcall+0xd6/0x261
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: python        S CA721B48  2476  2637      1          2654  2632 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca721b5c 00200086 00000002 ca721b48 ca721b44 00000000 0003ea1c c04057e7 
>Feb 16 15:24:14 rh5cluster2 kernel:        c0406fb4 c0406fb4 0000000a cbea7560 c06ae440 e7318859 00000187 00016947 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbea7684 c9204400 00000000 cbeff900 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488012>] __getblk+0x2c/0x277
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: cupsd         S CC31EB48  2040  2654      1          2673  2637 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cc31eb5c 00200082 00000002 cc31eb48 cc31eb44 00000000 cc130c68 d0859b10 
>Feb 16 15:24:14 rh5cluster2 kernel:        0063f04f 00000000 00000007 cbea7030 c06ae440 9a21bfcf 0000001d 00008dde 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbea7154 c9204400 00000000 cce33740 cc31eb6c ffffffff ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0471a24>] pipe_poll+0x24/0x7a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 last message repeated 3 times
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08936a5>] ext3_ordered_commit_write+0xb4/0xd2 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457aab>] __pagevec_lru_add+0x7d/0x88
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04533e1>] generic_file_buffered_write+0x589/0x5f8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453931>] __generic_file_aio_write_nolock+0x4e1/0x55a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ff5c>] signal_wake_up+0x1e/0x2c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04301fb>] __group_send_sig_info+0x74/0x7f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04313fc>] sys_kill+0xee/0xfa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: sshd          S CB443B48  2476  2673      1  3166    2699  2654 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cb443b5c 00200082 00000002 cb443b48 cb443b44 00000000 cb203a2c d0859b10 
>Feb 16 15:24:14 rh5cluster2 kernel:        00638802 00000000 00000007 cbea7a90 c06ae440 8c927a69 0000015e 000067b3 
>Feb 16 15:24:14 rh5cluster2 kernel:        cbea7bb4 c9204400 00000000 cbeffc80 d0891698 00001000 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04775f2>] free_poll_entry+0xe/0x16
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089c97a>] __ext3_journal_dirty_metadata+0x16/0x3a [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b5e>] journal_get_write_access+0x21/0x26 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891523>] ext3_mark_iloc_dirty+0x270/0x2d6 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d089885e>] __ext3_journal_stop+0x19/0x34 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0437720>] __wake_up_bit+0x28/0x33
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05a9f1d>] sock_alloc_send_skb+0x71/0x19c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ab1b5>] sock_def_readable+0x12/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0461d88>] anon_vma_prepare+0x11/0xa5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: sendmail      S CBC37B48  2476  2699      1          2707  2673 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cbc37b5c 00200082 00000002 cbc37b48 cbc37b44 00000000 cb203a2c d0859b10 
>Feb 16 15:24:14 rh5cluster2 kernel:        0063d888 00000000 0000000a cc891a90 c06ae440 1d057340 00000187 0001fc3e 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc891bb4 c9204400 00000000 cbeff740 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05d5400>] tcp_poll+0x19/0x127
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0524a91>] extract_buf+0x9d/0xce
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04697cf>] cache_alloc_refill+0x143/0x44b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0499ce9>] proc_alloc_inode+0x3e/0x63
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047cee1>] alloc_inode+0xde/0x166
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047cff4>] iget_locked+0x8b/0xfb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048f6fb>] inotify_d_instantiate+0x41/0x67
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0499d9c>] proc_get_inode+0x8e/0x116
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c049cf16>] proc_lookup+0x72/0xa4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0499f7f>] proc_root_lookup+0xe/0x26
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d08900d8>] ext3_sync_file+0x34/0xbc [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04df75c>] vsnprintf+0x450/0x48c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04df806>] sprintf+0x20/0x23
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c049e6a7>] loadavg_read_proc+0xce/0xfa
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060e8e1>] lock_kernel+0x14/0x2e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0499eb3>] proc_delete_inode+0x0/0x41
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: sendmail      S CA26FF98  2476  2707      1          2724  2699 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca26ffac 00200086 00000002 ca26ff98 ca26ff94 00000000 00000000 00000002 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000002 caf9a000 00000001 cc891030 c06ae440 ab35b889 0000001d 0000e0f1 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc891154 c9204400 00000000 cf9b5ac0 ca26ffb8 c040794d ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c043157e>] sys_pause+0x11/0x17
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: gpm           S CDBD39B0  2476  2724      1          2740  2707 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cafaeb5c 00000082 00000000 cdbd39b0 00000000 cdbd39b0 cb203a2c d0859b10 
>Feb 16 15:24:14 rh5cluster2 kernel:        0063a001 00000000 00000001 cc4f8560 cc4f8030 b193838a 0000001d 0001d5e7 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc4f8684 c9204400 00000000 cbeff580 c07a5c00 00000286 c042ef10 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0859b10>] do_get_write_access+0x435/0x462 [jbd]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0891907>] ext3_mark_inode_dirty+0x29/0x30 [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c05ad23d>] __alloc_skb+0x49/0xf7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472e39>] __follow_mount+0x1e/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0474fba>] __link_path_walk+0xb53/0xc2c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b60d>] __d_lookup+0x96/0xd9
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047b9fa>] dput+0x22/0xe8
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0474f52>] __link_path_walk+0xaeb/0xc2c
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042d3ac>] __capable+0xc/0x1f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c052c256>] vt_ioctl+0x555/0x148f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046b988>] get_unused_fd+0x50/0xb6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c052bd01>] vt_ioctl+0x0/0x148f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047652a>] fasync_helper+0x3c/0xae
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060dbc4>] mutex_lock+0xb/0x1a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046dbf7>] file_kill+0x11/0x2a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046bc23>] do_filp_open+0x32/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04e0397>] copy_to_user+0x2d/0x42
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: crond         S CA6D8F24  2128  2740      1          2778  2724 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca6d8f38 00000086 00000002 ca6d8f24 ca6d8f20 00000000 ca6d8efc c920554c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 c0439f90 00000007 cc891560 c06ae440 24543aa5 00000185 00020dd8 
>Feb 16 15:24:14 rh5cluster2 kernel:        cc891684 c9204400 00000000 cbeff200 6f10eeb9 00000189 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: xfs           S CA8F5B48  1936  2778      1          2809  2740 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca8f5b5c 00000082 00000002 ca8f5b48 ca8f5b44 00000000 005d8030 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 cf4606c0 00000007 cb2b6a90 c06ae440 b481b55d 00000134 0000407d 
>Feb 16 15:24:14 rh5cluster2 kernel:        cb2b6bb4 c9204400 00000000 cc818740 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0488721>] ll_rw_block+0x84/0x93
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04555cd>] free_pages_bulk+0x100/0x16e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04557f4>] __pagevec_free+0x14/0x1a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457dc8>] __pagevec_release_nonlru+0x44/0x4f
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0458a9f>] remove_mapping+0x84/0xa7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04590b7>] shrink_inactive_list+0x5f5/0x6c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047af31>] d_rehash+0x1c/0x2b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04585e7>] isolate_lru_pages+0x64/0x7d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0457d7c>] __pagevec_release+0x15/0x1d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0458986>] shrink_active_list+0x386/0x38e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0455ca5>] get_page_from_freelist+0x23b/0x2a5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0475e9c>] open_namei+0x6e/0x58d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c045c83e>] __handle_mm_fault+0x35f/0x8c6
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c046bc16>] do_filp_open+0x25/0x39
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: atd           S CF934F24  2476  2809      1          2828  2778 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        cf934f38 00000082 00000002 cf934f24 cf934f20 00000000 cf934efc c920554c 
>Feb 16 15:24:14 rh5cluster2 kernel:        00000000 c0439f90 00000007 cb2b6030 c06ae440 9a3cf81d 0000017a 0000d09a 
>Feb 16 15:24:14 rh5cluster2 kernel:        cb2b6154 c9204400 00000000 cc818c80 c174e775 000001b6 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:14 rh5cluster2 kernel:  =======================
>Feb 16 15:24:14 rh5cluster2 kernel: yum-updatesd  S CA666BE8  1640  2828      1          2845  2809 (NOTLB)
>Feb 16 15:24:14 rh5cluster2 kernel:        ca666bfc 00200086 00000002 ca666be8 ca666be4 00000000 ca666fac 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel:        c04057e7 ca666c0c 0000000a cb2b6560 c06ae440 ec8adee8 00000187 000051c7 
>Feb 16 15:24:14 rh5cluster2 kernel:        cb2b6684 c9204400 00000000 cc818ac0 c07a5c00 00200286 ffffffff 00000000 
>Feb 16 15:24:14 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:14 rh5cluster2 kernel:  [<d084b6c1>] dm_request+0xce/0xd4 [dm_mod]
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d1e3f>] generic_make_request+0x28e/0x29e
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048b4a7>] bio_add_page+0x31/0x3a
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c048e6a0>] mpage_end_io_read+0x0/0x66
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04d3cc6>] submit_bio+0xc0/0xc7
>Feb 16 15:24:14 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c045eba5>] find_extend_vma+0x12/0x49
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043afcf>] hash_futex+0x53/0x8b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043c05e>] do_futex+0x202/0xb3b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04057e7>] common_interrupt+0x23/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: avahi-daemon  S CAFB3BE8  2072  2845      1  2846    2862  2828 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cafb3bfc 00000082 00000002 cafb3be8 cafb3be4 00000000 cfa44480 cafb3c30 
>Feb 16 15:24:15 rh5cluster2 kernel:        c05c71fc ccea0000 0000000a cc4f8a90 c06ae440 a40ceca6 00000079 0000d2fe 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc4f8bb4 c9204400 00000000 cc818e40 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05c71fc>] nf_iterate+0x38/0x6a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 last message repeated 8 times
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a965d>] sock_common_recvmsg+0x3e/0x54
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a7fd7>] sock_recvmsg+0xec/0x107
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05abe42>] skb_dequeue+0x39/0x3f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420d07>] task_running_tick+0x1ab/0x234
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ade4>] current_fs_time+0x4f/0x5b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047db3f>] touch_atime+0x9c/0xcd
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0472894>] pipe_read+0x311/0x31d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044f208>] handle_fasteoi_irq+0x0/0xa2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05e95fe>] udp_ioctl+0x5a/0x5e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05e95a4>] udp_ioctl+0x0/0x5e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: avahi-daemon  S CE1F2DB8  3204  2846   2845                     (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ce1f2dcc 00000082 00000002 ce1f2db8 ce1f2db4 00000000 00000001 cf4987dc 
>Feb 16 15:24:15 rh5cluster2 kernel:        cf9adc9c 00000000 00000001 ccf45560 c06ae440 8ba94daf 0000001e 000007a2 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccf45684 c9204400 00000000 cbeff040 c0472eca ccf45560 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald          S CBA74BE8  1632  2862      1  2863    3072  2845 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cba74bfc 00000086 00000002 cba74be8 cba74be4 00000000 00000000 00000801 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfbe8318 00000000 00000007 cc4f8030 c06ae440 84154c15 00000182 0002fb74 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc4f8154 c9204400 00000000 cc818900 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 last message repeated 11 times
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d089e901>] ext3_get_acl+0x51/0x2ae [ext3]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047b923>] d_alloc+0x141/0x16f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0472eca>] do_lookup+0x4f/0x140
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0475069>] __link_path_walk+0xc02/0xc2c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04e0193>] copy_from_user+0x2b/0x57
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a8c3a>] sys_recvmsg+0x11b/0x1cd
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04742c1>] getname+0x59/0xad
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-runner   S CBEE3BE8  2068  2863   2862  2869               (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cbee3bfc 00000086 00000002 cbee3be8 cbee3be4 00000000 00000001 cbc776e8 
>Feb 16 15:24:15 rh5cluster2 kernel:        cff76000 cff76000 00000003 ccbb9a90 cbac9560 50819f84 00000020 0000e820 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccbb9bb4 c9204400 00000000 cc818200 cbee3c0c ffffffff 00000286 c042ee65 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ee65>] try_to_del_timer_sync+0x44/0x4a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0604ea7>] unix_poll+0x17/0x8f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047789b>] do_sys_poll+0x255/0x327
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0487cc3>] __find_get_block_slow+0x110/0x11a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0892547>] ext3_getblk+0x10b/0x24b [ext3]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0487fdc>] __find_get_block+0x14c/0x156
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ab1dc>] sock_def_readable+0x39/0x63
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0606eb0>] unix_stream_sendmsg+0x240/0x305
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c045b95e>] do_wp_page+0x381/0x3bc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c045cd3f>] __handle_mm_fault+0x860/0x8c6
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04779ae>] sys_poll+0x41/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-addon-ac S CCFA63EC  2172  2869   2863          2873       (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ce1d5dcc 00000082 cc130c68 ccfa63ec d085ab14 cf406600 000000c7 d0891698 
>Feb 16 15:24:15 rh5cluster2 kernel:        00001000 00000000 00000004 ca6a4560 cc4f8030 e17fea4b 0000001e 00004d04 
>Feb 16 15:24:15 rh5cluster2 kernel:        ca6a4684 c9204400 00000000 cc818040 00000000 00000003 00000282 00000246 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d085ab14>] journal_dirty_metadata+0x192/0x1b6 [jbd]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0891698>] __ext3_get_inode_loc+0x10f/0x2e7 [ext3]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0605da2>] unix_stream_recvmsg+0x1e4/0x4aa
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0460254>] do_mmap_pgoff+0x57b/0x6db
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d641>] vfs_read+0xba/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-addon-ke S CBF1BF3C  2172  2873   2863          2881  2869 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cbf1bf50 00000082 00000002 cbf1bf3c cbf1bf38 00000000 ca6a4030 cbf1bf24 
>Feb 16 15:24:15 rh5cluster2 kernel:        c0420d86 ca6a4030 0000000a ca6a4030 c06ae440 cfbdb7a0 000000c1 00003a31 
>Feb 16 15:24:15 rh5cluster2 kernel:        ca6a4154 c9204400 00000000 cbe91e40 00000046 00000000 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420d86>] task_running_tick+0x22a/0x234
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-addon-ke S 00000000  2868  2881   2863          2884  2873 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ce9a0f50 00000086 cc8183c0 00000000 00000000 00000000 00000000 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 ceb4c710 00000001 ce99fa90 ccbb9a90 136f0b55 0000001f 00089ea0 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce99fbb4 c9204400 00000000 cc8183c0 c06bc760 00100073 ce9a0f58 cf8a4c00 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043788b>] prepare_to_wait+0x24/0x3f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-addon-ke S CBF0AF3C  2868  2884   2863          2889  2881 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cbf0af50 00000086 00000002 cbf0af3c cbf0af38 00000000 00000000 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 cbda8e48 00000001 ce99f030 c06ae440 15e8bd1a 0000001f 0008d3b7 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce99f154 c9204400 00000000 cbe91ac0 c06bc760 00100073 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058dfe1>] evdev_read+0xcb/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c058df16>] evdev_read+0x0/0x15a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: hald-addon-st S CF395F24  2172  2889   2863                2884 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cf395f38 00000086 00000002 cf395f24 cf395f20 00000000 cf395efc c920554c 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 c0439f90 00000007 ce414a90 c06ae440 d4df5f34 00000187 00002c74 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce414bb4 c9204400 00000000 cbe91900 9fb902d5 0000017e ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: scsi_eh_1     S C9279A90  3752  2941      7          2947  1294 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cff64f6c 00000046 c92048c0 c9279a90 c0420b4c c9204400 c9279a90 cff64f28 
>Feb 16 15:24:15 rh5cluster2 kernel:        c0420dac 00000000 00000005 cfd9a030 cfd9fa90 77a24451 0000001f 00000456 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd9a154 c9204400 00000000 cbeffac0 c92048c0 c9fc8c84 c060d307 cff64fc8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d086c867>] scsi_error_handler+0x53/0x462 [scsi_mod]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d086c814>] scsi_error_handler+0x0/0x462 [scsi_mod]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: qla2xxx_1_dpc S CFF62F8C  2920  2947      7          2951  2941 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cff62fa0 00000046 00000002 cff62f8c cff62f88 00000000 fffcd8eb cff62f4c 
>Feb 16 15:24:15 rh5cluster2 kernel:        cff62f4c 000003e8 00000009 cfd9a560 c9279030 4def62b3 00000054 000024bd 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd9a684 c9204400 00000000 cfdaf580 00000100 00000000 00000000 d08de28d 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d08de28d>] qla2x00_loop_resync+0x5d/0xae [qla2xxx]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d08d9488>] qla2x00_do_dpc+0x31/0x442 [qla2xxx]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d08d9457>] qla2x00_do_dpc+0x0/0x442 [qla2xxx]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: scsi_wq_1     S 00000003  1832  2951      7          2952  2947 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        c9c4df8c 00000046 cb838848 00000003 cfbea8e0 00000001 cfbea8e0 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000001 c9c4df64 00000009 ccf45030 cc4f8030 10604a58 00000020 000084d3 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccf45154 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 c9c4dfa8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: fc_wq_1       S 00000003  3436  2952      7          2953  2951 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cff67f8c 00000046 c0546f9e 00000003 cdb09ce0 cbfa88c0 cdb09ce0 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000001 cff67f64 00000009 ccf45a90 c9279030 46d4bb79 00000054 00005a11 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccf45bb4 c9204400 00000000 cfdaf580 00000000 00000000 c042f8b3 cff67fa8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0546f9e>] device_release+0x37/0x7c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: fc_dl_1       S 00000086  3784  2953      7          3303  2952 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd1af8c 00000046 00000000 00000086 c0422130 00000000 00000000 cfd9fc60 
>Feb 16 15:24:15 rh5cluster2 kernel:        c9201700 00000000 00000007 cfd15030 cfd9fa90 887ebba7 0000001f 000004e4 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd15154 c9204400 00000000 cbeffac0 cfd06184 00000000 c042f8b3 cfd1afa8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042f8b3>] do_sigaction+0x110/0x14b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: smartd        S C042E00A  3136  3072      1          3076  2862 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        c9fc8f38 00000082 c0453f95 c042e00a c9fc8efc 0002ec18 c9fc8efc c920554c 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 c0439f90 00000005 ce414560 ce414030 940c2758 00000020 00120009 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce414684 c9204400 00000000 cd46ce40 621cda46 000001b9 49c08a46 00000016 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0453f95>] filemap_nopage+0x18b/0x2f7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439f90>] ktime_get_ts+0x16/0x44
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060dd41>] do_nanosleep+0x3b/0x66
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439e6b>] hrtimer_nanosleep+0x50/0x106
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439b3d>] hrtimer_wakeup+0x0/0x18
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0439f6a>] sys_nanosleep+0x49/0x59
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: login         S C0403D2D  2120  3076      1  3217    3077  3072 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd99f54 00000086 c9d55fb8 c0403d2d ce99f560 00000000 c91f1a2c bffb30e0 
>Feb 16 15:24:15 rh5cluster2 kernel:        09d13065 cf8d1ecc 00000008 cfd9f030 ccfe8030 8bb2a884 000000b6 00007638 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd9f154 c9204400 00000000 cd46c200 cd46c200 00000ecc cb4abbfc cfd99f84 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0403d2d>] copy_thread+0x2f/0x1bd
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: mingetty      S C05337DB  2952  3077      1          3078  3076 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ce73fed4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000013 caf9a400 00000006 ca6a4a90 ce414030 a07bc494 00000020 000129cf 
>Feb 16 15:24:15 rh5cluster2 kernel:        ca6a4bb4 c9204400 00000000 cd46cac0 00000000 00000065 cfc40800 cf041b40 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: mingetty      S C05337DB  2952  3078      1          3079  3077 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        c9eeaed4 00000086 00000020 c05337db 00000000 00000000 00001000 cfaa7440 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000013 cb838400 00000005 ce414030 cbac9030 a088996b 00000020 00026877 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce414154 c9204400 00000000 cbeff3c0 00000000 00000065 cfc40800 cf002c80 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: mingetty      S C05337DB  2952  3079      1          3080  3078 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd1ced4 00000086 00000020 c05337db 00000000 00000000 00001000 cf041540 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000013 cdbfa400 00000004 cb4fa560 cd922030 a0f55035 00000020 0002879a 
>Feb 16 15:24:15 rh5cluster2 kernel:        cb4fa684 c9204400 00000000 cd46c900 00000000 00000065 cfc40800 cbdce680 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: mingetty      S C05337DB  2952  3080      1          3081  3079 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ca59fed4 00000082 00000020 c05337db 00000000 00000000 00001000 cec9e0c0 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000013 ce332000 00000003 cfd9fa90 cb4faa90 a1359f74 00000020 00028c54 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd9fbb4 c9204400 00000000 cd46cc80 00000000 00000065 cfc40800 cfac5e40 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05337db>] do_con_write+0x14fc/0x152f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d00>] find_get_page+0x18/0x38
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: mingetty      S CF4D7EC0  2952  3081      1          3311  3080 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cf4d7ed4 00000086 00000002 cf4d7ec0 cf4d7ebc 00000000 00001000 cf364600 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000013 cf68e400 00000002 cbac9560 c06ae440 a41b4d7f 00000020 00008d48 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbac9684 c9204400 00000000 cbeffac0 00000000 00000065 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04270d5>] release_console_sem+0x17e/0x197
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: sshd          S CB809B48  1772  3166   2673  3168    3373       (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cb809b5c 00000082 00000002 cb809b48 cb809b44 00000000 c0423f8f 00000004 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000046 00000000 0000000a cbac9a90 cbac9030 72ca7814 00000159 0000bc5d 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbac9bb4 c9204400 00000000 cd46c740 c0420985 00000000 00000000 00000003 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 last message repeated 3 times
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0419ccd>] smp_apic_timer_interrupt+0x7b/0x86
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04058a4>] apic_timer_interrupt+0x28/0x30
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c045007b>] relay_reset+0x5f/0x68
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060e79d>] _spin_lock_irqsave+0x0/0x27
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: bash          S 00000159  1772  3168   3166  3371               (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cb8dff40 00000082 72d4975e 00000159 cd94d128 00000086 ca11be64 c9d4fa04 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000001 cb8dff28 00000009 cbac9030 cbc3d560 72d4daac 00000159 00029ce2 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbac9154 c9204400 00000000 cd46c580 00000000 00000001 cb8dff4c cb8dff70 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04294f4>] sys_waitpid+0x27/0x2b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: bash          S C9D55EC0  2844  3217   3076                     (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        c9d55ed4 00000082 00000002 c9d55ec0 c9d55ebc 00000000 c0525b0b 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000016 cb112000 00000009 ce99f560 c06ae440 cae63a06 000000c1 0005d72c 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce99f684 c9204400 00000000 cd46c040 00000000 00000000 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b0b>] tty_ldisc_deref+0x55/0x64
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a8be>] read_chan+0x2f4/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0525b48>] tty_ldisc_try+0x2e/0x32
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c052a5ca>] read_chan+0x0/0x53a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527c3b>] tty_read+0x64/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0527bd7>] tty_read+0x0/0xb8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: clvmd         S CEACBB48  2580  3302      1          3318  3312 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ceacbb5c 00000082 00000002 ceacbb48 ceacbb44 00000000 ce487a90 c9204400 
>Feb 16 15:24:15 rh5cluster2 kernel:        ceacbb24 c0423f8f 00000007 ce487a90 c06ae440 f16b2fc0 00000186 000014de 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce487bb4 c9204400 00000000 cce33c80 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0423f8f>] scheduler_tick+0x79/0xd2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 last message repeated 3 times
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0422130>] try_to_wake_up+0x3aa/0x3b4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420dac>] __activate_task+0x1c/0x29
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c06074f7>] unix_write_space+0x47/0x71
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ab3fc>] sock_wfree+0x21/0x36
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05acbea>] __kfree_skb+0xa3/0xf2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0605f47>] unix_stream_recvmsg+0x389/0x4aa
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a77c5>] sock_aio_read+0xfc/0x108
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046cc1d>] do_sync_readv_writev+0xc1/0xfe
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a8e0a>] sys_recvfrom+0x11e/0x12b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d307>] __sched_text_start+0x937/0x9e7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04785fd>] sys_select+0xa0/0x187
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: clvmd         S CE8EFEDC  3484  3318      1          3319  3302 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ce8eff18 00000082 c043c5c4 ce8efedc c0663bcf 00000003 cc140584 c07a8468 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc4c5400 00000001 0000000a cb4faa90 cc6d6030 2a8f1852 00000133 000033bf 
>Feb 16 15:24:15 rh5cluster2 kernel:        cb4fabb4 c9204400 00000000 cce33c80 d0b7f1b4 0000007b ccc200d8 ce8eff58 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043c5c4>] do_futex+0x768/0xb3b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b7f1b4>] __put_lkb+0xba/0xd5 [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b855eb>] device_read+0xa2/0x22b [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b85549>] device_read+0x0/0x22b [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: clvmd         S 00000133  1780  3319      1                3318 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cb603e80 00000082 2a9bdea7 00000133 c047ef4e 00000082 cc248f34 cc248f0c 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 cb603e58 00000007 ccc20030 cfd9f560 2abc8320 00000133 00003158 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccc20154 c9204400 00000000 cce33c80 00000000 00000000 00000003 00000286 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c047ef4e>] mntput_no_expire+0x11/0x6a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043b07d>] get_futex_key+0x3a/0xe3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043b4ad>] futex_wake+0xa3/0xad
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378c2>] add_wait_queue+0x1c/0x28
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043bff0>] do_futex+0x194/0xb3b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04753cf>] do_path_lookup+0x172/0x1c2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046fb37>] vfs_stat_fd+0x19/0x40
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043cab4>] sys_futex+0x11d/0x130
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: dlm_astd      S C041FA15  3724  3303      7          3304  2953 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cc0f2fb0 00000046 cc0f2f6c c041fa15 00000000 00000003 cfa509b4 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfa509b4 00000000 00000009 ce487560 ccde1030 defb2b02 00000147 000001b4 
>Feb 16 15:24:15 rh5cluster2 kernel:        ce487684 c9204400 00000000 cfdaf580 d0b7f125 d0a23923 00000000 d0a238a0 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c041fa15>] __wake_up_common+0x32/0x55
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b7f125>] __put_lkb+0x2b/0xd5 [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a23923>] gdlm_bast+0x83/0x8c [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a238a0>] gdlm_bast+0x0/0x8c [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b7b151>] dlm_astd+0x27/0x11e [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b7b12a>] dlm_astd+0x0/0x11e [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: dlm_scand     S CC508F80  3660  3304      7          3305  3303 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cc508f94 00000046 00000002 cc508f80 cc508f7c 00000000 cc541694 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc1cfd80 cc0730a8 0000000a cafcc560 c06ae440 ecc67fbf 00000186 000026ae 
>Feb 16 15:24:15 rh5cluster2 kernel:        cafcc684 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b8223c>] dlm_scand+0x0/0x3e [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b8226d>] dlm_scand+0x31/0x3e [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: dlm_recv/0    S 00000003  2576  3305      7          3306  3304 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cc572f8c 00000046 00000000 00000003 c08c8960 00000002 c08c8960 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000001 cc572f64 00000009 cc24ea90 ccde1030 1aabe186 00000148 00001494 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc24ebb4 c9204400 00000000 cce333c0 00000000 00000000 ffffffff cc572fa8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: dlm_send      S 00000003  2944  3306      7          3307  3305 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ccaa0f8c 00000046 00000000 00000003 c08c8760 c91b1720 c08c8760 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000001 ccaa0f64 0000000a ccde1030 c9279030 1aac2683 00000148 000044fd 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccde1154 c9204400 00000000 cce333c0 00000000 00000000 ffffffff ccaa0fa8 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043506f>] worker_thread+0xd0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0434f9f>] worker_thread+0x0/0x124
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: dlm_recoverd  S CC5A5F80  3540  3307      7          3333  3306 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cc5a5f94 00000046 00000002 cc5a5f80 cc5a5f7c 00000000 d0b82ca2 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        d0b8370a 00000000 0000000a ccc20560 c06ae440 dc2babe1 00000130 00001c92 
>Feb 16 15:24:15 rh5cluster2 kernel:        ccc20684 c9204400 00000000 cc818ac0 00000001 cc4c5400 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b82ca2>] disallow_sync_reply+0xe/0x24 [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b8370a>] dlm_rcom_status+0xb8/0x18e [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b845f0>] dlm_recoverd+0x49/0x4aa [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0b845a7>] dlm_recoverd+0x0/0x4aa [dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: lock_dlm2     D C9204448  2744  3333      7          3338  3307 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cdbe0e6c 00000046 cc834a5c c9204448 cf78d030 c0420b4c c9204400 cf78d030 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000009 c04209fd 00000009 cb3d6a90 c9279030 1ac5aacf 00000148 00000576 
>Feb 16 15:24:15 rh5cluster2 kernel:        cb3d6bb4 c9204400 00000000 cfdaf580 00000001 c9204400 00000000 c9204400 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04209fd>] task_rq_lock+0x31/0x58
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d45f>] wait_for_completion+0x63/0x87
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437485>] kthread_stop+0x4e/0x6c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a23e25>] gdlm_withdraw+0x96/0xad [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a4a492>] gfs2_withdraw_lockproto+0x16/0x30 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a47ada>] gfs2_lm_withdraw+0x6d/0x8d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a58eff>] gfs2_assert_withdraw_i+0x3b/0x4e [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a4ab0a>] gfs2_meta_inval+0x41/0x86 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a45658>] meta_go_inval+0xd/0x17 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a44dfe>] drop_bh+0xab/0x12a [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a44692>] gfs2_glock_cb+0xae/0x11c [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a24769>] gdlm_thread+0x5ac/0x5f9 [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a241bd>] gdlm_thread+0x0/0x5f9 [lock_dlm]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: gfs2_scand    S CBBACF80  3660  3338      7          3339  3333 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cbbacf94 00000046 00000002 cbbacf80 cbbacf7c 00000000 00000000 cac07fb0 
>Feb 16 15:24:15 rh5cluster2 kernel:        c0406fb4 c0406fb4 0000000a cbc3d030 c06ae440 e785e10e 00000184 000cf805 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbc3d154 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d464>] gfs2_scand+0x0/0x3d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a43b46>] examine_bucket+0x25/0x7b [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d494>] gfs2_scand+0x30/0x3d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: gfs2_glockd   D C9205088  3492  3339      7          3342  3338 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cac07efc 00000046 cc5416c0 c9205088 c9205084 c9204400 cac07f38 c04dbc9c 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 c041ff8c 0000000a cbf9aa90 ce487030 fd524dc3 0000016f 0000154c 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbf9abb4 c9204400 00000000 cbe91040 00000000 00000000 00000080 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04de521>] rwsem_down_failed_common+0x131/0x14c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060e48d>] rwsem_down_write_failed+0x18/0x1f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060e526>] call_rwsem_down_write_failed+0x6/0x8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c043a073>] down_write+0x1c/0x1e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a488c4>] gfs2_log_flush+0x18/0x27c [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a44e63>] drop_bh+0x110/0x12a [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a45943>] inode_go_sync+0x35/0x8c [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a4432f>] run_queue+0x129/0x3a8 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a4530b>] gfs2_glmutex_unlock+0x26/0x38 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a453ac>] gfs2_reclaim_glock+0x8f/0x98 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d3cf>] gfs2_glockd+0x13/0xa8 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d3bc>] gfs2_glockd+0x0/0xa8 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: gfs2_recoverd S CC7DEF80  3660  3342      7          3343  3339 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cc7def94 00000046 00000002 cc7def80 cc7def7c 00000000 c9201700 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000000 cff72030 00000001 cebbba90 c06ae440 6f3fcb7f 00000181 00000c02 
>Feb 16 15:24:15 rh5cluster2 kernel:        cebbbbb4 c9204400 00000000 cce333c0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d4a1>] gfs2_recoverd+0x0/0x3d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d4d1>] gfs2_recoverd+0x30/0x3d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: gfs2_logd     D CEA6CE88  3412  3343      7          3344  3342 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cea6ce9c 00000046 00000002 cea6ce88 cea6ce84 00000000 0001c968 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        00000400 cc4af0ac 0000000a cebbb560 c06ae440 8bad5447 00000149 0000a297 
>Feb 16 15:24:15 rh5cluster2 kernel:        cebbb684 c9204400 00000000 cce333c0 c042e00a cff7775c ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0488586>] sync_buffer+0x30/0x33
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060dae7>] __wait_on_bit+0x33/0x58
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060db6f>] out_of_line_wait_on_bit+0x63/0x6b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0488504>] __wait_on_buffer+0x24/0x27
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0488f22>] sync_dirty_buffer+0x89/0xbe
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a48493>] log_write_header+0x138/0x3b6 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a48a7c>] gfs2_log_flush+0x1d0/0x27c [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d570>] gfs2_logd+0x92/0x11d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d4de>] gfs2_logd+0x0/0x11d [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: gfs2_quotad   S CAD74F70  2540  3344      7          3412  3343 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cad74f84 00000046 00000002 cad74f70 cad74f6c 00000000 cffdd240 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        d0a4476f cf7fa000 0000000a cebbb030 c06ae440 415b3a87 00000187 00000d1e 
>Feb 16 15:24:15 rh5cluster2 kernel:        cebbb154 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a4476f>] gfs2_glock_dq+0x6f/0x79 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d5fb>] gfs2_quotad+0x0/0x129 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a51d15>] gfs2_quota_scan+0x20/0x16a [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:24:15 rh5cluster2 kernel:  [<d0a3d70e>] gfs2_quotad+0x113/0x129 [gfs2]
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: umount        S CA11BF40  2116  3371   3168  3372               (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ca11bf54 00000086 00000002 ca11bf40 ca11bf3c 00000000 c915804c 485f0018 
>Feb 16 15:24:15 rh5cluster2 kernel:        0e84b065 cac027c0 00000007 cbc3d560 c06ae440 74b2a0b7 00000159 00003ef8 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbc3d684 c9204400 00000000 cbe91200 cbe91200 000007c0 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04293c7>] do_wait+0x88c/0x95e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c044cd06>] audit_syscall_entry+0x111/0x143
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04294ca>] sys_wait4+0x31/0x34
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0600000>] xfrm_find_acq+0x15/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: umount.gfs2   D CF279DA0  1980  3372   3371                     (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        cf279db4 00000082 00000002 cf279da0 cf279d9c 00000000 c04573ec 000000d0 
>Feb 16 15:24:15 rh5cluster2 kernel:        cf1ba540 cc4af458 00000007 cbc3da90 c06ae440 76852d96 00000159 000b1f66 
>Feb 16 15:24:15 rh5cluster2 kernel:        cbc3dbb4 c9204400 00000000 cbe91740 c042e00a cff7718c ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04573ec>] __do_page_cache_readahead+0x1a3/0x1bd
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d86>] sync_page+0x0/0x3b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451dbe>] sync_page+0x38/0x3b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060da21>] __wait_on_bit_lock+0x2a/0x52
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451d78>] __lock_page+0x58/0x5e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0452456>] do_generic_mapping_read+0x1d1/0x43b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0454274>] generic_file_aio_read+0x173/0x1a3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0451ba0>] file_read_actor+0x0/0xd1
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046cd64>] do_sync_read+0x0/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: sshd          S CEB60B48  1772  3373   2673  3375          3166 (NOTLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ceb60b5c 00000082 00000002 ceb60b48 ceb60b44 00000000 00000008 0000000a 
>Feb 16 15:24:15 rh5cluster2 kernel:        c0406fb4 00000078 0000000a cb3d6560 cb3d6030 f036270f 00000187 0000c721 
>Feb 16 15:24:15 rh5cluster2 kernel:        cb3d6684 c9204400 00000000 cbe91580 c0420985 00000000 00000000 00000003 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0420985>] __wake_up+0x32/0x43
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d959>] schedule_timeout+0x13/0x8d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0526922>] tty_poll+0x53/0x60
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0477d1a>] do_select+0x369/0x3c0
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04782b5>] __pollwait+0x0/0xac
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:24:15 rh5cluster2 last message repeated 3 times
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05bf863>] __qdisc_run+0x30/0x196
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05b2bd1>] dev_queue_xmit+0x233/0x252
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d1341>] ip_output+0x1e3/0x21e
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d0b37>] ip_queue_xmit+0x3b2/0x3f4
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05ce858>] dst_output+0x0/0x7
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05de975>] tcp_transmit_skb+0x634/0x662
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0455d68>] __alloc_pages+0x59/0x29b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ee07>] lock_timer_base+0x15/0x2f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c042ef10>] __mod_timer+0x91/0x9b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a9dac>] sk_reset_timer+0xc/0x16
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05e02a2>] __tcp_push_pending_frames+0x6eb/0x79f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060e7dc>] _spin_lock_bh+0x8/0x18
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a9c6a>] release_sock+0x12/0x9d
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05d66ae>] tcp_sendmsg+0x8dc/0x9ca
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0478013>] core_sys_select+0x2a2/0x2c3
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c05a76bd>] sock_aio_write+0xf6/0x102
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c046cd21>] do_sync_write+0xc7/0x10a
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0478633>] sys_select+0xd6/0x187
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c040794d>] do_syscall_trace+0x124/0x16b
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:24:15 rh5cluster2 kernel: pdflush       S CA70FF88  3668  3412      7                3344 (L-TLB)
>Feb 16 15:24:15 rh5cluster2 kernel:        ca70ff9c 00000046 00000002 ca70ff88 ca70ff84 00000000 c9204400 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel:        cc818ac0 c04209fd 00000009 cfd9f560 c06ae440 18713c4f 00000175 0000051c 
>Feb 16 15:24:15 rh5cluster2 kernel:        cfd9f684 c9204400 00000000 cc818ac0 c060d3dd 00000000 ffffffff 00000000 
>Feb 16 15:24:15 rh5cluster2 kernel: Call Trace:
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04209fd>] task_rq_lock+0x31/0x58
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c060d3dd>] cond_resched+0x26/0x31
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04570ae>] pdflush+0xb0/0x19f
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:24:15 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:24:15 rh5cluster2 kernel:  =======================
>Feb 16 15:26:06 rh5cluster2 kernel: SysRq : Show Blocked State
>Feb 16 15:26:06 rh5cluster2 kernel: 
>Feb 16 15:26:06 rh5cluster2 kernel:                          free                        sibling
>Feb 16 15:26:06 rh5cluster2 kernel:   task             PC    stack   pid father child younger older
>Feb 16 15:26:06 rh5cluster2 kernel: pdflush       D CFD74EEC  2420   167      7           168   166 (L-TLB)
>Feb 16 15:26:06 rh5cluster2 kernel:        cfd74f00 00000046 00000002 cfd74eec cfd74ee8 00000000 00000000 c0406fb4 
>Feb 16 15:26:06 rh5cluster2 kernel:        cfd74f10 0015b186 0000000a cfd75a90 c06ae440 4197ccdf 000001a2 00000ccb 
>Feb 16 15:26:06 rh5cluster2 kernel:        cfd75bb4 c9204400 00000000 cc818ac0 c07a5c00 00000286 ffffffff 00000000 
>Feb 16 15:26:06 rh5cluster2 kernel: Call Trace:
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0406fb4>] do_IRQ+0xce/0xe2
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060d9b6>] schedule_timeout+0x70/0x8d
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0450a41>] delayacct_end+0x70/0x77
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042e6ec>] process_timeout+0x0/0x5
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060d504>] io_schedule_timeout+0x3f/0x63
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c045a78e>] congestion_wait+0x50/0x64
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0456d64>] wb_kupdate+0x98/0xde
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0456ffe>] pdflush+0x0/0x19f
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0457107>] pdflush+0x109/0x19f
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0456ccc>] wb_kupdate+0x0/0xde
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:26:06 rh5cluster2 kernel:  =======================
>Feb 16 15:26:06 rh5cluster2 kernel: lock_dlm2     D C9204448  2744  3333      7          3338  3307 (L-TLB)
>Feb 16 15:26:06 rh5cluster2 kernel:        cdbe0e6c 00000046 cc834a5c c9204448 cf78d030 c0420b4c c9204400 cf78d030 
>Feb 16 15:26:06 rh5cluster2 kernel:        00000009 c04209fd 00000009 cb3d6a90 c9279030 1ac5aacf 00000148 00000576 
>Feb 16 15:26:06 rh5cluster2 kernel:        cb3d6bb4 c9204400 00000000 cfdaf580 00000001 c9204400 00000000 c9204400 
>Feb 16 15:26:06 rh5cluster2 kernel: Call Trace:
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0420b4c>] enqueue_task+0x29/0x39
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04209fd>] task_rq_lock+0x31/0x58
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060d45f>] wait_for_completion+0x63/0x87
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437485>] kthread_stop+0x4e/0x6c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a23e25>] gdlm_withdraw+0x96/0xad [lock_dlm]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a4a492>] gfs2_withdraw_lockproto+0x16/0x30 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a47ada>] gfs2_lm_withdraw+0x6d/0x8d [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a58eff>] gfs2_assert_withdraw_i+0x3b/0x4e [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a4ab0a>] gfs2_meta_inval+0x41/0x86 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a45658>] meta_go_inval+0xd/0x17 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a44dfe>] drop_bh+0xab/0x12a [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a44692>] gfs2_glock_cb+0xae/0x11c [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04378e4>] remove_wait_queue+0x16/0x22
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a24769>] gdlm_thread+0x5ac/0x5f9 [lock_dlm]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042213a>] default_wake_function+0x0/0xc
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a241bd>] gdlm_thread+0x0/0x5f9 [lock_dlm]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:26:06 rh5cluster2 kernel:  =======================
>Feb 16 15:26:06 rh5cluster2 kernel: gfs2_glockd   D C9205088  3492  3339      7          3342  3338 (L-TLB)
>Feb 16 15:26:06 rh5cluster2 kernel:        cac07efc 00000046 cc5416c0 c9205088 c9205084 c9204400 cac07f38 c04dbc9c 
>Feb 16 15:26:06 rh5cluster2 kernel:        00000000 c041ff8c 0000000a cbf9aa90 ce487030 fd524dc3 0000016f 0000154c 
>Feb 16 15:26:06 rh5cluster2 kernel:        cbf9abb4 c9204400 00000000 cbe91040 00000000 00000000 00000080 00000000 
>Feb 16 15:26:06 rh5cluster2 kernel: Call Trace:
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04dbc9c>] __next_cpu+0x12/0x21
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c041ff8c>] find_busiest_group+0x1b4/0x4c3
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04de521>] rwsem_down_failed_common+0x131/0x14c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060e48d>] rwsem_down_write_failed+0x18/0x1f
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060e526>] call_rwsem_down_write_failed+0x6/0x8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c043a073>] down_write+0x1c/0x1e
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a488c4>] gfs2_log_flush+0x18/0x27c [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a44e63>] drop_bh+0x110/0x12a [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a45943>] inode_go_sync+0x35/0x8c [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a4432f>] run_queue+0x129/0x3a8 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a4530b>] gfs2_glmutex_unlock+0x26/0x38 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a453ac>] gfs2_reclaim_glock+0x8f/0x98 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a3d3cf>] gfs2_glockd+0x13/0xa8 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a3d3bc>] gfs2_glockd+0x0/0xa8 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:26:06 rh5cluster2 kernel:  =======================
>Feb 16 15:26:06 rh5cluster2 kernel: gfs2_logd     D CEA6CE88  3412  3343      7          3344  3342 (L-TLB)
>Feb 16 15:26:06 rh5cluster2 kernel:        cea6ce9c 00000046 00000002 cea6ce88 cea6ce84 00000000 0001c968 00000000 
>Feb 16 15:26:06 rh5cluster2 kernel:        00000400 cc4af0ac 0000000a cebbb560 c06ae440 8bad5447 00000149 0000a297 
>Feb 16 15:26:06 rh5cluster2 kernel:        cebbb684 c9204400 00000000 cce333c0 c042e00a cff7775c ffffffff 00000000 
>Feb 16 15:26:06 rh5cluster2 kernel: Call Trace:
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0488586>] sync_buffer+0x30/0x33
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060dae7>] __wait_on_bit+0x33/0x58
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0488556>] sync_buffer+0x0/0x33
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060db6f>] out_of_line_wait_on_bit+0x63/0x6b
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0488504>] __wait_on_buffer+0x24/0x27
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0488f22>] sync_dirty_buffer+0x89/0xbe
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a48493>] log_write_header+0x138/0x3b6 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a48a7c>] gfs2_log_flush+0x1d0/0x27c [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a47ffd>] gfs2_ail1_empty+0x17/0x78 [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a3d570>] gfs2_logd+0x92/0x11d [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04208f3>] complete+0x39/0x48
>Feb 16 15:26:06 rh5cluster2 kernel:  [<d0a3d4de>] gfs2_logd+0x0/0x11d [gfs2]
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437678>] kthread+0xb0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04375c8>] kthread+0x0/0xd8
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0405a27>] kernel_thread_helper+0x7/0x10
>Feb 16 15:26:06 rh5cluster2 kernel:  =======================
>Feb 16 15:26:06 rh5cluster2 kernel: umount.gfs2   D CF279DA0  1980  3372   3371                     (NOTLB)
>Feb 16 15:26:06 rh5cluster2 kernel:        cf279db4 00000082 00000002 cf279da0 cf279d9c 00000000 c04573ec 000000d0 
>Feb 16 15:26:06 rh5cluster2 kernel:        cf1ba540 cc4af458 00000007 cbc3da90 c06ae440 76852d96 00000159 000b1f66 
>Feb 16 15:26:06 rh5cluster2 kernel:        cbc3dbb4 c9204400 00000000 cbe91740 c042e00a cff7718c ffffffff 00000000 
>Feb 16 15:26:06 rh5cluster2 kernel: Call Trace:
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c04573ec>] __do_page_cache_readahead+0x1a3/0x1bd
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c042e00a>] getnstimeofday+0x30/0xb6
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060d915>] io_schedule+0x3a/0x5c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0451d86>] sync_page+0x0/0x3b
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0451dbe>] sync_page+0x38/0x3b
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c060da21>] __wait_on_bit_lock+0x2a/0x52
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0451d78>] __lock_page+0x58/0x5e
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437776>] wake_bit_function+0x0/0x3c
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0452456>] do_generic_mapping_read+0x1d1/0x43b
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0454274>] generic_file_aio_read+0x173/0x1a3
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0451ba0>] file_read_actor+0x0/0xd1
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c046ce2b>] do_sync_read+0xc7/0x10a
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c046bbe2>] nameidata_to_filp+0x24/0x33
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0437741>] autoremove_wake_function+0x0/0x35
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c046cd64>] do_sync_read+0x0/0x10a
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c046d62d>] vfs_read+0xa6/0x128
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c046da2b>] sys_read+0x41/0x67
>Feb 16 15:26:06 rh5cluster2 kernel:  [<c0404e4c>] syscall_call+0x7/0xb
>Feb 16 15:26:06 rh5cluster2 kernel:  =======================
>Feb 16 15:26:40 rh5cluster2 kernel: SysRq : Show Regs

Actions: View

Attachments on bug 229081: 148240