Description of problem: Running several benchmarks(connectathon,iozone,fss_stress,fsx,ffsb) simultaneously under a loopback NFSV4 mount (one machine used as client and server), those benchmarks hang after one hour and half or two hours because the mounted repertory no longer accessible. Version-Release number of selected component (if applicable): Linux version 2.6.18-53.el5 (brewbuilder.redhat.com) (gcc version 4.1.2 20070626 (Red Hat 4.1.2-14)) #1 SMP Wed Oct 10 16:34:19 EDT 2007 How reproducible: Run the connectathon benchmark in a while true loop: while true do ./runtests -a -t /mnt/nosec/cthon_nfs4_nfs1_gb date sleep 30 done Run the iozone benchmark in a while true loop: while true do ./iozone -+q 30 -ace -r 64 -g 2G -i 0 -i 1 -i 2 -f /mnt/nosec/file1_nfs1_alto_gb date sleep 30 done Run the fsx benchmark in a while true loop: while true; do ./fsx-linux -N 50000 /mnt/nosec/fsx_nfs1_nfs4_gb; date; sleep 30; done Run the fssstress benchmark in a while true loop: while true; do for p in 2 4 8 16 32 ; do for n in 1 2 4 8 16 32 64 128; do ./fsstress -d /mnt/nosec/fsstress_nfs1_nfs4_gb -p $p -n $n ; date; sleep 60; done; done; done Run the ffsb benchmark in a while true loop: while true do ../ffsb profile_stress_test date sleep 120 done After one hour and half , benchmarks hang with the following in /var/log/messages if we do echo t > /proc/sysrq-trigger kernel 64a9d>] do_page_fault+0x4eb/0x81d kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff800b27ae>] audit_syscall_entry+0x14d/0x180 kernel: [<ffffffff80049ff7>] sys_poll+0x2c/0x33 kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: rpc.statd S 7fffffffffffffff 0 2379 1 2404 2350 (NOTLB) kernel: ffff81006de6da18 0000000000000086 ffff81007f616690 0000004400000000 kernel: 0000000000000008 ffff8100762b07a0 ffff81007723b100 0000002dfffd85e0 kernel: 000000000000352d ffff8100762b0988 0000000000000000 ffff81000000fc10 kernel: Call Trace: kernel: [<ffffffff8024135d>] inet_fill_ifaddr+0x1f2/0x27b kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff8001334f>] tcp_poll+0x1c/0x12f kernel: [<ffffffff80011341>] do_select+0x3fe/0x462 kernel: [<ffffffff8001e234>] __pollwait+0x0/0xe2 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe last message repeated 2 times kernell: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff800532b4>] skb_queue_tail+0x17/0x3e kernel: [<ffffffff80012874>] sock_def_readable+0x10/0x5f kernel: [<ffffffff8025a870>] unix_dgram_sendmsg+0x43d/0x4cf kernel: [<ffffffff80052bd5>] sock_sendmsg+0xf3/0x110 kernel: [<ffffffff8000e630>] link_path_walk+0xd3/0xe5 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff800dd334>] core_sys_select+0x1bc/0x265 kernel: [<ffffffff8000c1a5>] _atomic_dec_and_lock+0x39/0x57 kernel: [<ffffffff8002c86b>] mntput_no_expire+0x19/0x89 kernel: [<ffffffff80209578>] sys_sendto+0x11c/0x14f kernel: [<ffffffff80016004>] sys_select+0xc1/0x17c kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: kernel: rpc.idmapd S 00002aaaaaece560 0 2404 1 2454 2379 (NOTLB) kernel: ffff81006df5de78 0000000000000086 0000000000000292 0000000000000003 kernel: 000000000000000a ffff810037fd77e0 ffff81007f9f4100 00000a054a8cd83e kernel: 000000000000151d ffff810037fd79c8 ffff810000000001 ffff81007f9f4100 kernel: Call Trace: kernel: [<ffffffff80061839>] schedule_timeout+0x8a/0xad kernel: [<ffffffff80092ada>] process_timeout+0x0/0x5 kernel: [<ffffffff800ebc37>] sys_epoll_wait+0x184/0x3f9 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: last message repeated 5 times kernel: kernel: rpciod/0 S ffff810074da00c0 0 2526 27 2527 1996 (L-TLB) kernel: ffff81006d01be70 0000000000000046 00000000fffffff5 ffff81006bef6b90 kernel: 000000000000000a ffff8100780d6080 ffff81007f232820 000004da29e49afa kernel: 0000000000001e7c ffff8100780d6268 0000000000000000 0000000000000003 kernel: Call Trace: kernel: [<ffffffff8002dfa2>] __wake_up+0x38/0x4f kernel: [<ffffffff8833dc30>] :sunrpc:rpc_async_schedule+0x0/0x5 kernel: [<ffffffff8004b2d1>] run_workqueue+0xba/0xe5 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff80047c09>] worker_thread+0x0/0x122 kernel: [<ffffffff80047cc9>] worker_thread+0xc0/0x122 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800321d8>] kthread+0xfe/0x132 kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800320da>] kthread+0x0/0x132 kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: rpciod/1 S ffffffff8009b283 0 2527 27 3130 2526 (L-TLB) kernel: ffff81006da9de70 0000000000000046 00000000fffffff5 ffff81006bef6128 kernel: 000000000000000a ffff8100755a30c0 ffff81007f9f4100 000009fcd6b10c51 kernel: 0000000000000a73 ffff8100755a32a8 0000000000000001 0000000000000003 kernel: Call Trace: kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff80047c09>] worker_thread+0x0/0x122 kernel: [<ffffffff80047cc9>] worker_thread+0xc0/0x122 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800321d8>] kthread+0xfe/0x132 kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800320da>] kthread+0x0/0x132 kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: kernel: lockd S 7fffffffffffffff 0 3129 1 3131 3126 (L-TLB) kernel: ffff81007737be60 0000000000000046 0000000000000000 0000000000000021 kernel: 0000000000000001 ffff8100755a3820 ffffffff802dcae0 00000069dfc08918 kernel: 0000000000000c3f ffff8100755a3a08 ffff810000000000 ffff810075bb47a0 kernel: Call Trace: kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff80062aa9>] _spin_lock_bh+0x9/0x14 kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff88341e7e>] :sunrpc:svc_recv+0x306/0x4bc kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8837608b>] :lockd:lockd+0x0/0x275 kernel: [<ffffffff8837608b>] :lockd:lockd+0x0/0x275 kernel: [<ffffffff883761da>] :lockd:lockd+0x14f/0x275 kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8837608b>] :lockd:lockd+0x0/0x275 kernel: [<ffffffff8837608b>] :lockd:lockd+0x0/0x275 kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: kernel: nfsd4 S ffffffff8009b283 0 3130 27 13680 2527 (L-TLB) kernel: ffff810061eb7e70 0000000000000046 0000000000000286 ffffffff8001c40f kernel: 000000000000000a ffff8100761b6080 ffffffff802dcae0 00000a0585a47ce6 kernel: 0000000000000af8 ffff8100761b6268 0000000000000000 0000000000000003 kernel: Call Trace: kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff80047c09>] worker_thread+0x0/0x122 kernel: [<ffffffff80047cc9>] worker_thread+0xc0/0x122 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800321d8>] kthread+0xfe/0x132 kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff800320da>] kthread+0x0/0x132 kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd D ffff810067eea4c4 0 3131 1 3132 3129 (L-TLB) kernel: ffff810061f597a0 0000000000000046 ffff810075509e80 0000000800000001 kernel: 000000000000000a ffff8100765dc860 ffff81007807e820 000004b4574d695d kernel: 00000000000256b8 ffff8100765dca48 0000000000000000 ffffffff800e0578 kernel: Call Trace: kernel: [<ffffffff800e0578>] inode_wait+0x0/0xd kernel: [<ffffffff800619c4>] __wait_on_bit+0x60/0x6f kernel: [<ffffffff800e0578>] inode_wait+0x0/0xd kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff8003d03b>] lock_timer_base+0x1b/0x3c kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd S ffffffff88451fb5 0 3132 1 3133 3131 (L-TLB) kernel: ffff810061ce7980 0000000000000046 0000000000000007 ffffffff80025003 kernel: 0000000000000009 ffff81007f2367a0 ffff81007f9f4100 000004b0472a57c3 kernel: 0000000000005e64 ffff81007f236988 0000000000000001 ffff81a00fd00000 kernel: Call Trace: kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff88451fd7>] :nfs:nfs_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff88451f9b>] :nfs:nfs_wait_on_request+0x56/0x70 kernel: [<ffffffff884556d6>] :nfs:nfs_wait_on_requests_locked+0x70/0xca kernel: [<ffffffff8845684e>] :nfs:nfs_sync_inode_wait+0x60/0x1db kernel: [<ffffffff8844ce8c>] :nfs:nfs_release_page+0x2c/0x4d kernel: [<ffffffff800c2489>] shrink_inactive_list+0x4d8/0x7e7 kernel: [<ffffffff800c1a53>] isolate_lru_pages+0x98/0xbf kernel: [<ffffffff80045d41>] __pagevec_release+0x19/0x22 kernel: [<ffffffff800c1e90>] shrink_active_list+0x416/0x426 kernel: [<ffffffff800129ad>] shrink_zone+0xe3/0x108 kernel: [<ffffffff800c2e5f>] try_to_free_pages+0x163/0x275 kernel: [<ffffffff8000eff7>] __alloc_pages+0x1b1/0x2b4 kernel: [<ffffffff88341c7c>] :sunrpc:svc_recv+0x104/0x4bc kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd D ffff810067eea4c4 0 3133 1 3134 3132 (L-TLB) kernel: ffff810061f957a0 0000000000000046 ffffffff8009b283 ffffffff80060f29 kernel: 0000000000000009 ffff81007800c0c0 ffff8100787ba7a0 000004b1ec639b49 kernel: 000000000000b7ad ffff81007800c2a8 ffffffff00000001 ffff81000a37e400 kernel: Call Trace: kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff80087d09>] __activate_task+0x27/0x39 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff8003d03b>] lock_timer_base+0x1b/0x3c kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd D ffff810067eea4c4 0 3134 1 3135 3133 (L-TLB) kernel: ffff810061f5f7a0 0000000000000046 ffff810055bb14f0 ffff81001d482670 kernel: 0000000000000009 ffff81007807e820 ffff810075497100 000004b4574f58ee kernel: 000000000000bae1 ffff81007807ea08 0000000300000000 ffff81007e4b9ed8 kernel: Call Trace: kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 ernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff80021b63>] tcp_transmit_skb+0x64a/0x682 kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd S ffffffff88451fb5 0 3135 1 3136 3134 (L-TLB) kernel: ffff810061e5b030 0000000000000046 000000000000bf18 ffff810061fc12b0 kernel: 0000000000000008 ffff810061fc10c0 ffffffff802dcae0 000004b04729aed0 kernel: 000000000000e747 ffff810061fc12a8 0000000000000000 ffff81000bb1d310 kernel: Call Trace: kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff88451fd7>] :nfs:nfs_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff88451f9b>] :nfs:nfs_wait_on_request+0x56/0x70 kernel: [<ffffffff884556d6>] :nfs:nfs_wait_on_requests_locked+0x70/0xca kernel: [<ffffffff8845684e>] :nfs:nfs_sync_inode_wait+0x60/0x1db kernel: [<ffffffff8844ce8c>] :nfs:nfs_release_page+0x2c/0x4d kernel: [<ffffffff800c2489>] shrink_inactive_list+0x4d8/0x7e7 kernel: [<ffffffff800c1a53>] isolate_lru_pages+0x98/0xbf kernel: [<ffffffff80045d41>] __pagevec_release+0x19/0x22 kernel: [<ffffffff800c1e90>] shrink_active_list+0x416/0x426 kernel: [<ffffffff800129ad>] shrink_zone+0xe3/0x108 kernel: [<ffffffff800c2e5f>] try_to_free_pages+0x163/0x275 kernel: [<ffffffff8000eff7>] __alloc_pages+0x1b1/0x2b4 kernel: [<ffffffff8000f8fd>] generic_file_buffered_write+0x19d/0x6d8 kernel: [<ffffffff883ac68f>] :nfsd:nfsd4_encode_fattr+0x13a2/0x1440 kernel: [<ffffffff8000ddc8>] current_fs_time+0x3b/0x40 kernel: [<ffffffff80015db5>] __generic_file_aio_write_nolock+0x36c/0x3b8 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff800733d3>] smp_send_reschedule+0x4e/0x53 kernel: [<ffffffff800bdf22>] __generic_file_write_nolock+0x8f/0xa8 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80061bf0>] __mutex_lock_slowpath+0x5d/0x90 kernel: [<ffffffff80061a94>] mutex_lock+0xd/0x1d kernel: [<ffffffff800bdf83>] generic_file_writev+0x48/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff80021b63>] tcp_transmit_skb+0x64a/0x682 kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd D ffff810067eea4c4 0 3136 1 3137 3135 (L-TLB) kernel: ffff810061c657a0 0000000000000046 ffffffff8009b283 ffffffff80060f29 kernel: 0000000000000009 ffff81006c0cb080 ffff81007800c0c0 000004b18aaea968 kernel: 000000000000c907 ffff81006c0cb268 ffffffff00000001 ffff81000a37e400 kernel: Call Trace: kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff80087d09>] __activate_task+0x27/0x39 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff88340f5d>] :sunrpc:svc_tcp_data_ready+0x40/0x6c kernel: [<ffffffff8003d03b>] lock_timer_base+0x1b/0x3c kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: kernel: nfsd D ffff810061c87948 0 3137 1 3138 3136 (L-TLB) kernel: ffff810061c877a0 0000000000000046 ffff81005c33d000 ffff81007d264b70 kernel: 0000000000000009 ffff8100787ba7a0 ffff81007f9f4100 000004b1ec65fb98 kernel: 0000000000005161 ffff8100787ba988 ffff810000000001 ffffffff800bdf22 kernel: Call Trace: kernel: [<ffffffff800bdf22>] __generic_file_write_nolock+0x8f/0xa8 kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8002b610>] local_bh_enable+0x90/0xa1 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80033dd3>] ip_queue_xmit+0x3fb/0x450 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff8003d03b>] lock_timer_base+0x1b/0x3c kernel: [<ffffffff8001c40f>] __mod_timer+0xb0/0xbe kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: nfsd D ffff810067eea4c4 0 3138 1 3141 3137 (L-TLB) kernel: ffff810061cf17a0 0000000000000046 ffffffff8833dc30 ffffffff80060f29 kernel: 0000000000000009 ffff8100777bd100 ffff81006c0cb080 000004b09932bc07 kernel: 000000000000b68f ffff8100777bd2e8 ffffffff00000001 ffff81000a37e400 kernel: Call Trace: kernel: [<ffffffff8833dc30>] :sunrpc:rpc_async_schedule+0x0/0x5 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff80087d09>] __activate_task+0x27/0x39 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff80061be8>] __mutex_lock_slowpath+0x55/0x90 kernel: [<ffffffff80061c32>] .text.lock.mutex+0xf/0x14 kernel: [<ffffffff800bdf72>] generic_file_writev+0x37/0xa2 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff800d5eae>] do_readv_writev+0x176/0x295 kernel: [<ffffffff80017880>] do_sync_write+0x0/0x104 kernel: [<ffffffff8839a8a0>] :nfsd:nfsd_acceptable+0x0/0xd8 kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff8839b77d>] :nfsd:nfsd_vfs_write+0xf2/0x2e1 kernel: [<ffffffff80095d48>] set_current_groups+0x159/0x164 kernel: [<ffffffff8839bfa9>] :nfsd:nfsd_write+0x6d/0xd5 kernel: [<ffffffff883a82e5>] :nfsd:nfsd4_proc_compound+0x149a/0x1655 kernel: [<ffffffff80031648>] sock_common_recvmsg+0x2d/0x42 kernel: [<ffffffff801b1b56>] loopback_xmit+0xa7/0xad kernel: [<ffffffff8002ffe2>] sock_recvmsg+0x101/0x120 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff80052373>] tcp_cong_avoid+0xe/0x1d kernel: [<ffffffff80025003>] tcp_ack+0x1719/0x1871 kernel: [<ffffffff8001b75a>] tcp_rcv_established+0x86c/0x917 kernel: [<ffffffff80209ce9>] kernel_recvmsg+0x3b/0x4d kernel: [<ffffffff8833fd96>] :sunrpc:svc_recvfrom+0x89/0xbf kernel: [<ffffffff88340000>] :sunrpc:svc_sock_enqueue+0x234/0x247 kernel: [<ffffffff88341afa>] :sunrpc:svc_tcp_recvfrom+0x7a2/0x820 kernel: [<ffffffff883425be>] :sunrpc:ip_map_match+0x19/0x30 kernel: [<ffffffff800c35e8>] zone_statistics+0x3e/0x6d kernel: [<ffffffff88345c6a>] :sunrpc:sunrpc_cache_lookup+0x4b/0x127 kernel: [<ffffffff883aa44f>] :nfsd:nfs4svc_decode_compoundargs+0x362/0xe54 kernel: [<ffffffff883981c4>] :nfsd:nfsd_dispatch+0xd7/0x198 kernel: [<ffffffff8833f514>] :sunrpc:svc_process+0x44d/0x70b kernel: [<ffffffff800625bf>] __down_read+0x12/0x92 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff883986fb>] :nfsd:nfsd+0x1ae/0x2db kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8839854d>] :nfsd:nfsd+0x0/0x2db kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: rpc.mountd S 7fffffffffffffff 0 3141 1 3169 3138 (NOTLB) kernel: ffff810061dbba18 0000000000000082 0000000000000000 000000448021486d kernel: 0000000000000009 ffff810076247860 ffff810050346040 000005b4aaa177b8 kernel: 00000000000073ff ffff810076247a48 ffff810000000000 ffff810061dbbac8 kernel: Call Trace: kernel: [<ffffffff8804e0f7>] :ext3:ext3_get_blocks_handle+0xc6/0x9f1 kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff8001334f>] tcp_poll+0x1c/0x12f kernel: [<ffffffff80011341>] do_select+0x3fe/0x462 kernel: [<ffffffff8001e234>] __pollwait+0x0/0xe2 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe last message repeated 4 times kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff8009b283>] keventd_create_kthread+0x0/0x61 kernel: [<ffffffff880322e8>] :jbd:__journal_file_buffer+0x13e/0x243 kernel: [<ffffffff880322e8>] :jbd:__journal_file_buffer+0x13e/0x243 kernel: [<ffffffff800dd334>] core_sys_select+0x1bc/0x265 kernel: [<ffffffff88033e55>] :jbd:journal_dirty_metadata+0x1b5/0x1e3 kernel: [<ffffffff880327ac>] :jbd:journal_stop+0x1f3/0x1ff kernel: [<ffffffff80016004>] sys_select+0xc1/0x17c kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: nfsv4-svc S 7fffffffffffffff 0 3169 1 3141 (L-TLB) kernel: ffff810061eb1e60 0000000000000046 0000000000000000 ffffffff80062c51 kernel: 0000000000000004 ffff810037f61100 ffff8100772880c0 00000079f975b25d kernel: 0000000000000dd2 ffff810037f612e8 0000000000000000 ffff810037f61100 kernel: Call Trace: kernel: [<ffffffff80062c51>] __reacquire_kernel_lock+0x2c/0x45 kernel: [<ffffffff80044cf9>] try_to_wake_up+0x407/0x418 kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff80062aa9>] _spin_lock_bh+0x9/0x14 kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff88341e7e>] :sunrpc:svc_recv+0x306/0x4bc kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff88469dc5>] :nfs:nfs_callback_svc+0x0/0x15a kernel: [<ffffffff88469dc5>] :nfs:nfs_callback_svc+0x0/0x15a kernel: [<ffffffff88469e85>] :nfs:nfs_callback_svc+0xc0/0x15a kernel: [<ffffffff8005bfb1>] child_rip+0xa/0x11 kernel: [<ffffffff88469dc5>] :nfs:nfs_callback_svc+0x0/0x15a kernel: [<ffffffff88469dc5>] :nfs:nfs_callback_svc+0x0/0x15a kernel: [<ffffffff8005bfa7>] child_rip+0x0/0x11 kernel: kernel: kernel: iozone S ffffffff80060765 0 4084 3645 (NOTLB) kernel: ffff810060da3d48 0000000000000082 ffff81006d235100 ffffffff800172d6 kernel: 000000000000000a ffff8100608c9820 ffff810061fc10c0 000004b025c545fa kernel: 0000000000002153 ffff8100608c9a08 ffff810000000000 ffff81003dc5c7e0 kernel: Call Trace: kernel: [<ffffffff800172d6>] cache_grow+0x356/0x395 kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff88451fd7>] :nfs:nfs_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff88451fb5>] :nfs:nfs_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff88451f9b>] :nfs:nfs_wait_on_request+0x56/0x70 kernel: [<ffffffff884556d6>] :nfs:nfs_wait_on_requests_locked+0x70/0xca kernel: [<ffffffff88455a82>] :nfs:nfs_flush_inode+0x5c/0x6f kernel: [<ffffffff8845744e>] :nfs:nfs_writepages+0xe6/0x13e kernel: [<ffffffff80059036>] do_writepages+0x20/0x2f kernel: [<ffffffff8004d593>] __filemap_fdatawrite_range+0x51/0x5b kernel: [<ffffffff8004de5b>] do_fsync+0x2f/0xa3 kernel: [<ffffffff800d644e>] __do_fsync+0x23/0x36 kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: ffsb S 7fffffffffffffff 0 16092 5048 16138 (NOTLB) kernel: ffff81001d48fd48 0000000000000082 0000000000000001 0000000000000000 kernel: 0000000000000009 ffff8100765c8040 ffff810070e24820 000004850a30bcb8 kernel: 0000000000000c09 ffff8100765c8228 0000000000000001 0000000000000001 kernel: Call Trace: kernel: [<ffffffff8002dfa2>] __wake_up+0x38/0x4f kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff8003d7a1>] do_futex+0x1da/0xbc7 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff80039d8a>] tty_ldisc_deref+0x68/0x7b kernel: [<ffffffff8009e935>] sys_futex+0x101/0x123 kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: ffsb S ffff810070e24820 0 16138 5048 16139 16092 (NOTLB) kernel: ffff81002503fd48 0000000000000082 0000000300000000 0000000000000096 kernel: 0000000000000007 ffff810070e24820 ffffffff802dcae0 000004cadc527330 kernel: 00000000000013ef ffff810070e24a08 ffff810000000000 ffffffff8008692a kernel: Call Trace: kernel: [<ffffffff8008692a>] __wake_up_common+0x3e/0x68 kernel: [<ffffffff800617cd>] schedule_timeout+0x1e/0xad kernel: [<ffffffff800476f7>] add_wait_queue+0x24/0x34 kernel: [<ffffffff8003d7a1>] do_futex+0x1da/0xbc7 kernel: [<ffffffff80087ccd>] enqueue_task+0x41/0x56 kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8833dc30>] :sunrpc:rpc_async_schedule+0x0/0x5 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb kernel: [<ffffffff8009e935>] sys_futex+0x101/0x123 kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: ffsb D ffff81006df081c0 0 16139 5048 16140 16138 (NOTLB) kernel: ffff810072ca7c58 0000000000000082 ffff810072ca7bf8 ffff810072ca7d18 kernel: 0000000000000009 ffff810070e240c0 ffff81007f2367a0 000004b02705c7bf kernel: 0000000000005e5b ffff810070e242a8 0000000000000001 000000003b376ed0 kernel: Call Trace: kernel: [<ffffffff884574a6>] :nfs:nfs_writepage+0x0/0x283 kernel: [<ffffffff884561c7>] :nfs:nfs_wait_on_write_congestion+0x172/0x18f kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff884573ce>] :nfs:nfs_writepages+0x66/0x13e kernel: [<ffffffff80059036>] do_writepages+0x20/0x2f kernel: [<ffffffff8004d593>] __filemap_fdatawrite_range+0x51/0x5b kernel: [<ffffffff800bdd7d>] filemap_write_and_wait+0x17/0x32 kernel: [<ffffffff8844ede6>] :nfs:nfs_sync_mapping+0x25/0x45 kernel: [<ffffffff8844f5c1>] :nfs:nfs_revalidate_mapping+0xa4/0x1b4 kernel: [<ffffffff8844d632>] :nfs:nfs_file_read+0x45/0x97 kernel: [<ffffffff8000c927>] do_sync_read+0xc7/0x104 kernel: [<ffffffff8009b446>] autoremove_wake_function+0x0/0x2e kernel: [<ffffffff8000b270>] vfs_read+0xcb/0x171 kernel: [<ffffffff800114f7>] sys_read+0x45/0x6e kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: fsstress S 00000000fffffff6 0 16621 4109 16623 (NOTLB) kernel: ffff810047995ee8 0000000000000086 ffff810047995e68 ffff810037ff19c0 kernel: 0000000000000009 ffff81004ad7a080 ffff8100772880c0 00000495c048acc0 kernel: 0000000000002b39 ffff81004ad7a268 ffff810000000000 ffffffff800fa2ce kernel: Call Trace: kernel: [<ffffffff800fa2ce>] proc_flush_task+0x4e/0x1f6 kernel: [<ffffffff8008be59>] sched_exit+0x9f/0xaa kernel: [<ffffffff80027dea>] do_wait+0x936/0xa0a kernel: [<ffffffff800884ac>] default_wake_function+0x0/0xe kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: fsstress S ffffffff8833d37e 0 16623 16621 16627 (NOTLB) kernel: ffff81001fadda58 0000000000000082 0000000000000246 ffffffff8001bf35 kernel: 0000000000000007 ffff8100606d00c0 ffff81007f9f4100 000004b475ad244a kernel: 0000000000003525 ffff8100606d02a8 ffff810000000001 ffff81007f616670 kernel: Call Trace: kernel: [<ffffffff8001bf35>] cfq_set_request+0x246/0x389 kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff8833d3a0>] :sunrpc:rpc_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff8833daf2>] :sunrpc:__rpc_execute+0x110/0x24e kernel: [<ffffffff88338596>] :sunrpc:rpc_call_sync+0x7e/0xa8 kernel: [<ffffffff8845c36e>] :nfs:_nfs4_proc_access+0xb4/0xe9 kernel: [<ffffffff8845d1a2>] :nfs:nfs4_proc_access+0x21/0x47 kernel: [<ffffffff8844b27f>] :nfs:nfs_permission+0x136/0x1cb kernel: [<ffffffff8000d41d>] permission+0x81/0xc8 kernel: [<ffffffff8000965b>] __link_path_walk+0x173/0xf42 kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80060f29>] thread_return+0x0/0xeb ernel: [<ffffffff8000e5b9>] link_path_walk+0x5c/0xe5 kernel: [<ffffffff8000c7e8>] do_path_lookup+0x270/0x2e8 kernel: [<ffffffff800232cf>] __path_lookup_intent_open+0x56/0x97 kernel: [<ffffffff8001a7ae>] open_namei+0xb9/0x6fd kernel: [<ffffffff80047dee>] wait_on_page_writeback_range+0xc3/0x10e kernel: [<ffffffff800270ce>] do_filp_open+0x1c/0x38 kernel: [<ffffffff80019523>] do_sys_open+0x44/0xbe kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: kernel: fsstress S ffffffff8833d37e 0 16627 16621 16628 16623 (NOTLB) kernel: ffff8100577abb28 0000000000000082 ffff81007fb54030 0000000000000000 kernel: 0000000000000007 ffff81005e71d860 ffff81007f9f4100 000004b476b0c9c0 kernel: 0000000000001e45 ffff81005e71da48 0000000000000001 0000000100000002 kernel: Call Trace: kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff8833d3a0>] :sunrpc:rpc_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff8833daf2>] :sunrpc:__rpc_execute+0x110/0x24e kernel: [<ffffffff88338596>] :sunrpc:rpc_call_sync+0x7e/0xa8 kernel: [<ffffffff8833e9aa>] :sunrpc:rpcauth_lookup_credcache+0x12e/0x24c kernel: [<ffffffff8845c36e>] :nfs:_nfs4_proc_access+0xb4/0xe9 kernel: [<ffffffff8844ae5c>] :nfs:nfs_access_free_entry+0x15/0x1f kernel: [<ffffffff8845d1a2>] :nfs:nfs4_proc_access+0x21/0x47 kernel: [<ffffffff8844b27f>] :nfs:nfs_permission+0x136/0x1cb kernel: [<ffffffff8000d41d>] permission+0x81/0xc8 kernel: kernel: kernel: fsx-linux S ffffffff8833d37e 0 17658 4583 (NOTLB) kernel: ffff81006f777bf8 0000000000000082 ffff81006f777bc8 ffffffff80051deb kernel: 000000000000000a ffff810076df0080 ffffffff802dcae0 000004b1ec65836b kernel: 0000000000005a3c ffff810076df0268 0000000000000000 ffffffff80062aa9 kernel: Call Trace: kernel: [<ffffffff80051deb>] tcp_init_tso_segs+0x2b/0x3a kernel: [<ffffffff80062aa9>] _spin_lock_bh+0x9/0x14 kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff8833d3a0>] :sunrpc:rpc_wait_bit_interruptible+0x22/0x28 kernel: [<ffffffff800619a4>] __wait_on_bit+0x40/0x6f kernel: [<ffffffff8833d37e>] :sunrpc:rpc_wait_bit_interruptible+0x0/0x28 kernel: [<ffffffff80061a3f>] out_of_line_wait_on_bit+0x6c/0x78 kernel: [<ffffffff8009b474>] wake_bit_function+0x0/0x23 kernel: [<ffffffff8833daf2>] :sunrpc:__rpc_execute+0x110/0x24e kernel: [<ffffffff88338596>] :sunrpc:rpc_call_sync+0x7e/0xa8 kernel: [<ffffffff8845d353>] :nfs:nfs4_proc_getattr+0x83/0xa2 kernel: [<ffffffff8844e89f>] :nfs:__nfs_revalidate_inode+0x14a/0x294 kernel: [<ffffffff88451fa8>] :nfs:nfs_wait_on_request+0x63/0x70 kernel: [<ffffffff88455715>] :nfs:nfs_wait_on_requests_locked+0xaf/0xca kernel: [<ffffffff88456871>] :nfs:nfs_sync_inode_wait+0x83/0x1db kernel: [<ffffffff8844f4fa>] :nfs:nfs_getattr+0x80/0xa3 kernel: [<ffffffff80021c2d>] vfs_fstat+0x29/0x3a kernel: [<ffffffff8002a6b9>] sys_newfstat+0x11/0x29 kernel: [<ffffffff8005b229>] tracesys+0x71/0xe0 kernel: [<ffffffff8005b28d>] tracesys+0xd5/0xe0 kernel: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
This looks to me that it could be memory contention problem which is a k
This request was evaluated by Red Hat Product Management for inclusion, but this component is not scheduled to be updated in the current Red Hat Enterprise Linux release. If you would like this request to be reviewed for the next minor release, ask your support representative to set the next rhel-x.y flag to "?".
Unfortunately the previous automated notification about the non-inclusion of this request in Red Hat Enterprise Linux 5.3 used the wrong text template. It should have read: this request has been reviewed by Product Management and is not planned for inclusion in the current minor release of Red Hat Enterprise Linux. If you would like this request to be reviewed for the next minor release, ask your support representative to set the next rhel-x.y flag to "?" or raise an exception.
Development Management has reviewed and declined this request. You may appeal this decision by reopening this request.