Bug 644085 - Kernel oopses when using fuse with sshfs doing heavy i/o
Summary: Kernel oopses when using fuse with sshfs doing heavy i/o
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-10-18 21:29 UTC by Christian
Modified: 2010-12-03 12:10 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2010-12-03 12:10:15 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Christian 2010-10-18 21:29:39 UTC
Description of problem:

A kernel oops during otherwise normal use of the system.

Version-Release number of selected component (if applicable):

Fedora 12 machine, using kernel-2.6.32.21-168.fc12.x86_64, fuse-2.8.4-1.fc12.x86_64, and fuse-sshfs-2.2-5.fc12.x86_64.

How reproducible:

We suspect fuse, specifically fuse using sshfs, to be the source
of the problem. The machine is running fine until we sshfs-mount
remote files and access them heavily, when the oops occurs 
approximately once per day.

Steps to Reproduce:
1. Mount remote files via sshfs
2. Do heavy i/o on the mountpoint
3. Wait for kernel oops
  
Actual results:

Here are two oopses, directly from /var/log/messages:

Oct 15 17:04:26 junkshot kernel: ------------[ cut here ]------------
Oct 15 17:04:26 junkshot kernel: kernel BUG at fs/inode.c:323!
Oct 15 17:04:26 junkshot kernel: invalid opcode: 0000 [#1] SMP 
Oct 15 17:04:26 junkshot kernel: last sysfs file: /sys/devices/system/cpu/cpu15/cache/index2/shared_cpu_map
Oct 15 17:04:26 junkshot kernel: CPU 13 
Oct 15 17:04:26 junkshot kernel: Modules linked in: fuse ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 dm_multipath uinput joydev power_meter iTCO_wdt bnx2 iTCO_vendor_support dcdbas ses enclosure serio_raw megaraid_sas [last unloaded: speedstep_lib]
Oct 15 17:04:26 junkshot kernel: Pid: 147, comm: kswapd1 Not tainted 2.6.32.21-168.fc12.x86_64 #1 PowerEdge R410
Oct 15 17:04:26 junkshot kernel: RIP: 0010:[<ffffffff81131671>]  [<ffffffff81131671>] clear_inode+0x3b/0xf9
Oct 15 17:04:26 junkshot kernel: RSP: 0018:ffff8803270e7ba0  EFLAGS: 00010202
Oct 15 17:04:26 junkshot kernel: RAX: 0000000000000000 RBX: ffff880297450f00 RCX: 0000000000000000
Oct 15 17:04:26 junkshot kernel: RDX: 0000000000000000 RSI: 0000000000000140 RDI: ffff880297450f00
Oct 15 17:04:26 junkshot kernel: RBP: ffff8803270e7bb0 R08: ffffea000907d2c0 R09: ffff8803270e7b90
Oct 15 17:04:26 junkshot kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Oct 15 17:04:26 junkshot kernel: R13: ffff8803270e7cdc R14: 0000000000000000 R15: ffff8803270e7c60
Oct 15 17:04:26 junkshot kernel: FS:  0000000000000000(0000) GS:ffff8800074c0000(0000) knlGS:0000000000000000
Oct 15 17:04:26 junkshot kernel: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Oct 15 17:04:26 junkshot kernel: CR2: 00007f00fada51d0 CR3: 0000000001001000 CR4: 00000000000006e0
Oct 15 17:04:26 junkshot kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 15 17:04:26 junkshot kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 15 17:04:26 junkshot kernel: Process kswapd1 (pid: 147, threadinfo ffff8803270e6000, task ffff8803288aaec0)
Oct 15 17:04:26 junkshot kernel: Stack:
Oct 15 17:04:26 junkshot kernel: ffff8803270e7bb0 ffff880297450f00 ffff8803270e7bd0 ffffffff81131c68
Oct 15 17:04:26 junkshot kernel: <0> ffff880297450f00 ffff880297450f00 ffff8803270e7bf0 ffffffff81130e3c
Oct 15 17:04:26 junkshot kernel: <0> ffff8803270e7bf0 ffff8802997d7bc0 ffff8803270e7c10 ffffffff8112e420
Oct 15 17:04:26 junkshot kernel: Call Trace:
Oct 15 17:04:26 junkshot kernel: [<ffffffff81131c68>] generic_delete_inode+0xfb/0x171
Oct 15 17:04:26 junkshot kernel: [<ffffffff81130e3c>] iput+0x66/0x6a
Oct 15 17:04:26 junkshot kernel: [<ffffffff8112e420>] dentry_iput+0xb8/0xca
Oct 15 17:04:26 junkshot kernel: [<ffffffff8112e4fa>] d_kill+0x26/0x46
Oct 15 17:04:26 junkshot kernel: [<ffffffff8112e7e8>] __shrink_dcache_sb+0x2ce/0x370
Oct 15 17:04:26 junkshot kernel: [<ffffffff8112e976>] shrink_dcache_memory+0xec/0x179
Oct 15 17:04:26 junkshot kernel: [<ffffffff810e10e9>] shrink_slab+0xe3/0x16c
Oct 15 17:04:26 junkshot kernel: [<ffffffff810e3239>] balance_pgdat+0x3d0/0x5a7
Oct 15 17:04:26 junkshot kernel: [<ffffffff810e0857>] ? isolate_pages_global+0x0/0x1f7
Oct 15 17:04:26 junkshot kernel: [<ffffffff810e353a>] kswapd+0x12a/0x12c
Oct 15 17:04:26 junkshot kernel: [<ffffffff81074977>] ? autoremove_wake_function+0x0/0x39
Oct 15 17:04:26 junkshot kernel: [<ffffffff810e3410>] ? kswapd+0x0/0x12c
Oct 15 17:04:26 junkshot kernel: [<ffffffff8107468a>] kthread+0x7f/0x87
Oct 15 17:04:26 junkshot kernel: [<ffffffff81012d6a>] child_rip+0xa/0x20
Oct 15 17:04:26 junkshot kernel: [<ffffffff8107460b>] ? kthread+0x0/0x87
Oct 15 17:04:26 junkshot kernel: [<ffffffff81012d60>] ? child_rip+0x0/0x20
Oct 15 17:04:26 junkshot kernel: Code: be 40 01 00 00 48 89 fb 48 c7 c7 3b a8 5e 81 e8 db 44 f1 ff e8 f3 5e 32 00 48 89 df e8 20 f6 00 00 48 83 bb 68 01 00 00 00 74 04 <0f> 0b eb fe 48 8b 83 18 02 00 00 a8 20 75 04 0f 0b eb fe a8 40 
Oct 15 17:04:26 junkshot kernel: RIP  [<ffffffff81131671>] clear_inode+0x3b/0xf9
Oct 15 17:04:26 junkshot kernel: RSP <ffff8803270e7ba0>
Oct 15 17:04:26 junkshot kernel: ---[ end trace 4d32094979468bea ]---


Oct 14 17:45:02 junkshot kernel: ------------[ cut here ]------------
Oct 14 17:45:02 junkshot kernel: kernel BUG at fs/inode.c:323!
Oct 14 17:45:02 junkshot kernel: invalid opcode: 0000 [#1] SMP 
Oct 14 17:45:02 junkshot kernel: last sysfs file: /sys/devices/system/cpu/cpu15/cache/index2/shared_cpu_map
Oct 14 17:45:02 junkshot kernel: CPU 0 
Oct 14 17:45:02 junkshot kernel: Modules linked in: fuse ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 dm_multipath uinput dcdbas power_meter bnx2 iTCO_wdt ses iTCO_vendor_support serio_raw joydev enclosure megaraid_sas [last unloaded: speedstep_lib]
Oct 14 17:45:02 junkshot kernel: Pid: 12044, comm: ssh Not tainted 2.6.32.21-168.fc12.x86_64 #1 PowerEdge R410
Oct 14 17:45:02 junkshot kernel: RIP: 0010:[<ffffffff81131671>]  [<ffffffff81131671>] clear_inode+0x3b/0xf9
Oct 14 17:45:02 junkshot kernel: RSP: 0018:ffff88027fd0f708  EFLAGS: 00010202
Oct 14 17:45:02 junkshot kernel: RAX: 0000000000000000 RBX: ffff88023776f200 RCX: 0000000000000000
Oct 14 17:45:02 junkshot kernel: RDX: 0000000000000000 RSI: 0000000000000140 RDI: ffff88023776f200
Oct 14 17:45:02 junkshot kernel: RBP: ffff88027fd0f718 R08: ffff88027fd0f568 R09: ffff88027fd0f6f8
Oct 14 17:45:02 junkshot kernel: R10: ffff88021cfceba8 R11: 0000000000000001 R12: 0000000000000000
Oct 14 17:45:02 junkshot kernel: R13: ffff88027fd0f844 R14: 0000000000000000 R15: ffff88027fd0f7c8
Oct 14 17:45:02 junkshot kernel: FS:  00007fe44f9527c0(0000) GS:ffff8801b5800000(0000) knlGS:0000000000000000
Oct 14 17:45:02 junkshot kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 14 17:45:02 junkshot kernel: CR2: 00007fa5e80333ef CR3: 000000028de6b000 CR4: 00000000000006f0
Oct 14 17:45:02 junkshot kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 14 17:45:02 junkshot kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Oct 14 17:45:02 junkshot kernel: Process ssh (pid: 12044, threadinfo ffff88027fd0e000, task ffff8801b5a4aec0)
Oct 14 17:45:02 junkshot kernel: Stack:
Oct 14 17:45:02 junkshot kernel: ffff88027fd0f718 ffff88023776f200 ffff88027fd0f738 ffffffff81131c68
Oct 14 17:45:02 junkshot kernel: <0> ffff88023776f200 ffff88023776f200 ffff88027fd0f758 ffffffff81130e3c
Oct 14 17:45:02 junkshot kernel: <0> ffff88027fd0f758 ffff88021449de00 ffff88027fd0f778 ffffffff8112e420
Oct 14 17:45:02 junkshot kernel: Call Trace:
Oct 14 17:45:02 junkshot kernel: [<ffffffff81131c68>] generic_delete_inode+0xfb/0x171
Oct 14 17:45:02 junkshot kernel: [<ffffffff81130e3c>] iput+0x66/0x6a
Oct 14 17:45:02 junkshot kernel: [<ffffffff8112e420>] dentry_iput+0xb8/0xca
Oct 14 17:45:02 junkshot kernel: [<ffffffff8112e4fa>] d_kill+0x26/0x46
Oct 14 17:45:02 junkshot kernel: [<ffffffff8112e7e8>] __shrink_dcache_sb+0x2ce/0x370
Oct 14 17:45:02 junkshot kernel: [<ffffffff8112e976>] shrink_dcache_memory+0xec/0x179
Oct 14 17:45:02 junkshot kernel: [<ffffffff810e10e9>] shrink_slab+0xe3/0x16c
Oct 14 17:45:02 junkshot kernel: [<ffffffff810e27be>] do_try_to_free_pages+0x265/0x3a0
Oct 14 17:45:02 junkshot kernel: [<ffffffff810e29f3>] try_to_free_pages+0x6e/0x70
Oct 14 17:45:02 junkshot kernel: [<ffffffff810e0857>] ? isolate_pages_global+0x0/0x1f7
Oct 14 17:45:02 junkshot kernel: [<ffffffff810dc9c3>] __alloc_pages_nodemask+0x3f3/0x630
Oct 14 17:45:02 junkshot kernel: [<ffffffff8110c9f4>] kmalloc_large_node+0x5f/0x9c
Oct 14 17:45:02 junkshot kernel: [<ffffffff8110e224>] __kmalloc_node_track_caller+0x31/0x11e
Oct 14 17:45:02 junkshot kernel: [<ffffffff813a91c4>] ? sock_alloc_send_pskb+0xc5/0x2d0
Oct 14 17:45:02 junkshot kernel: [<ffffffff813ad476>] __alloc_skb+0x80/0x170
Oct 14 17:45:02 junkshot kernel: [<ffffffff813a91c4>] sock_alloc_send_pskb+0xc5/0x2d0
Oct 14 17:45:02 junkshot kernel: [<ffffffff813a93e4>] sock_alloc_send_skb+0x15/0x17
Oct 14 17:45:02 junkshot kernel: [<ffffffff8143067f>] unix_stream_sendmsg+0x119/0x2c8
Oct 14 17:45:02 junkshot kernel: [<ffffffff813a57a7>] __sock_sendmsg+0x61/0x6c
Oct 14 17:45:02 junkshot kernel: [<ffffffff813a7a41>] sock_aio_write+0xc0/0xd4
Oct 14 17:45:02 junkshot kernel: [<ffffffff8111bd14>] do_sync_write+0xe8/0x125
Oct 14 17:45:02 junkshot kernel: [<ffffffff81074977>] ? autoremove_wake_function+0x0/0x39
Oct 14 17:45:02 junkshot kernel: [<ffffffff8104fc1c>] ? finish_task_switch+0x48/0xb8
Oct 14 17:45:02 junkshot kernel: [<ffffffff811e6f21>] ? security_file_permission+0x16/0x18
Oct 14 17:45:02 junkshot kernel: [<ffffffff8111c2eb>] vfs_write+0xc1/0x10b
Oct 14 17:45:02 junkshot kernel: [<ffffffff8111c3f5>] sys_write+0x4a/0x6e
Oct 14 17:45:02 junkshot kernel: [<ffffffff81011d32>] system_call_fastpath+0x16/0x1b
Oct 14 17:45:02 junkshot kernel: Code: be 40 01 00 00 48 89 fb 48 c7 c7 3b a8 5e 81 e8 db 44 f1 ff e8 f3 5e 32 00 48 89 df e8 20 f6 00 00 48 83 bb 68 01 00 00 00 74 04 <0f> 0b eb fe 48 8b 83 18 02 00 00 a8 20 75 04 0f 0b eb fe a8 40 
Oct 14 17:45:02 junkshot kernel: RIP  [<ffffffff81131671>] clear_inode+0x3b/0xf9
Oct 14 17:45:02 junkshot kernel: RSP <ffff88027fd0f708>
Oct 14 17:45:02 junkshot kernel: ---[ end trace c3a5eb969977b09f ]---


Expected results:

Normal i/o using fuse and sshfs.

Additional info:

The machine has remained available and usable after the oopses in both cases.

Comment 1 Miklos Szeredi 2010-10-27 09:04:39 UTC
Looks like the same bug:

https://bugzilla.kernel.org/show_bug.cgi?id=15927

Comment 2 Bug Zapper 2010-11-03 09:25:55 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 3 Bug Zapper 2010-12-03 12:10:15 UTC
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.