Bug 240077
Summary: | Panic under high disk I/O (stack overflow: XFS + LVM) | ||||||
---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Nathan Valentine <nvalentine> | ||||
Component: | kernel | Assignee: | Eric Sandeen <esandeen> | ||||
Status: | CLOSED DUPLICATE | QA Contact: | Brian Brock <bbrock> | ||||
Severity: | high | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 6 | ||||||
Target Milestone: | --- | ||||||
Target Release: | --- | ||||||
Hardware: | i686 | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2007-08-09 19:42:38 UTC | Type: | --- | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
Nathan Valentine
2007-05-14 22:16:52 UTC
Created attachment 154693 [details]
loghost capture of kern
xfs on lvm volumes will do that. Kernels from http://www.linuxant.com/driverloader/wlan/full/downloads.php may work but the 16K stacks can cause other problems (out of memory when trying to start a new task.) Setting /proc/sys/vm/min_free_kbytes to 16000 can help prevent that but it's not guaranteed. Is ext3 on LVM known to exhibit this behavior as well? Actually, I guess a better question would be "Where in the Fedora community should I have found documentation about this and similar issues?" (In reply to comment #3) > Is ext3 on LVM known to exhibit this behavior as well? ext3 should be fine. Raw devices probably would work best though. (Stack overflows with lvm + xfs are well-known in the Linux kernel community. Probably not known about enough in the user groups though.) I'll put this under my name - if nothing else I may dup it to another bug that might get WONTFIXed eventually, unfortunately. Stacked IO + XFS + 4k stacks is tough; lots of stack reductions have been done in xfs but it's unlikely that this is ever going to be 100% robust. 16K stacks are probably overkill; default 8K stacks (kernel config option on x86) or 8k stacks on x86_64 will probably work fine. As we get more stacked filesystems in the kernel (think unionfs, ecryptfs) the 4k stacks may get more interesting too. FWIW, I was able to reproduce this panic on both 8k and 16k stacks. We eventually "solved" the problem by one of two methods depending on the role of the server: 1) Swap XFS for ext3. 2) Move XFS filesystem from LVM to raw partitions. Since we made these changes, things have been stable and performant. But I agree with the assessment that it is likely that stacked storage management is not going away and thus this will continue to be a problem. Thanks for your help. On 16k stacks? Yikes... ok, that's unexpected. Do you happen to have any stack traces from that kernel? I wonder if you're hitting recursion.... I'll look more closely at the kernel log you have posted already. I'm a bit skeptical of it, it seems to show *hundreds* of functions on the stack...? Even with the false positives from dump_stack() it doesn't seem quite right. Unfortunately, I didn't save any of the debugging information from testing alternative stack sizes. Duping this to an earlier 4k+xfs+lvm bug, though the root cause may be slightly different it's the same issue as far as I can tell - and one without a good solution I'm afraid. *** This bug has been marked as a duplicate of 227331 *** |