Description of problem: I'm seeing a lot of these messages on our Fedora 9 NFS/samba/apache/amanda server. Otherwise, doesn't seem to be having any other trouble. Here's the most recent: Jun 26 05:01:37 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 26 05:01:37 saga kernel: Pid: 206, comm: kswapd0 Not tainted 2.6.25.6-55.fc9.i686 #1 Jun 26 05:01:37 saga kernel: [<c04668ce>] __alloc_pages+0x2cf/0x2e6 Jun 26 05:01:37 saga kernel: [<c0466978>] __get_free_pages+0x3f/0x4f Jun 26 05:01:37 saga kernel: [<c047f6de>] __kmalloc_track_caller+0x31/0xca Jun 26 05:01:37 saga kernel: [<c05b566b>] ? __netdev_alloc_skb+0x17/0x34 Jun 26 05:01:37 saga kernel: [<c05b4b61>] __alloc_skb+0x49/0xf8 Jun 26 05:01:37 saga kernel: [<c05b566b>] __netdev_alloc_skb+0x17/0x34 Jun 26 05:01:37 saga kernel: [<f8a5074e>] e1000_alloc_rx_buffers+0x94/0x26e [e1000] Jun 26 05:01:37 saga kernel: [<f8a50d50>] e1000_clean_rx_irq+0x428/0x460 [e1000] Jun 26 05:01:37 saga kernel: [<f8a52b5c>] e1000_clean+0x5f/0x1fa [e1000] Jun 26 05:01:37 saga kernel: [<c05b7af4>] net_rx_action+0xa6/0x1b2 Jun 26 05:01:37 saga kernel: [<c042b2a1>] __do_softirq+0x79/0xe7 Jun 26 05:01:37 saga kernel: [<c0407ddb>] do_softirq+0x74/0xb5 Jun 26 05:01:37 saga kernel: [<c045c797>] ? handle_fasteoi_irq+0x0/0xaf Jun 26 05:01:37 saga kernel: [<c042b0a9>] irq_exit+0x38/0x6b Jun 26 05:01:37 saga kernel: [<c0407ec8>] do_IRQ+0xac/0xc4 Jun 26 05:01:37 saga kernel: [<c04065e7>] common_interrupt+0x23/0x28 Jun 26 05:01:37 saga kernel: [<c046007b>] ? marker_entry_remove_probe+0xf8/0x194 Jun 26 05:01:37 saga kernel: [<c04600d8>] ? marker_entry_remove_probe+0x155/0x194 Jun 26 05:01:37 saga kernel: [<c0469906>] ? remove_mapping+0xb4/0xde Jun 26 05:01:37 saga kernel: [<c0469dcb>] shrink_page_list+0x49b/0x544 Jun 26 05:01:37 saga kernel: [<c0469fa0>] shrink_inactive_list+0x12c/0x309 Jun 26 05:01:37 saga kernel: [<c0492e2c>] ? dispose_list+0xb6/0xc9 Jun 26 05:01:37 saga kernel: [<f8dfc6a7>] ? nfs_access_cache_shrinker+0x1f/0x198 [nfs] Jun 26 05:01:37 saga kernel: [<c046a238>] shrink_zone+0xbb/0xda Jun 26 05:01:37 saga kernel: [<c046a690>] kswapd+0x305/0x423 Jun 26 05:01:37 saga kernel: [<c046938f>] ? isolate_pages_global+0x0/0x3e Jun 26 05:01:37 saga kernel: [<c0437b0f>] ? autoremove_wake_function+0x0/0x33 Jun 26 05:01:37 saga kernel: [<c046a38b>] ? kswapd+0x0/0x423 Jun 26 05:01:37 saga kernel: [<c04378ad>] kthread+0x3b/0x61 Jun 26 05:01:37 saga kernel: [<c0437872>] ? kthread+0x0/0x61 Jun 26 05:01:37 saga kernel: [<c0406833>] kernel_thread_helper+0x7/0x10 Jun 26 05:01:37 saga kernel: ======================= Jun 26 05:01:37 saga kernel: Mem-info: Jun 26 05:01:37 saga kernel: DMA per-cpu: Jun 26 05:01:37 saga kernel: CPU 0: hi: 0, btch: 1 usd: 0 Jun 26 05:01:37 saga kernel: CPU 1: hi: 0, btch: 1 usd: 0 Jun 26 05:01:37 saga kernel: CPU 2: hi: 0, btch: 1 usd: 0 Jun 26 05:01:37 saga kernel: CPU 3: hi: 0, btch: 1 usd: 0 Jun 26 05:01:37 saga kernel: Normal per-cpu: Jun 26 05:01:37 saga kernel: CPU 0: hi: 186, btch: 31 usd: 176 Jun 26 05:01:37 saga kernel: CPU 1: hi: 186, btch: 31 usd: 167 Jun 26 05:01:37 saga kernel: CPU 2: hi: 186, btch: 31 usd: 36 Jun 26 05:01:37 saga kernel: CPU 3: hi: 186, btch: 31 usd: 51 Jun 26 05:01:37 saga kernel: HighMem per-cpu: Jun 26 05:01:37 saga kernel: CPU 0: hi: 42, btch: 7 usd: 17 Jun 26 05:01:37 saga kernel: CPU 1: hi: 42, btch: 7 usd: 36 Jun 26 05:01:37 saga kernel: CPU 2: hi: 42, btch: 7 usd: 2 Jun 26 05:01:37 saga kernel: CPU 3: hi: 42, btch: 7 usd: 30 Jun 26 05:01:37 saga kernel: Active:55353 inactive:72236 dirty:6322 writeback:962 unstable:0 Jun 26 05:01:37 saga kernel: free:3361 slab:88379 mapped:5918 pagetables:537 bounce:0 Jun 26 05:01:37 saga kernel: DMA free:3588kB min:68kB low:84kB high:100kB active:384kB inactive:560kB present:16256kB pages_scanned:0 all_unreclaimable? no Jun 26 05:01:37 saga kernel: lowmem_reserve[]: 0 873 999 999 Jun 26 05:01:37 saga kernel: Normal free:9408kB min:3744kB low:4680kB high:5616kB active:162304kB inactive:223244kB present:894080kB pages_scanned:0 all_unreclaimable? no Jun 26 05:01:37 saga kernel: lowmem_reserve[]: 0 0 1012 1012 Jun 26 05:01:37 saga kernel: HighMem free:448kB min:128kB low:260kB high:396kB active:58724kB inactive:65140kB present:129540kB pages_scanned:0 all_unreclaimable? no Jun 26 05:01:37 saga kernel: lowmem_reserve[]: 0 0 0 0 Jun 26 05:01:37 saga kernel: DMA: 73*4kB 176*8kB 118*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3620kB Jun 26 05:01:37 saga kernel: Normal: 1153*4kB 545*8kB 1*16kB 5*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 9468kB Jun 26 05:01:37 saga kernel: HighMem: 64*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 448kB Jun 26 05:01:37 saga kernel: 94251 total pagecache pages Jun 26 05:01:37 saga kernel: Swap cache: add 348, delete 330, find 35/58 Jun 26 05:01:37 saga kernel: Free swap = 4192212kB Jun 26 05:01:37 saga kernel: Total swap = 4192944kB Jun 26 05:01:37 saga kernel: Free swap: 4192212kB Jun 26 05:01:37 saga kernel: 262016 pages of RAM Jun 26 05:01:37 saga kernel: 32640 pages of HIGHMEM Jun 26 05:01:37 saga kernel: 36231 reserved pages Jun 26 05:01:37 saga kernel: 118513 pages shared Jun 26 05:01:37 saga kernel: 18 pages swap cached Jun 26 05:01:37 saga kernel: 6389 pages dirty Jun 26 05:01:37 saga kernel: 962 pages writeback Jun 26 05:01:37 saga kernel: 5918 pages mapped Jun 26 05:01:37 saga kernel: 88379 pages slab Jun 26 05:01:37 saga kernel: 537 pages pagetables System did seem to start doing some io, but I'm really not sure what. Probably multiple systems backing up to a nfs volume. Linux 2.6.25.6-55.fc9.i686 (saga.cora.nwra.com) 06/26/2008 12:00:03 AM CPU %user %nice %system %iowait %steal %idle 04:50:02 AM all 0.11 0.00 0.63 0.93 0.00 98.34 05:00:04 AM all 0.15 0.00 0.65 1.17 0.00 98.03 05:10:02 AM all 0.34 0.00 4.08 24.56 0.00 71.02 That 5am time is popular: May 30 05:01:03 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 1 05:04:26 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 2 05:04:23 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 3 05:01:05 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 4 05:04:22 saga kernel: swapper: page allocation failure. order:3, mode:0x4020 Jun 6 05:04:24 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 8 05:00:58 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 9 05:04:25 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 11 04:02:25 saga kernel: swapper: page allocation failure. order:3, mode:0x4020 Jun 11 05:01:02 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 12 05:04:21 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 13 05:01:04 saga kernel: swapper: page allocation failure. order:3, mode:0x4020 Jun 14 05:01:02 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 16 05:04:24 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 17 20:37:27 saga kernel: tar: page allocation failure. order:3, mode:0x4020 Jun 18 05:04:20 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 19 05:01:02 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 19 12:11:35 saga kernel: swapper: page allocation failure. order:3, mode:0x4020 Jun 20 05:01:00 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 21 04:02:16 saga kernel: kswapd0: page allocation failure. order:3, mode:0x4020 Jun 23 05:01:01 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Jun 25 05:01:03 saga kernel: nfsd: page allocation failure. order:3, mode:0x4020 Version-Release number of selected component (if applicable): kernel-2.6.25.6-55.fc9.i686
The good news is this is just diagnostic, and not indication of an actual failure. The problem is we're trying to satisfy a fairly large allocation of a certain type, which after the machine has been up for a while, is difficult. We retry atomic allocations when they get failed, so the system keeps going.. That said, the driver shouldn't be asking for such obscene amounts of memory, which is almost guaranteed to fail. There is ongoing work upstream to make the family of e100* drivers not do this.
Thank you very much for the explanation.
Andrew Morton propsed this problem fixes about three month ago. unfortunately, it was rejected... http://kerneltrap.org/mailarchive/linux-kernel/2008/4/2/1323484
One thing that was suggested was to see if the e1000e driver supports your hardware. Apparently it uses a different allocation strategy, which should avoid this problem.
How do I find out if e1000e will work? I've tried to get it to load (modprobe.conf), but I keep getting e1000.
This message is a reminder that Fedora 9 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 9. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '9'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 9's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 9 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
I hope to post a fix to the mailing lists shortly for e1000 to avoid order > 0 allocations when using jumbo frames.
Fedora 9 changed to end-of-life (EOL) status on 2009-07-10. Fedora 9 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. Thank you for reporting this bug and we are sorry it could not be fixed.