Bug 1258499 - aarch64 mustang: unhandled level [23] translation faults
aarch64 mustang: unhandled level [23] translation faults
Status: CLOSED INSUFFICIENT_DATA
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
23
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Peter Robinson
Fedora Extras Quality Assurance
:
Depends On:
Blocks: ARM64/F-ExcludeArch-aarch64
  Show dependency treegraph
 
Reported: 2015-08-31 09:58 EDT by Peter Robinson
Modified: 2017-07-03 08:12 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-10-26 13:00:51 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Peter Robinson 2015-08-31 09:58:09 EDT
It's taken me a while to be able to actually capture this.

We see this on the aarch64 koji builders after a period of uptime which ultimately takes the network offline. Everything looks fine on the console, everything comes back fine with a reboot. Time varies I believe depending on how many builds we're doing, and I suspect how much they're being hammered by builds. In this one the network was still around but very slow.

This is the 4.0.8 kernel, but we've seen the issues on older ones, I'm just moving the builders to 4.1.5 now.


[ 6689.041334] HelpLinker: page allocation failure: order:0, mode:0x200020
[ 6689.047929] CPU: 6 PID: 4561 Comm: HelpLinker Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[ 6689.056758] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[ 6689.064033] Call trace:
[ 6689.066475] [<fffffe0000097648>] dump_backtrace+0x0/0x164
[ 6689.071851] [<fffffe00000977c8>] show_stack+0x1c/0x28
[ 6689.076882] [<fffffe00007547b0>] dump_stack+0x74/0x94
[ 6689.081910] [<fffffe00001a128c>] warn_alloc_failed+0xe8/0x140
[ 6689.087628] [<fffffe00001a4d94>] __alloc_pages_nodemask+0x5f4/0x8d4
[ 6689.093864] [<fffffe00001efdd0>] new_slab+0x300/0x464
[ 6689.098891] [<fffffe00001f2d44>] __slab_alloc.constprop.63+0x344/0x42c
[ 6689.105385] [<fffffe00001f308c>] __kmalloc+0x260/0x2c8
[ 6689.110498] [<fffffe000037dd4c>] context_struct_to_string.part.6+0xb4/0x1b8
[ 6689.117428] [<fffffe000037debc>] context_struct_to_string+0x6c/0x94
[ 6689.123663] [<fffffe000037e0c8>] security_sid_to_context_core+0xd4/0x168
[ 6689.130331] [<fffffe00003800d8>] security_sid_to_context_force+0x30/0x40
[ 6689.137000] [<fffffe000036c8a4>] selinux_inode_init_security+0x11c/0x280
[ 6689.143669] [<fffffe0000364130>] security_inode_init_security+0x78/0x10c
[ 6689.150344] [<fffffe00002d2f68>] ext4_init_security+0x44/0x50
[ 6689.156068] [<fffffe0000288d80>] __ext4_new_inode+0xf80/0x1584
[ 6689.161879] [<fffffe0000298e6c>] ext4_create+0xc4/0x19c
[ 6689.167085] [<fffffe0000213404>] vfs_create+0x104/0x174
[ 6689.172290] [<fffffe0000213aa0>] do_last+0x62c/0xd80
[ 6689.177236] [<fffffe0000216240>] path_openat+0x8c/0x638
[ 6689.182440] [<fffffe0000217640>] do_filp_open+0x3c/0xa8
[ 6689.187641] [<fffffe000020618c>] do_sys_open+0x148/0x228
[ 6689.192932] [<fffffe00002062ec>] SyS_openat+0x38/0x48
[ 6689.197963] Mem-Info:
[ 6689.200227] DMA per-cpu:
[ 6689.202751] CPU    0: hi:    6, btch:   1 usd:   1
[ 6689.207522] CPU    1: hi:    6, btch:   1 usd:   0
[ 6689.212292] CPU    2: hi:    6, btch:   1 usd:   1
[ 6689.217061] CPU    3: hi:    6, btch:   1 usd:   0
[ 6689.221832] CPU    4: hi:    6, btch:   1 usd:   0
[ 6689.226603] CPU    5: hi:    6, btch:   1 usd:   3
[ 6689.231374] CPU    6: hi:    6, btch:   1 usd:   1
[ 6689.236144] CPU    7: hi:    6, btch:   1 usd:   0
[ 6689.240912] Normal per-cpu:
[ 6689.243693] CPU    0: hi:    6, btch:   1 usd:   3
[ 6689.248464] CPU    1: hi:    6, btch:   1 usd:   2
[ 6689.253234] CPU    2: hi:    6, btch:   1 usd:   2
[ 6689.258001] CPU    3: hi:    6, btch:   1 usd:   2
[ 6689.262771] CPU    4: hi:    6, btch:   1 usd:   0
[ 6689.267542] CPU    5: hi:    6, btch:   1 usd:   0
[ 6689.272312] CPU    6: hi:    6, btch:   1 usd:   2
[ 6689.277082] CPU    7: hi:    6, btch:   1 usd:   1
[ 6689.281854] active_anon:1607 inactive_anon:13595 isolated_anon:0
 active_file:113678 inactive_file:113762 isolated_file:0
 unevictable:0 dirty:10915 writeback:15840 unstable:0
 free:1104 slab_reclaimable:12194 slab_unreclaimable:3649
 mapped:564 shmem:34 pagetables:162 bounce:0
 free_cma:103
[ 6689.314471] DMA free:53504kB min:3968kB low:4928kB high:5952kB active_anon:24832kB inactive_anon:302848kB active_file:1742720kB inactive_file:1741696kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4194304kB managed:4113984kB mlocked:0kB dirty:166080kB writeback:253760kB mapped:9472kB shmem:1408kB slab_reclaimable:153984kB slab_unreclaimable:87808kB kernel_stack:800kB pagetables:2496kB unstable:0kB bounce:0kB free_cma:6592kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 6689.358147] lowmem_reserve[]: 0 765 765
[ 6689.362016] Normal free:18048kB min:12288kB low:15360kB high:18432kB active_anon:78016kB inactive_anon:567232kB active_file:5532096kB inactive_file:5539648kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:12582912kB managed:12537792kB mlocked:0kB dirty:539520kB writeback:760000kB mapped:26624kB shmem:768kB slab_reclaimable:626432kB slab_unreclaimable:145728kB kernel_stack:2592kB pagetables:7872kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 6689.406295] lowmem_reserve[]: 0 0 0
[ 6689.409806] DMA: 760*64kB (UMRC) 15*128kB (MR) 4*256kB (R) 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB 0*65536kB = 52096kB
[ 6689.423060] Normal: 243*64kB (UEM) 2*128kB (M) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB 0*65536kB = 15808kB
[ 6689.435592] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=524288kB
[ 6689.444165] 227425 total pagecache pages
[ 6689.448069] 31 pages in swap cache
[ 6689.451454] Swap cache stats: add 100, delete 69, find 22/26
[ 6689.457086] Free swap  = 4189248kB
[ 6689.460470] Total swap = 4194240kB
[ 6689.463855] 262144 pages RAM
[ 6689.466720] 0 pages HighMem/MovableOnly
[ 6689.470537] 18446744073709545384 pages reserved
[ 6689.475045] 8192 pages cma reserved
[ 6689.478518] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[ 6689.484583]   cache: kmalloc-64, object size: 64, buffer size: 64, default order: 0, min order: 0
[ 6689.493415]   node 0: slabs: 381, objs: 390144, free: 0
[24175.016118] a.out[16464]: unhandled level 3 translation fault (11) at 0x00000000, esr 0x92000047
[24175.024901] pgd = fffffe0058560000
[24175.028288] [00000000] *pgd=000000405e3b0003, *pud=000000405e3b0003, *pmd=000000405e3b0003, *pte=0000000000000000

[24175.040068] CPU: 4 PID: 16464 Comm: a.out Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[24175.048558] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[24175.055847] task: fffffe03cf485c00 ti: fffffe0246ac0000 task.ti: fffffe0246ac0000
[24175.063313] PC is at 0x3ffb7442e00
[24175.066700] LR is at 0x3ffb7442e00
[24175.070095] pc : [<000003ffb7442e00>] lr : [<000003ffb7442e00>] pstate: 60000000
[24175.077459] sp : 000003ffece5b7b0
[24175.080767] x29: 000003ffece5b7b0 x28: 0000000000000000 
[24175.086075] x27: 0000000000000000 x26: 0000000000000000 
[24175.091392] x25: 0000000000000000 x24: 0000000000000000 
[24175.096698] x23: 000003ffb7443000 x22: 0000000000000000 
[24175.102012] x21: 0000000000000000 x20: 0000000000000000 
[24175.107318] x19: 0000000000000000 x18: 0000000000000001 
[24175.112636] x17: 000003ffb7320088 x16: 000003ffb745ff80 
[24175.117941] x15: 000003ffb71b2320 x14: 736d656d62696c2f 
[24175.123257] x13: 7362696c2e2f2e2e x12: 0000000000000028 
[24175.128564] x11: 0101010101010101 x10: 7f7f7f7f7f7f7f7f 
[24175.133881] x9 : fefefefefefefeff x8 : 0000000000000040 
[24175.139187] x7 : fefeff095c363634 x6 : 0000000000808080 
[24175.144505] x5 : 8080800000000000 x4 : 000003ffb741f000 
[24175.149810] x3 : 000000000000f90f x2 : 000003ffb7421270 
[24175.155127] x1 : 09e7e94f4add0d00 x0 : 0000000000000000 

[28770.322995] gsi[24340]: unhandled level 2 translation fault (11) at 0x00000008, esr 0x92000006
[28770.331580] pgd = fffffe03d59d0000
[28770.334984] [00000008] *pgd=0000000000000000, *pud=0000000000000000, *pmd=0000000000000000

[28770.344752] CPU: 4 PID: 24340 Comm: gsi Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[28770.353077] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[28770.360358] task: fffffe00c4047e80 ti: fffffe0246aac000 task.ti: fffffe0246aac000
[28770.367831] PC is at 0x3ff7c9d39a0
[28770.371218] LR is at 0x3ff7c9d3910
[28770.374615] pc : [<000003ff7c9d39a0>] lr : [<000003ff7c9d3910>] pstate: 80000000
[28770.381982] sp : 000003ffda3f4950
[28770.385290] x29: 000003ffda3f4bd0 x28: 000003ff7cbb43b8 
[28770.390599] x27: 0000000000000003 x26: 000003ff885e0064 
[28770.395916] x25: 0000000000000000 x24: 000003ff885e0010 
[28770.401223] x23: 0000000000000000 x22: 000003ffda3f4ab0 
[28770.406538] x21: 000003ff7cd93288 x20: 000003ff7cd93460 
[28770.411846] x19: 0000000000000000 x18: 0000000000000001 
[28770.417164] x17: 000003ff7c50fb58 x16: 000003ff7cbd6780 
[28770.422469] x15: 00183fd16aa56b88 x14: 0000000000000000 
[28770.427785] x13: 0000000000000000 x12: 0000000000000000 
[28770.433097] x11: 0000000000000000 x10: 0000000000000000 
[28770.438405] x9 : 000000000000802e x8 : 0000000000000001 
[28770.443719] x7 : 000003ff885e4650 x6 : 000003ff885e4630 
[28770.449026] x5 : 000003ff7cbb4d11 x4 : 0000000000000020 
[28770.454350] x3 : 0000000000000028 x2 : e8850956ad882c00 
[28770.459657] x1 : 0000000000000002 x0 : 0000000000000000 

[156099.842595] gsi[16171]: unhandled level 2 translation fault (11) at 0x00000008, esr 0x92000006
[156099.851277] pgd = fffffe020c860000
[156099.854750] [00000008] *pgd=0000000000000000, *pud=0000000000000000, *pmd=0000000000000000

[156099.864683] CPU: 3 PID: 16171 Comm: gsi Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[156099.873089] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[156099.880458] task: fffffe02125b6780 ti: fffffe00ad09c000 task.ti: fffffe00ad09c000
[156099.888001] PC is at 0x3ff85ce39a0
[156099.891482] LR is at 0x3ff85ce3910
[156099.894956] pc : [<000003ff85ce39a0>] lr : [<000003ff85ce3910>] pstate: 80000000
[156099.902410] sp : 000003fffa065be0
[156099.905795] x29: 000003fffa065e60 x28: 000003ff85ec43b8 
[156099.911194] x27: 0000000000000003 x26: 000003ffc3030064 
[156099.916586] x25: 0000000000000000 x24: 000003ffc3030010 
[156099.921984] x23: 0000000000000000 x22: 000003fffa065d40 
[156099.927376] x21: 000003ff860a3288 x20: 000003ff860a3460 
[156099.932775] x19: 0000000000000000 x18: 0000000000000001 
[156099.938169] x17: 000003ff8581fb58 x16: 000003ff85ee6780 
[156099.943567] x15: 00188007a7ddd805 x14: 0000000000000000 
[156099.948964] x13: 0000000000000000 x12: 0000000000000000 
[156099.954356] x11: 0000000000000000 x10: 0000000000000000 
[156099.959754] x9 : 000000000000802e x8 : 0000000000000001 
[156099.965146] x7 : 000003ffc3034650 x6 : 000003ffc3034630 
[156099.970545] x5 : 000003ff85ec4d11 x4 : 0000000000000020 
[156099.975938] x3 : 0000000000000028 x2 : 1af66e1daa1c6c00 
[156099.981336] x1 : 0000000000000002 x0 : 0000000000000000 

[186580.855181] embed_3[7916]: unhandled level 3 translation fault (11) at 0x00000060, esr 0x92000007
[186580.864119] pgd = fffffe0204140000
[186580.867591] [00000060] *pgd=0000004204170003, *pud=0000004204170003, *pmd=0000004204170003, *pte=0000000000000000

[186580.879534] CPU: 4 PID: 7916 Comm: embed_3 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186580.888211] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186580.895588] task: fffffe021e56a100 ti: fffffe0021444000 task.ti: fffffe0021444000
[186580.903133] PC is at 0x3ff8bf06904
[186580.906613] LR is at 0x3ff8bf068f8
[186580.910088] pc : [<000003ff8bf06904>] lr : [<000003ff8bf068f8>] pstate: 80000000
[186580.917545] sp : 000003fff53dfae0
[186580.920931] x29: 000003fff53dfae0 x28: 0000000000000000 
[186580.926331] x27: 0000000000000000 x26: 0000000000000000 
[186580.931724] x25: 0000000000000000 x24: 0000000031c31c20 
[186580.937127] x23: 0000000000000000 x22: 00000000004022a0 
[186580.942519] x21: 0000000000000000 x20: 0000000000000003 
[186580.947922] x19: 0000000031d18620 x18: 0000000031cb8970 
[186580.953315] x17: 000003ff8a048040 x16: 000003ff8c0ddb00 
[186580.958716] x15: 000003ff8beb2d7c x14: 0000000000000000 
[186580.964117] x13: ffffffffffffffff x12: 0000000000000018 
[186580.969510] x11: 0000000000000014 x10: 0101010101010101 
[186580.974912] x9 : 6e6e651f6174722d x8 : 7f7f7f7f7f7f7f7f 
[186580.980306] x7 : fefefefeff51484f x6 : 0000000031c33382 
[186580.985706] x5 : 0000000000000c18 x4 : 0000000000008100 
[186580.991099] x3 : 0000000000000052 x2 : 0000000000000000 
[186580.996500] x1 : 0000000031d18620 x0 : 0000000031c31c20 

[186581.530690] embed_4[7925]: unhandled level 3 translation fault (11) at 0x00000060, esr 0x92000007
[186581.539634] pgd = fffffe020d750000
[186581.543106] [00000060] *pgd=000000420cd10003, *pud=000000420cd10003, *pmd=000000420cd10003, *pte=0000000000000000

[186581.555056] CPU: 5 PID: 7925 Comm: embed_4 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186581.563717] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186581.571092] task: fffffe00980f2e00 ti: fffffe00bdf9c000 task.ti: fffffe00bdf9c000
[186581.578644] PC is at 0x3ff90a26904
[186581.582117] LR is at 0x3ff90a268f8
[186581.585598] pc : [<000003ff90a26904>] lr : [<000003ff90a268f8>] pstate: 80000000
[186581.593048] sp : 000003ffd50a6c60
[186581.596441] x29: 000003ffd50a6c60 x28: 0000000000000000 
[186581.601835] x27: 0000000000000000 x26: 0000000000000000 
[186581.607237] x25: 0000000000000000 x24: 0000000004f91c20 
[186581.612631] x23: 0000000000000000 x22: 00000000004022a0 
[186581.618034] x21: 0000000000000000 x20: 0000000000000003 
[186581.623426] x19: 0000000005078620 x18: 0000000005018970 
[186581.628830] x17: 000003ff8eb68040 x16: 000003ff90bfdb00 
[186581.634230] x15: 000003ff909d2d7c x14: 0000000000000000 
[186581.639624] x13: ffffffffffffffff x12: 0000000000000018 
[186581.645025] x11: 0000000000000014 x10: 0101010101010101 
[186581.650417] x9 : 646e631f72686753 x8 : 7f7f7f7f7f7f7f7f 
[186581.655819] x7 : fefefefeff6e6e45 x6 : 0000000004f93382 
[186581.661213] x5 : 0000000000000c18 x4 : 0000000000008100 
[186581.666615] x3 : 000000000000006f x2 : 0000000000000000 
[186581.672007] x1 : 0000000005078620 x0 : 0000000004f91c20 

[186582.022497] embed_5[7934]: unhandled level 3 translation fault (11) at 0x00000060, esr 0x92000007
[186582.031438] pgd = fffffe020a470000
[186582.034926] [00000060] *pgd=000000420a4b0003, *pud=000000420a4b0003, *pmd=000000420a4b0003, *pte=0000000000000000

[186582.046851] CPU: 7 PID: 7934 Comm: embed_5 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186582.055518] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186582.062882] task: fffffe021e565080 ti: fffffe0021434000 task.ti: fffffe0021434000
[186582.070431] PC is at 0x3ff7f696904
[186582.073904] LR is at 0x3ff7f6968f8
[186582.077382] pc : [<000003ff7f696904>] lr : [<000003ff7f6968f8>] pstate: 80000000
[186582.084836] sp : 000003ffddd13970
[186582.088221] x29: 000003ffddd13970 x28: 0000000000000000 
[186582.093614] x27: 0000000000000000 x26: 0000000000000000 
[186582.099013] x25: 0000000000000000 x24: 000000001c871c20 
[186582.104411] x23: 0000000000000000 x22: 00000000004022b0 
[186582.109803] x21: 0000000000000000 x20: 0000000000000003 
[186582.115201] x19: 000000001c958620 x18: 000000001c8f8970 
[186582.120594] x17: 000003ff7d7d8040 x16: 000003ff7f86db00 
[186582.125993] x15: 000003ff7f642d7c x14: 0000000000000000 
[186582.131385] x13: ffffffffffffffff x12: 0000000000000018 
[186582.136784] x11: 0000000000000014 x10: 0101010101010101 
[186582.142177] x9 : 64646b721f646753 x8 : 7f7f7f7f7f7f7f7f 
[186582.147575] x7 : fefefefeff51484f x6 : 000000001c873382 
[186582.152968] x5 : 0000000000000c18 x4 : 0000000000008100 
[186582.158367] x3 : 0000000000000052 x2 : 0000000000000000 
[186582.163759] x1 : 000000001c958620 x0 : 000000001c871c20 

[186582.529525] embed_6[7943]: unhandled level 3 translation fault (11) at 0x00000060, esr 0x92000007
[186582.538463] pgd = fffffe021c110000
[186582.541937] [00000060] *pgd=000000421bdd0003, *pud=000000421bdd0003, *pmd=000000421bdd0003, *pte=0000000000000000

[186582.553876] CPU: 6 PID: 7943 Comm: embed_6 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186582.562549] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186582.569927] task: fffffe0213355080 ti: fffffe019b0f8000 task.ti: fffffe019b0f8000
[186582.577480] PC is at 0x3ff7d826904
[186582.580953] LR is at 0x3ff7d8268f8
[186582.584437] pc : [<000003ff7d826904>] lr : [<000003ff7d8268f8>] pstate: 80000000
[186582.591887] sp : 000003fff536cc90
[186582.595281] x29: 000003fff536cc90 x28: 0000000000000000 
[186582.600676] x27: 0000000000000000 x26: 0000000000000000 
[186582.606078] x25: 0000000000000000 x24: 00000000053e1c20 
[186582.611472] x23: 0000000000000000 x22: 0000000000402310 
[186582.616875] x21: 0000000000000000 x20: 0000000000000003 
[186582.622269] x19: 00000000054c8740 x18: 0000000005468970 
[186582.627673] x17: 000003ff7b968040 x16: 000003ff7d9fdb00 
[186582.633066] x15: 000003ff7d7d2d7c x14: 0000000000000000 
[186582.638469] x13: ffffffffffffffff x12: 0000000000000018 
[186582.643861] x11: 0000000000000014 x10: 0101010101010101 
[186582.649263] x9 : 68606c1f6174722d x8 : 7f7f7f7f7f7f7f7f 
[186582.654663] x7 : fefefefeff51484f x6 : 00000000053e3422 
[186582.660056] x5 : 0000000000000c1d x4 : 0000000000008100 
[186582.665458] x3 : 0000000000000052 x2 : 0000000000000000 
[186582.670852] x1 : 00000000054c8740 x0 : 00000000053e1c20 

[186590.662386] extend_15[8118]: unhandled level 3 translation fault (11) at 0x00000058, esr 0x92000007
[186590.671508] pgd = fffffe021bb30000
[186590.674997] [00000058] *pgd=000000421b7d0003, *pud=000000421b7d0003, *pmd=000000421b7d0003, *pte=0000000000000000

[186590.686929] CPU: 7 PID: 8118 Comm: extend_15 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186590.695772] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186590.703139] task: fffffe003b370b80 ti: fffffe00dbf60000 task.ti: fffffe00dbf60000
[186590.710691] PC is at 0x3ff83cc3a98
[186590.714165] LR is at 0x3ff83ce53f0
[186590.717646] pc : [<000003ff83cc3a98>] lr : [<000003ff83ce53f0>] pstate: 80000000
[186590.725105] sp : 000003ffcc928710
[186590.728490] x29: 000003ffcc928710 x28: 0000000000000000 
[186590.733885] x27: 0000000000000000 x26: 0000000000000000 
[186590.739288] x25: 0000000000000000 x24: 0000000000000000 
[186590.744681] x23: 0000000000000000 x22: 0000000000000000 
[186590.750083] x21: 00000000353074d0 x20: 00000000353074f8 
[186590.755484] x19: 0000000035211c20 x18: 0000000035298970 
[186590.760877] x17: 000003ff83cc3a90 x16: 000003ff83e2e2c0 
[186590.766277] x15: 000003ff83c01bdc x14: 0000000000000000 
[186590.771670] x13: ffffffff00000000 x12: 0000000000000000 
[186590.777071] x11: 0000000000000014 x10: 0101010101010101 
[186590.782465] x9 : 0000000000000004 x8 : 0000000000000040 
[186590.787867] x7 : 0000000000000000 x6 : 000003ffcc92867c 
[186590.793260] x5 : 000000003529eab8 x4 : 0000000000000002 
[186590.798662] x3 : 0000000000000001 x2 : 0000000000000001 
[186590.804055] x1 : 0000000000000000 x0 : 0000000035211c20 

[186591.142195] extend_16[8127]: unhandled level 3 translation fault (11) at 0x00000058, esr 0x92000007
[186591.151311] pgd = fffffe0216d40000
[186591.154785] [00000058] *pgd=00000042169b0003, *pud=00000042169b0003, *pmd=00000042169b0003, *pte=0000000000000000

[186591.166735] CPU: 6 PID: 8127 Comm: extend_16 Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[186591.175577] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[186591.182943] task: fffffe021e56cf00 ti: fffffe0021464000 task.ti: fffffe0021464000
[186591.190497] PC is at 0x3ff82273a98
[186591.193970] LR is at 0x3ff822953f0
[186591.197452] pc : [<000003ff82273a98>] lr : [<000003ff822953f0>] pstate: 80000000
[186591.204902] sp : 000003ffce652510
[186591.208297] x29: 000003ffce652510 x28: 0000000000000000 
[186591.213691] x27: 0000000000000000 x26: 0000000000000000 
[186591.219096] x25: 0000000000000000 x24: 0000000000000000 
[186591.224488] x23: 0000000000000000 x22: 0000000000000000 
[186591.229892] x21: 00000000124774d0 x20: 00000000124774f8 
[186591.235294] x19: 0000000012381c20 x18: 0000000012408970 
[186591.240687] x17: 000003ff82273a90 x16: 000003ff823de2c0 
[186591.246089] x15: 000003ff821b1bdc x14: 0000000000000000 
[186591.251482] x13: ffffffffff000000 x12: 0000000000000000 
[186591.256883] x11: 0000000000000014 x10: 0101010101010101 
[186591.262275] x9 : 0000000000000004 x8 : 0000000000000040 
[186591.267679] x7 : 0000000000000000 x6 : 000003ffce65247c 
[186591.273072] x5 : 000000001240eab8 x4 : 0000000000000002 
[186591.278475] x3 : 0000000000000001 x2 : 0000000000000001 
[186591.283868] x1 : 0000000000000000 x0 : 0000000012381c20 

[226170.516815] urlgrabber-ext-: page allocation failure: order:0, mode:0x20
[226170.523594] CPU: 0 PID: 15916 Comm: urlgrabber-ext- Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[226170.533039] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[226170.540409] Call trace:
[226170.542943] [<fffffe0000097648>] dump_backtrace+0x0/0x164
[226170.548412] [<fffffe00000977c8>] show_stack+0x1c/0x28
[226170.553535] [<fffffe00007547b0>] dump_stack+0x74/0x94
[226170.558648] [<fffffe00001a128c>] warn_alloc_failed+0xe8/0x140
[226170.564461] [<fffffe00001a4d94>] __alloc_pages_nodemask+0x5f4/0x8d4
[226170.570792] [<fffffe00006083dc>] __alloc_page_frag+0xc0/0x110
[226170.576603] [<fffffe0000608464>] __netdev_alloc_frag+0x38/0x4c
[226170.582498] [<fffffe000060f210>] __alloc_rx_skb+0xf0/0x11c
[226170.588049] [<fffffe000060f270>] __netdev_alloc_skb+0x34/0x64
[226170.593861] [<fffffe000050c140>] xgene_enet_refill_bufpool+0x120/0x278
[226170.600450] [<fffffe000050d540>] xgene_enet_process_ring+0x170/0x348
[226170.606865] [<fffffe000050d810>] xgene_enet_napi+0x28/0x60
[226170.612416] [<fffffe000061df6c>] net_rx_action+0x26c/0x334
[226170.617966] [<fffffe00000c3d5c>] __do_softirq+0x128/0x2a8
[226170.623429] [<fffffe00000c4164>] irq_exit+0x98/0xd8
[226170.628374] [<fffffe00001094f8>] __handle_domain_irq+0x6c/0xbc
[226170.634270] [<fffffe0000090408>] gic_handle_irq+0x38/0x88
[226170.639732] Exception stack(0xfffffe0206b6feb0 to 0xfffffe0206b6ffd0)
[226170.646233] fea0:                                     00000000 00000000 cdf26500 000003ff
[226170.654465] fec0: ffffffff ffffffff 97b78864 000003ff 00000001 00000000 cdf26890 000003ff
[226170.662697] fee0: ffffffd0 00000000 ffffffd8 00000000 cdf26520 000003ff cdf26261 000003ff
[226170.670927] ff00: 00000020 00000000 ffffffd0 ffffff80 00000001 00000000 00000000 00000000
[226170.679158] ff20: 99999999 19999999 00000000 00000000 00000005 00000000 ffffffff ffffffff
[226170.687389] ff40: 82000000 00146a13 00000000 003b9aca 908d0320 000003ff 97b84dc4 000003ff
[226170.695621] ff60: 00000072 00000000 00000006 00000000 cdf26500 000003ff 908a3ab0 000003ff
[226170.703853] ff80: 0000000a 00000000 00000040 00000000 00000100 00000000 00000040 00000000
[226170.712084] ffa0: 0000002e 00000000 cdf266d0 000003ff 00000000 00000000 cdf26370 000003ff
[226170.720315] ffc0: 97b76934 000003ff cdf26260 000003ff
[226170.725430] Mem-Info:
[226170.727781] DMA per-cpu:
[226170.730391] CPU    0: hi:    6, btch:   1 usd:   4
[226170.735248] CPU    1: hi:    6, btch:   1 usd:   5
[226170.740105] CPU    2: hi:    6, btch:   1 usd:   0
[226170.744962] CPU    3: hi:    6, btch:   1 usd:   0
[226170.749819] CPU    4: hi:    6, btch:   1 usd:   0
[226170.754676] CPU    5: hi:    6, btch:   1 usd:   5
[226170.759532] CPU    6: hi:    6, btch:   1 usd:   0
[226170.764388] CPU    7: hi:    6, btch:   1 usd:   0
[226170.769245] Normal per-cpu:
[226170.772113] CPU    0: hi:    6, btch:   1 usd:   1
[226170.776970] CPU    1: hi:    6, btch:   1 usd:   0
[226170.781826] CPU    2: hi:    6, btch:   1 usd:   0
[226170.786683] CPU    3: hi:    6, btch:   1 usd:   0
[226170.791539] CPU    4: hi:    6, btch:   1 usd:   0
[226170.796396] CPU    5: hi:    6, btch:   1 usd:   5
[226170.801252] CPU    6: hi:    6, btch:   1 usd:   0
[226170.806109] CPU    7: hi:    6, btch:   1 usd:   0
[226170.810969] active_anon:6027 inactive_anon:6171 isolated_anon:0
 active_file:115510 inactive_file:115431 isolated_file:0
 unevictable:0 dirty:3115 writeback:752 unstable:0
 free:900 slab_reclaimable:12702 slab_unreclaimable:2545
 mapped:957 shmem:31 pagetables:232 bounce:0
 free_cma:0
[226170.843505] DMA free:44864kB min:3968kB low:4928kB high:5952kB active_anon:126976kB inactive_anon:127360kB active_file:1811392kB inactive_file:1811328kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4194304kB managed:4113984kB mlocked:0kB dirty:53888kB writeback:4672kB mapped:12864kB shmem:1024kB slab_reclaimable:137728kB slab_unreclaimable:43648kB kernel_stack:912kB pagetables:3648kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[226170.886922] lowmem_reserve[]: 0 765 765
[226170.890881] Normal free:12736kB min:12288kB low:15360kB high:18432kB active_anon:258752kB inactive_anon:267584kB active_file:5581248kB inactive_file:5576256kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:12582912kB managed:12537792kB mlocked:0kB dirty:145472kB writeback:43456kB mapped:48384kB shmem:960kB slab_reclaimable:675200kB slab_unreclaimable:119232kB kernel_stack:2704kB pagetables:11200kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[226170.935334] lowmem_reserve[]: 0 0 0
[226170.938940] DMA: 166*64kB (MR) 0*128kB 3*256kB (R) 4*512kB (R) 1*1024kB (R) 0*2048kB 4*4096kB (R) 1*8192kB (R) 1*16384kB (R) 0*32768kB 0*65536kB = 55424kB
[226170.953012] Normal: 29*64kB (UEM) 116*128kB (UM) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB 0*32768kB 0*65536kB = 16704kB
[226170.965818] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=524288kB
[226170.974480] 230964 total pagecache pages
[226170.978471] 92 pages in swap cache
[226170.981944] Swap cache stats: add 782, delete 690, find 1182/1239
[226170.988094] Free swap  = 4151360kB
[226170.991566] Total swap = 4194240kB
[226170.995037] 262144 pages RAM
[226170.997989] 0 pages HighMem/MovableOnly
[226171.001893] 18446744073709545384 pages reserved
[226171.006488] 8192 pages cma reserved
[233173.075665] lt-sanity-test[26704]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000046
[233173.085362] pgd = fffffe021a490000
[233173.088879] [00000000] *pgd=0000000000000000, *pud=0000000000000000, *pmd=0000000000000000

[233173.098836] CPU: 6 PID: 26704 Comm: lt-sanity-test Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[233173.108208] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[233173.115581] task: fffffe021dd27300 ti: fffffe037687c000 task.ti: fffffe037687c000
[233173.123149] PC is at 0x3ffb058320c
[233173.126630] LR is at 0x3ffb0583dc4
[233173.130129] pc : [<000003ffb058320c>] lr : [<000003ffb0583dc4>] pstate: 00000000
[233173.137597] sp : 000003fff4fa3360
[233173.140990] x29: 000003fff4fa3360 x28: 000003ffb0585cf0 
[233173.146389] x27: 000003ffb0585ce0 x26: 0000000000000013 
[233173.151808] x25: 000003fff4fa3498 x24: 0000000000000019 
[233173.157207] x23: 0000000000000001 x22: 0000000000000003 
[233173.162625] x21: 000003ffb05a0278 x20: 0000000000000004 
[233173.168037] x19: 000003ffb05a0000 x18: 0000000000000001 
[233173.173437] x17: 000003ffb0583af0 x16: 000003ffb02e0038 
[233173.178852] x15: 0b0d1a16000a140a x14: 0000000000000000 
[233173.184251] x13: 0000000000000000 x12: 0000000000000000 
[233173.189662] x11: 0000000000000000 x10: 0000000000000000 
[233173.195061] x9 : 0000000000008040 x8 : 0000000000000039 
[233173.200474] x7 : 0000000000000000 x6 : 000003fff4fa32c8 
[233173.205872] x5 : 0000000000000042 x4 : 000003ffb02df000 
[233173.211287] x3 : 000000000000ff1f x2 : 000003ffb02e1240 
[233173.216686] x1 : 0000000000000000 x0 : 000003ffb05855e8 

[249697.057974] capability: warning: `memcheck-arm64-' uses 32-bit capabilities (legacy support in use)
[249703.963921] Bluetooth: Core ver 2.20
[249703.967682] NET: Registered protocol family 31
[249703.972215] Bluetooth: HCI device and connection manager initialized
[249703.978633] Bluetooth: HCI socket layer initialized
[249703.983584] Bluetooth: L2CAP socket layer initialized
[249703.988710] Bluetooth: SCO socket layer initialized
[249704.003124] Bluetooth: RFCOMM TTY layer initialized
[249704.008075] Bluetooth: RFCOMM socket layer initialized
[249704.013326] Bluetooth: RFCOMM ver 1.11
[251755.575597] cmsysTestProces[4630]: unhandled level 3 translation fault (11) at 0x00000004, esr 0x92000047
[251755.579324] cmsysTestProces[4633]: unhandled level 3 translation fault (11) at 0x00000004, esr 0x92000047
[251755.579326] pgd = fffffe00d2c80000
[251755.579331] [00000004] *pgd=00000040c26c0003, *pud=00000040c26c0003, *pmd=00000040c26c0003, *pte=0000000000000000

[251755.579337] CPU: 0 PID: 4633 Comm: cmsysTestProces Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[251755.579338] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[251755.579339] task: fffffe020b002280 ti: fffffe0205a84000 task.ti: fffffe0205a84000
[251755.579349] PC is at 0x4023c0
[251755.579350] LR is at 0x4023ac
[251755.579351] pc : [<00000000004023c0>] lr : [<00000000004023ac>] pstate: 60000000
[251755.579352] sp : 000003ffd93cd4e0
[251755.579354] x29: 000003ffd93cd4f0 x28: 0000000000000000 
[251755.579356] x27: 0000000000000000 x26: 0000000000000000 
[251755.579357] x25: 0000000000000000 x24: 0000000000000000 
[251755.579358] x23: 0000000000000000 x22: 0000000000000000 
[251755.579360] x21: 000003ff941c1590 x20: 000003ff941c1588 
[251755.579361] x19: 0000000000418000 x18: 0000000000000001 
[251755.579363] x17: 000003ff94098d98 x16: 0000000000418658 
[251755.579365] x15: 000003ff94031bbc x14: 0000000000000040 
[251755.579366] x13: 000003ff940407f8 x12: 0000000000000000 
[251755.579368] x11: 00000000000003f3 x10: 0101010101010101 
[251755.579369] x9 : 000003ff941c0a20 x8 : 0000000000000040 
[251755.579371] x7 : 73617263206d6f72 x6 : 00000000092b003f 
[251755.579372] x5 : 0000000000000000 x4 : 0000000000000000 
[251755.579374] x3 : 0000000000000004 x2 : 000000000000002e 
[251755.579375] x1 : 0000000000000001 x0 : 0000000000407250 

[251755.733941] pgd = fffffe02025e0000
[251755.737418] [00000004] *pgd=000000421e2e0003, *pud=000000421e2e0003, *pmd=000000421e2e0003, *pte=0000000000000000

[251755.749438] CPU: 4 PID: 4630 Comm: cmsysTestProces Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[251755.758819] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[251755.766199] task: fffffe020b839580 ti: fffffe0218bc0000 task.ti: fffffe0218bc0000
[251755.773774] PC is at 0x4023c0
[251755.776824] LR is at 0x4023ac
[251755.779894] pc : [<00000000004023c0>] lr : [<00000000004023ac>] pstate: 60000000
[251755.787356] sp : 000003ffdcd82230
[251755.790767] x29: 000003ffdcd82240 x28: 0000000000000000 
[251755.796163] x27: 0000000000000000 x26: 0000000000000000 
[251755.801568] x25: 0000000000000000 x24: 0000000000000000 
[251755.806962] x23: 0000000000000000 x22: 0000000000000000 
[251755.812368] x21: 000003ffb0ba1590 x20: 000003ffb0ba1588 
[251755.817790] x19: 0000000000418000 x18: 0000000000000001 
[251755.823190] x17: 000003ffb0a78d98 x16: 0000000000418658 
[251755.828605] x15: 000003ffb0a11bbc x14: 0000000000000040 
[251755.833998] x13: 000003ffb0a207f8 x12: 0000000000000000 
[251755.839409] x11: 00000000000003f3 x10: 0101010101010101 
[251755.844802] x9 : 000003ffb0ba0a20 x8 : 0000000000000040 
[251755.850202] x7 : 73617263206d6f72 x6 : 000000001237003f 
[251755.855595] x5 : 0000000000000000 x4 : 0000000000000000 
[251755.860996] x3 : 0000000000000004 x2 : 000000000000002e 
[251755.866392] x1 : 0000000000000001 x0 : 0000000000407250 

[251894.266755] Crash[12650]: unhandled level 3 translation fault (11) at 0x00000000, esr 0x92000047
[251894.275635] pgd = fffffe021b310000
[251894.279115] [00000000] *pgd=000000421baa0003, *pud=000000421baa0003, *pmd=000000421baa0003, *pte=0000000000000000

[251894.291124] CPU: 7 PID: 12650 Comm: Crash Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[251894.299712] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[251894.307111] task: fffffe00c07bda80 ti: fffffe020270c000 task.ti: fffffe020270c000
[251894.314689] PC is at 0x400910
[251894.317737] LR is at 0x3ff9cfef764
[251894.321233] pc : [<0000000000400910>] lr : [<000003ff9cfef764>] pstate: 60000000
[251894.328693] sp : 000003ffc4152ac0
[251894.332105] x29: 000003ffc4152ad0 x28: 0000000000000000 
[251894.337510] x27: 0000000000000000 x26: 0000000000000000 
[251894.342943] x25: 0000000000000000 x24: 0000000000000000 
[251894.348337] x23: 0000000000000000 x22: 0000000000000000 
[251894.353763] x21: 0000000000000000 x20: 0000000000000000 
[251894.359158] x19: 0000000000400920 x18: 0000000000000001 
[251894.364583] x17: 0000000000410d68 x16: 000003ff9cfef680 
[251894.369998] x15: 000003ff9cfd3a68 x14: 0000000000000041 
[251894.375427] x13: 000003ff9cfe07f8 x12: 0000000000000000 
[251894.380850] x11: 00000000000003f3 x10: 0101010101010101 
[251894.386243] x9 : 0000000000000064 x8 : 0000000000002710 
[251894.391663] x7 : 0000000000004000 x6 : 000003ffc4152c10 
[251894.397064] x5 : 0000000000000000 x4 : 0000000000000000 
[251894.402489] x3 : 0000000000400900 x2 : 000003ffc4152c28 
[251894.407890] x1 : 0000000000000001 x0 : 0000000000000000 

[277503.406004] xgene-enet 1f610000.ethernet eth2: Link is Up - 10Gbps
[277513.416898] xgene-enet 1f610000.ethernet eth2: Link is Down
[364896.789627] gsi[16281]: unhandled level 2 translation fault (11) at 0x00000008, esr 0x92000006
[364896.798296] pgd = fffffe021f810000
[364896.801793] [00000008] *pgd=0000000000000000, *pud=0000000000000000, *pmd=0000000000000000

[364896.811737] CPU: 5 PID: 16281 Comm: gsi Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[364896.820148] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[364896.827514] task: fffffe0039579580 ti: fffffe00dbf7c000 task.ti: fffffe00dbf7c000
[364896.835065] PC is at 0x3ffa3b539a0
[364896.838538] LR is at 0x3ffa3b53910
[364896.842019] pc : [<000003ffa3b539a0>] lr : [<000003ffa3b53910>] pstate: 80000000
[364896.849480] sp : 000003fff6f97350
[364896.852865] x29: 000003fff6f975d0 x28: 000003ffa3d343b8 
[364896.858258] x27: 0000000000000003 x26: 000003ffbd6d0064 
[364896.863661] x25: 0000000000000000 x24: 000003ffbd6d0010 
[364896.869064] x23: 0000000000000000 x22: 000003fff6f974b0 
[364896.874456] x21: 000003ffa3f13288 x20: 000003ffa3f13460 
[364896.879861] x19: 0000000000000000 x18: 0000000000000001 
[364896.885254] x17: 000003ffa368fb58 x16: 000003ffa3d56780 
[364896.890659] x15: 001c787e09691e07 x14: 0000000000000000 
[364896.896052] x13: 0000000000000000 x12: 0000000000000000 
[364896.901454] x11: 0000000000000000 x10: 0000000000000000 
[364896.906847] x9 : 000000000000802e x8 : 0000000000000001 
[364896.912252] x7 : 000003ffbd6d4650 x6 : 000003ffbd6d4630 
[364896.917644] x5 : 000003ffa3d34d11 x4 : 0000000000000020 
[364896.923048] x3 : 0000000000000028 x2 : ee3b1fed2877ec00 
[364896.928441] x1 : 0000000000000002 x0 : 0000000000000000 

[377849.420031] xgene-enet 1f610000.ethernet eth2: Link is Up - 10Gbps
[377859.450931] xgene-enet 1f610000.ethernet eth2: Link is Down
[432991.678118] gsi[2611]: unhandled level 3 translation fault (11) at 0x00000008, esr 0x92000007
[432991.686718] pgd = fffffe0212630000
[432991.690191] [00000008] *pgd=000000420b9f0003, *pud=000000420b9f0003, *pmd=000000420b9f0003, *pte=0000000000000000

[432991.702133] CPU: 5 PID: 2611 Comm: gsi Tainted: G        W       4.0.8-200.fc21.aarch64 #1
[432991.710464] Hardware name: AppliedMicro Mustang/Mustang, BIOS 1.1.0 Apr 22 2015
[432991.717842] task: fffffe003957da80 ti: fffffe004bd80000 task.ti: fffffe004bd80000
[432991.725385] PC is at 0x4bb8a8
[432991.728436] LR is at 0x4bb818
[432991.731477] pc : [<00000000004bb8a8>] lr : [<00000000004bb818>] pstate: 80000000
[432991.738936] sp : 000003ffffab8220
[432991.742324] x29: 000003ffffab84a0 x28: 000000000069c238 
[432991.747731] x27: 0000000000000003 x26: 000000000e020064 
[432991.753124] x25: 0000000000000000 x24: 000000000e020010 
[432991.758526] x23: 0000000000000000 x22: 000003ffffab8380 
[432991.763920] x21: 0000000000883690 x20: 0000000000883868 
[432991.769322] x19: 0000000000000000 x18: 000003ffb1563b8c 
[432991.774714] x17: 000003ffb15dd328 x16: 00000000006d0268 
[432991.780116] x15: 000c6c9957838c93 x14: 0000000000000000 
[432991.785510] x13: 0000000000000000 x12: 0000000000000000 
[432991.790913] x11: 0000000000000006 x10: 0101010101010101 
[432991.796308] x9 : 0000000000000000 x8 : 0000000000000001 
[432991.801711] x7 : 000000000e024650 x6 : 000000000e024630 
[432991.807113] x5 : 000000000069cb91 x4 : 0000000000000020 
[432991.812506] x3 : 0000000000000028 x2 : d0ba05bc8ce31100 
[432991.817907] x1 : 0000000000000002 x0 : 0000000000000000 

[435369.081386] xgene-enet 1f610000.ethernet eth2: Link is Up - 10Gbps
[435379.092285] xgene-enet 1f610000.ethernet eth2: Link is Down
[476944.860975] xgene-enet 1f610000.ethernet eth2: Link is Up - 10Gbps
[476954.891872] xgene-enet 1f610000.ethernet eth2: Link is Down
Comment 1 Mark Salter 2015-08-31 11:41:16 EDT
The thing that jumps out at me is the 18446744073709545384 pages reserved. This is calculated as total_reserved - total_cma and should never fall below zero (18446744073709545384 is -6232) because cma should be included in total_reserved.
So something bad happened to the page accounting, but I'm not sure what.
Comment 2 Peter Robinson 2015-08-31 12:21:40 EDT
(In reply to Mark Salter from comment #1)
> The thing that jumps out at me is the 18446744073709545384 pages reserved.
> This is calculated as total_reserved - total_cma and should never fall below
> zero (18446744073709545384 is -6232) because cma should be included in
> total_reserved.
> So something bad happened to the page accounting, but I'm not sure what.

Sounds lovely :-/ is there a debug option(s) we could enable to try and get more information? We can recreate it in about 5 days
Comment 3 Laura Abbott 2015-08-31 12:23:26 EDT
I think the CMA issue is known, see https://patchwork.kernel.org/patch/6825901/ which is now in the mmotm tree http://git.cmpxchg.org/cgit.cgi/linux-mmotm.git/commit/lib/show_mem.c?id=3a83eda52f34b97168b70098ef0e34dbcaeaaf8f 

The unhandled faults are all userspace crashes. Something not getting the memory it wants and then crashing?
Comment 4 Mark Salter 2015-08-31 14:34:37 EDT
I still think pages reserved is wrong. There should be more than cma pages. Agree about unhandled faults. I wasn't concerned about those. The page allocation failures are the issue with network going down (last one leaves driver without a ring buffer for rx packets). The question is why are we running out of memory?
Comment 5 Fedora End Of Life 2015-11-04 05:40:29 EST
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 6 Peter Robinson 2015-11-04 05:53:17 EST
Bumping to F-23, the patch is applied, we're still seeing some issues, we'll know more when we move the builders to F-23 (and hpefully 4.3) around the same time.
Comment 7 Marcin Juszkiewicz 2016-02-09 13:40:57 EST
updates?
Comment 8 Ronald Maas 2016-03-15 23:22:27 EDT
Try updating /etc/sysctl.conf with the following line and reboot:

vm.min_free_kbytes=131072

Default value was 15256 for Centos 7.2 arm64, which is apparently too low for X-Gene hardware. After this change the page allocation failure did not occur again.

Hope this helps,

Ronald
Comment 9 Laura Abbott 2016-09-23 15:55:39 EDT
*********** MASS BUG UPDATE **************
 
We apologize for the inconvenience.  There is a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 23 kernel bugs.
 
Fedora 23 has now been rebased to 4.7.4-100.fc23.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.
 
If you have moved on to Fedora 24 or 25, and are still experiencing this issue, please change the version to Fedora 24 or 25.
 
If you experience different issues, please open a new bug report for those.
Comment 10 Laura Abbott 2016-10-26 13:00:51 EDT
*********** MASS BUG UPDATE **************
This bug is being closed with INSUFFICIENT_DATA as there has not been a response in 4 weeks. If you are still experiencing this issue, please reopen and attach the relevant data from the latest kernel you are running and any data that might have been requested previously.
Comment 11 Peter Robinson 2017-07-03 08:12:37 EDT
clear needinfo

Note You need to log in before you can comment on or make changes to this bug.