Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1291328

Summary: aio_stress kernel BUG at mm/filemap.c:238
Product: Red Hat Enterprise Linux 7 Reporter: Bill Peck <bpeck>
Component: kernel-aarch64Assignee: fs-maint
kernel-aarch64 sub component: XFS QA Contact: Eryu Guan <eguan>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: unspecified    
Priority: unspecified CC: eguan, jburke, jfeeney, jstancek
Version: 7.3Keywords: TestOnly
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-13 03:45:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Bill Peck 2015-12-14 15:04:21 UTC
Description of problem:

Test: /kernel/Biscayne/ltp-lite

[ 2996.579489] ltptest ADSP015 start 
[ 2996.906474] ------------[ cut here ]------------ 
[ 2996.911089] kernel BUG at mm/filemap.c:238! 
[ 2996.915265] Internal error: Oops - BUG: 0 [#1] SMP 
[ 2996.920047] Modules linked in: tun ext4 mbcache jbd2 loop nls_koi8_u nls_cp932 ts_kmp nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack vfat fat ipmi_ssif i2c_core sg aes_ce_blk ablk_helper ipmi_si cryptd ipmi_msghandler ccp aes_ce_cipher ghash_ce sha2_ce sha1_ce hed nfsd xfs libcrc32c amd_xgbe ptp ahci_platform pps_core libahci_platform dm_mirror dm_region_hash dm_log dm_mod [last unloaded: zlib] 
[ 2996.954972] CPU: 6 PID: 19391 Comm: aiodio_sparse Not tainted 4.4.0-0.rc4.21.el7.aarch64 #1 
[ 2996.963315] Hardware name: Default string Default string/Default string, BIOS ROD0083B 07/17/2015 
[ 2996.972181] task: fffffe02b16eaf80 ti: fffffe02b1b24000 task.ti: fffffe02b1b24000 
[ 2996.979661] PC is at delete_from_page_cache+0x9c/0xa0 
[ 2996.984706] LR is at truncate_inode_page+0x8c/0xb8 
[ 2996.989488] pc : [<fffffe00001be584>] lr : [<fffffe00001cc958>] pstate: 00000145 
[ 2996.996875] sp : fffffe02b1b278e0 
[ 2997.000179] x29: fffffe02b1b278e0 x28: 0000000000000000  
[ 2997.005486] x27: 0000000000000000 x26: 0000000000000000  
[ 2997.010792] x25: 0000000000000000 x24: fffffe0019c17a38  
[ 2997.016096] x23: fffffe02b1b279b0 x22: ffffffffffffffff  
[ 2997.021401] x21: fffffe02b1b27a20 x20: fffffe0019c17a38  
[ 2997.026706] x19: fffffdff00239240 x18: 000003ffd15ac100  
[ 2997.032010] x17: 000003ffad7c0038 x16: fffffe00000c8d00  
[ 2997.037314] x15: 0000000000000000 x14: 0000000003011bd0  
[ 2997.042619] x13: 0000000003011bd1 x12: fffffe03fe188b58  
[ 2997.047924] x11: fffffe0000793460 x10: 00000000000009c0  
[ 2997.053229] x9 : fffffe02b1b27720 x8 : fffffe02b16eb9a0  
[ 2997.058534] x7 : fffffe03fe185020 x6 : 00000000002f8f86  
[ 2997.063838] x5 : 0000000000000000 x4 : 0000000000000010  
[ 2997.069143] x3 : 0000000000010000 x2 : 0000000000000008  
[ 2997.074447] x1 : fffffe0019c17a38 x0 : 0000000000000008  
[ 2997.079753]  
[ 2997.081235] Process aiodio_sparse (pid: 19391, stack limit = 0xfffffe02b1b24020) 
[ 2997.088631] Stack: (0xfffffe02b1b278e0 to 0xfffffe02b1b28000) 
[ 2997.094371] 78e0: fffffe02b1b27920 fffffe00001cc958 fffffdff00239240 fffffe0019c17a38 
[ 2997.102195] 7900: fffffe02b1b27a20 ffffffffffffffff fffffe02b1b279b0 fffffe0019c17a38 
[ 2997.110018] 7920: fffffe02b1b27940 fffffe00001ccce8 fffffdff00239240 0000000000000000 
[ 2997.117840] 7940: fffffe02b1b27aa0 fffffe00001cd078 fffffe0019c17a38 fffffe0019c17700 
[ 2997.125663] 7960: fffffe0019c178d0 fffffe0019c17958 fffffe0000e7f000 fffffe037350df80 
[ 2997.133485] 7980: fffffe035a338210 fffffe0000e7e078 fffffe02b16eaf80 fffffe02b1b24000 
[ 2997.141308] 79a0: fffffe02b1b279f0 0000000000000000 0000000000000000 0000000000000001 
[ 2997.149129] 79c0: 0000000000000002 0000000000000003 0000000000000001 0000000000000001 
[ 2997.156951] 79e0: fffffdff002d5e80 0000000000000000 fffffe02b1b27a90 fffffe00001c4294 
[ 2997.164774] 7a00: 0000000000000001 fffffe0000d38b08 00000003fd450000 fffffe03fe188b08 
[ 2997.172596] 7a20: 0000000000000004 0000000000000000 fffffdff00239240 fffffdff0036afc0 
[ 2997.180418] 7a40: fffffdff000c3900 fffffdff0036af40 fffffe037301d600 0000000000000007 
[ 2997.188240] 7a60: fffffe02b1b27ad0 fffffe0000255f68 fffffe0019c178d0 fffffe0019c179f0 
[ 2997.196062] 7a80: fffffe02b1b27aa0 fffffe00001cd054 fffffe0019c17a38 0000000000000007 
[ 2997.203884] 7aa0: fffffe02b1b27ac0 fffffdfffc5d3e40 fffffe0019c178d0 fffffe0000103800 
[ 2997.211709] 7ac0: fffffe02b1b27af0 fffffe0000240e70 fffffe0019c178d0 fffffe0019c179f0 
[ 2997.219533] 7ae0: fffffdfffc5f5b50 fffffe0000240e3c fffffe02b1b27b20 fffffe0000241964 
[ 2997.227355] 7b00: fffffe035b85f000 fffffe0019c178d0 fffffe0019c17a18 fffffe0019c17958 
[ 2997.235177] 7b20: fffffe02b1b27b60 fffffe000023bf64 fffffe037350df80 fffffe0019c178d0 
[ 2997.242999] 7b40: fffffe037350dfd8 fffffe0019c178d0 fffffe0019c178d0 0000000000000000 
[ 2997.250821] 7b60: fffffe02b1b27b90 fffffe000023c1a0 fffffe037350df80 fffffe001be512c0 
[ 2997.258643] 7b80: 0000000000000001 fffffe00002286b4 fffffe02b1b27bc0 fffffe000022884c 
[ 2997.266465] 7ba0: fffffe035a338200 0000000000000010 fffffe0019c178d0 fffffe035b7f07a0 
[ 2997.274287] 7bc0: fffffe02b1b27c20 fffffe0000228960 fffffe035a338200 0000000000000000 
[ 2997.282109] 7be0: fffffe02b16eaf80 fffffe00013718b0 0000000000000001 fffffe001ef56420 
[ 2997.289932] 7c00: fffffe03590d39c0 fffffe00000e3c70 fffffe02b16eaf80 0000000000000000 
[ 2997.297754] 7c20: fffffe02b1b27c40 fffffe00000e0c8c fffffe02b16eb710 fffffe00000c88ec 
[ 2997.305577] 7c40: fffffe02b1b27c70 fffffe00000c855c 0000000000000000 000000000000000f 
[ 2997.313399] 7c60: fffffe03590d3ac8 fffffe001ef563c0 fffffe02b1b27ce0 fffffe00000c8c90 
[ 2997.321221] 7c80: fffffe001d92dd00 000000000000000f fffffe03590d3ac8 fffffe03590d3ac0 
[ 2997.329043] 7ca0: 0000000000418004 0000000000000008 fffffe03590d39c0 fffffe0000e7e078 
[ 2997.336866] 7cc0: fffffe03590d41c8 fffffe02b1b24000 0000000000000000 fffffe02b1b27e28 
[ 2997.344688] 7ce0: fffffe02b1b27d10 fffffe00000d3ab8 0000000000000009 fffffe02b1b27e28 
[ 2997.352510] 7d00: fffffe03590d3ac8 fffffe00000d3960 fffffe02b1b27da0 fffffe00000963f4 
[ 2997.360332] 7d20: fffffe02b1b27ed0 fffffffffffffffc fffffe02b1b27e08 000003ffb391f074 
[ 2997.368154] 7d40: 000003ffb391f078 00000000fffffffc 000000000000011d 000000000000003f 
[ 2997.375977] 7d60: fffffe0000782000 fffffe02b1b24000 000000000000011d fffffe0000d80180 
[ 2997.383799] 7d80: fffffe0000d8134c fffffe00009ac160 fffffe001d92dd00 fffffe02b1b27e08 
[ 2997.391620] 7da0: fffffe02b1b27eb0 fffffe0000096978 0000000000000209 fffffe02b1b27ed0 
[ 2997.399442] 7dc0: ffffffffffffffff 000003ffb391f078 0000000080000000 0000000000000015 
[ 2997.407265] 7de0: 000000000000011d 0000000000000000 0000000000000000 0000000000000000 
[ 2997.415087] 7e00: fffffe02b1b27e30 fffffe000023038c fffffe02b1b27e50 fffffe0000158778 
[ 2997.422910] 7e20: fffffe02b1b27e50 0000000000000009 fffffe0200000000 0000000000000000 
[ 2997.430731] 7e40: fffffe02bb05aa48 0000000000000015 fffffe02b1b27ea0 fffffe0000096040 
[ 2997.438556] 7e60: 0000000000000200 fffffe02b1b27ed0 ffffffffffffffff 000003ffb391f078 
[ 2997.446378] 7e80: 0000000080000000 fffffe0000091a0c 0000000000000200 fffffffffffffffc 
[ 2997.454201] 7ea0: 000003ffeee16b20 fffffe0000091a18 000003ffeee16b20 fffffe00000918a4 
[ 2997.462023] 7ec0: 0000000000000200 0000000000000000 fffffffffffffffc 000003ffeee17b30 
[ 2997.469845] 7ee0: 0000000000001000 0000000000000000 000003ffb38d2000 0000000000000000 
[ 2997.477668] 7f00: 000003ffb38d2638 000003ffb38d0dd8 000000000000003f 000003ffeee12fd0 
[ 2997.485490] 7f20: 00000000ffffffff 000003ffeee15920 0000000000000012 0000000000000000 
[ 2997.493312] 7f40: 000003ffb3761a94 0000000000005749 0000000000000000 0000000000420290 
[ 2997.501134] 7f60: 000003ffeee168c0 000000000000270f 0000000000000000 0000000000000006 
[ 2997.508957] 7f80: 0000000000000000 000003ffeee17b30 0000000000800000 0000000000000000 
[ 2997.516779] 7fa0: 0000000000407460 0000000000421854 0000000000420000 000003ffeee16b20 
[ 2997.524601] 7fc0: 00000000004029d0 000003ffeee16af0 000003ffb391f078 0000000080000000 
[ 2997.532423] 7fe0: 0000000000000006 ffffffffffffffff 0000000000000000 0000000000000000 
[ 2997.540245] Call Trace: 
[ 2997.542687] [<fffffe00001be584>] delete_from_page_cache+0x9c/0xa0 
[ 2997.548776] [<fffffe00001cc958>] truncate_inode_page+0x8c/0xb8 
[ 2997.554600] [<fffffe00001ccce8>] truncate_inode_pages_range+0x300/0x5f4 
[ 2997.561207] [<fffffe00001cd078>] truncate_inode_pages_final+0x64/0x70 
[ 2997.567714] [<fffffdfffc5d3e40>] xfs_fs_evict_inode+0x38/0x118 [xfs] 
[ 2997.574061] [<fffffe0000240e70>] evict+0xac/0x17c 
[ 2997.578756] [<fffffe0000241964>] iput+0x154/0x1d8 
[ 2997.583453] [<fffffe000023bf64>] __dentry_kill+0x1b0/0x1fc 
[ 2997.588930] [<fffffe000023c1a0>] dput+0x1f0/0x250 
[ 2997.593626] [<fffffe000022884c>] __fput+0x138/0x1d4 
[ 2997.598494] [<fffffe0000228960>] ____fput+0x20/0x2c 
[ 2997.603364] [<fffffe00000e0c8c>] task_work_run+0xb0/0xcc 
[ 2997.608667] [<fffffe00000c855c>] do_exit+0x2c4/0x960 
[ 2997.613623] [<fffffe00000c8c90>] do_group_exit+0x48/0xb8 
[ 2997.618927] [<fffffe00000d3ab8>] get_signal+0x1b0/0x5d4 
[ 2997.624145] [<fffffe00000963f4>] do_signal+0x68/0x378 
[ 2997.629187] [<fffffe0000096978>] do_notify_resume+0x74/0x98 
[ 2997.634752] [<fffffe00000918a4>] work_pending+0x1c/0x20 
[ 2997.639969] Code: a9425bf5 f9401bf7 a8c47bfd d65f03c0 (d4210000)  

Version-Release number of selected component (if applicable):
4.4.0-0.rc4.21

How reproducible:
everytime

Comment 4 Eryu Guan 2015-12-15 12:57:50 UTC
(In reply to Jan Stancek from comment #3)
> Should be fixed in 4.4-rc5:
> 
> http://marc.info/?l=linux-mm&m=144950957622869&w=2
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/
> ?id=dfd01f026058a59a513f8a365b439a0681b803af

Just for future reference, this patch also fixed BUG_ON I've seen when running xfstests generic/095 against NFSv3 on ppc64 host(though it's different call trace). I also confirmed 4.4-rc5 passed the test.

[ 2015.935021] ------------[ cut here ]------------ 
[ 2015.935034] kernel BUG at mm/page-writeback.c:2654! 
[ 2015.935039] Oops: Exception in kernel mode, sig: 5 [#1] 
[ 2015.935042] SMP NR_CPUS=2048 NUMA pSeries 
[ 2015.935046] Modules linked in: loop(E) dm_mod(E) nfsv3(E) nfs(E) fscache(E) ext4(E) mbcache(E) jbd2(E) sg(E) pseries_rng(E) nfsd(E) auth_rpcgss(E) nfs_acl(E) lockd(E) grace(E) sunrpc(E) ip_tables(E) xfs(E) libcrc32c(E) sd_mod(E) ibmvscsi(E) ibmveth(E) scsi_transport_srp(E) 
[ 2015.935090] CPU: 2 PID: 5679 Comm: fio Tainted: G            E   4.4.0-rc4 #1 
[ 2015.935096] task: c00000059b380000 ti: c00000059a17c000 task.ti: c00000059a17c000 
[ 2015.935103] NIP: c0000000001f3e24 LR: c0000000001f3e10 CTR: c0000000001f3de0 
[ 2015.935109] REGS: c00000059a17f440 TRAP: 0700   Tainted: G            E    (4.4.0-rc4) 
[ 2015.935115] MSR: 8000000100029032 <SF,EE,ME,IR,DR,RI>  CR: 24004480  XER: 00000001 
[ 2015.935129] CFAR: c00000000020a59c SOFTE: 1  
GPR00: d00000000bd6e524 c00000059a17f6c0 c000000001325e00 c000000598eb1af8  
GPR04: 0000000000000000 0000000000000000 c00000059a17f830 0000000000000000  
GPR08: 0000000000000000 0000000000000001 0000000000000000 d00000000bd78bf8  
GPR12: 0000000024004480 c00000000e7e1300 00000100080a1b70 ffffffffffffffff  
GPR16: 0000000000000016 00003fff87fdedd0 000000003b9ac9ff f000000001651e18  
GPR20: c000000598eb1b10 0000000000000007 0000000000000000 0000000000000001  
GPR24: 0000000000000001 c000000598eb1af8 0000000000000006 c00000059a17f950  
GPR28: c00000059a17f9c0 c000000598eb1990 c000000598eb1af8 f000000001651e00  
[ 2015.935215] NIP [c0000000001f3e24] .clear_page_dirty_for_io+0x44/0x2a0 
[ 2015.935221] LR [c0000000001f3e10] .clear_page_dirty_for_io+0x30/0x2a0 
[ 2015.935226] Call Trace: 
[ 2015.935230] [c00000059a17f6c0] [0000000000000006] 0x6 (unreliable) 
[ 2015.935255] [c00000059a17f760] [d00000000bd6e524] .nfs_wb_page+0x84/0x210 [nfs] 
[ 2015.935277] [c00000059a17f850] [d00000000bd59250] .nfs_launder_page+0x70/0xc0 [nfs] 
[ 2015.935285] [c00000059a17f8d0] [c0000000001f98e8] .invalidate_inode_pages2_range+0x2f8/0x490 
[ 2015.935299] [c00000059a17faa0] [d00000000bd65940] .nfs_file_direct_write+0xf0/0x460 [nfs] 
[ 2015.935313] [c00000059a17fb70] [d00000000bd5a6d4] .nfs_file_write+0xf4/0x230 [nfs] 
[ 2015.935319] [c00000059a17fc10] [c000000000283278] .__vfs_write+0xf8/0x170 
[ 2015.935325] [c00000059a17fcf0] [c000000000283cf0] .vfs_write+0xb0/0x1f0 
[ 2015.935330] [c00000059a17fd90] [c000000000284d24] .SyS_pwrite64+0xb4/0xd0 
[ 2015.935337] [c00000059a17fe30] [c000000000009204] system_call+0x38/0xb4 
[ 2015.935341] Instruction dump: 
[ 2015.935344] fbc1fff0 fbe1fff8 7c7f1b78 f8010010 91810008 f821ff61 48016765 60000000  
[ 2015.935354] e93f0000 7d2948f8 7c7e1b78 792907e0 <0b090000> 2fa30000 419e0038 e8630000  
[ 2015.935367] ---[ end trace 23bfe706acc701f1 ]--- 
[ 2015.936730]  
[ 2017.936768] Kernel panic - not syncing: Fatal exception

Comment 5 John Feeney 2016-02-10 21:44:04 UTC
Okay so given comments #3 and #4, it looks like this should be fixed in the 4.4-final RHELSA kernel. I am going to move this on to MODIFIED then so it can take a step towards closure.

Comment 8 Eryu Guan 2016-07-13 03:44:48 UTC
Tested with kernel-4.5.0-0.44.el7.aarch64, all ltp-aiodio tests passed without kernel BUG_ON/hang/panic

https://beaker.engineering.redhat.com/jobs/1399681

Set to VERIFIED.