Bug 1558433

Summary:	vmcore generated due to discard file operation
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Nag Pavan Chilakam <nchilaka>
Component:	disperse	Assignee:	Sunil Kumar Acharya <sheggodu>
Status:	CLOSED ERRATA	QA Contact:	Nag Pavan Chilakam <nchilaka>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	rhgs-3.4	CC:	amukherj, ddouwsma, nchilaka, rhinduja, rhs-bugs, storage-qa-internal, ubansal
Target Milestone:	---	Keywords:	Reproducer
Target Release:	RHGS 3.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	kernel-3.10.0-693.29.1.el7	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1558511 (view as bug list)		Environment:
Last Closed:	2018-09-04 06:44:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1558659
Bug Blocks:	1503137

Description Nag Pavan Chilakam 2018-03-20 08:47:19 UTC

Description of problem:
=========================
raising a seperate bug to track issue discussed in https://bugzilla.redhat.com/show_bug.cgi?id=1499865#c4
Also refer other following comments mentioned in above bug


client:fuse:rhel7.4 + 3.12.2-5

hit vmcore when trying to punch holes using fallocate
[86405.542739] fuse init (API version 7.22)
[86581.662596] ------------[ cut here ]------------
[86581.662637] kernel BUG at mm/truncate.c:293!
[86581.662654] invalid opcode: 0000 [#1] SMP 
[86581.662673] Modules linked in: fuse ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep iosf_mbi snd_seq snd_seq_device ppdev sg snd_pcm snd_timer parport_pc snd parport pcspkr crc32_pclmul ghash_clmulni_intel aesni_intel lrw joydev gf128mul virtio_balloon soundcore glue_helper ablk_helper cryptd i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
[86581.663022]  ata_generic pata_acpi qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix virtio_console libata crct10dif_pclmul crct10dif_common crc32c_intel virtio_pci virtio_ring serio_raw virtio 8139too i2c_core 8139cp floppy mii dm_mirror dm_region_hash dm_log dm_mod
[86581.663160] CPU: 0 PID: 19909 Comm: fallocate Not tainted 3.10.0-693.el7.x86_64 #1
[86581.663187] Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
[86581.663209] task: ffff8800b6e6cf10 ti: ffff8800a3210000 task.ti: ffff8800a3210000
[86581.663236] RIP: 0010:[<ffffffff8119266b>]  [<ffffffff8119266b>] truncate_inode_pages_range+0x2eb/0x730
[86581.663275] RSP: 0018:ffff8800a3213d10  EFLAGS: 00010202
[86581.663295] RAX: 0000000000000802 RBX: 0000000000000801 RCX: ffff8800a3213fd8
[86581.663321] RDX: 0000000000000801 RSI: 0000000000000401 RDI: ffff880138148150
[86581.663347] RBP: ffff8800a3213e50 R08: 000060fec0000530 R09: ffffffffc05240e0
[86581.663373] R10: ffffe8ffffc00530 R11: ffffea0002b8f580 R12: 0000000000000401
[86581.663399] R13: ffff880138148150 R14: ffff8800ae1cb400 R15: ffff88013638c800
[86581.663438] FS:  00007f1b194bb740(0000) GS:ffff88013fc00000(0000) knlGS:0000000000000000
[86581.663468] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[86581.663490] CR2: 00007f1b19054820 CR3: 00000000a31f1000 CR4: 00000000000406f0
[86581.663519] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[86581.663545] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[86581.663571] Stack:
[86581.663581]  0000080200000401 ffff880135b10000 0000000000000000 ffff880138148150
[86581.663620]  ffffffff816a8f8d ffff8800b6e6cf10 ffff8800a3213fd8 0000000000000000
[86581.663650]  ffff8800ae3d6ef8 ffff8800a3213d68 0000000000000293 ffff8800a3213d98
[86581.663690] Call Trace:
[86581.663704]  [<ffffffff816a8f8d>] ? __schedule+0x39d/0x8b0
[86581.663728]  [<ffffffff810b16d6>] ? finish_wait+0x56/0x70
[86581.663751]  [<ffffffffc052226c>] ? wait_answer_interruptible+0xbc/0xe0 [fuse]
[86581.663779]  [<ffffffff810b1910>] ? wake_up_atomic_t+0x30/0x30
[86581.663802]  [<ffffffff816a7722>] ? mutex_lock+0x12/0x2f
[86581.663824]  [<ffffffff811afa30>] ? unmap_mapping_range+0x90/0x190
[86581.663848]  [<ffffffff81192c27>] truncate_pagecache_range+0x57/0x60
[86581.663874]  [<ffffffffc052cee0>] fuse_file_fallocate+0x230/0x290 [fuse]
[86581.663902]  [<ffffffff811fe077>] vfs_fallocate+0x117/0x1b0
[86581.663924]  [<ffffffff811ff05b>] SyS_fallocate+0x5b/0xa0
[86581.664207]  [<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
[86581.664477] Code: 49 39 c6 0f 84 68 01 00 00 49 89 c6 eb ba 66 90 4c 89 e7 e8 98 f0 ff ff e9 4c ff ff ff 8b 85 c4 fe ff ff 85 c0 0f 84 d7 fd ff ff <0f> 0b 0f 1f 00 4c 3b b5 c8 fe ff ff 75 0d 48 39 9d e0 fe ff ff 
[86581.665116] RIP  [<ffffffff8119266b>] truncate_inode_pages_range+0x2eb/0x730
[86581.665413]  RSP <ffff8800a3213d10>




steps:
1)created a 2x(4+2) ec volume
2)mounted the volume on c1->which had 3.3.1-async build ie 3.8.4-54
3)created a zero fill file ie using dd /dev/zero #dd if=/dev/zero of=newfile bs=1024 count=10000

4) mounted volume on c2->with rhel7.4 + 3.12.2-5
used fallocate to punch holes as below in b/w the file
fallocate -l -o 1025 -l 1024 newfile --->passed

fuse mount log shows

[2018-03-13 09:33:09.266420] W [fuse-bridge.c:1381:fuse_err_cbk] 0-glusterfs-fuse: 263: DISCARD() ERR => -1 (Success)


5)again reissued above command 
fuse mount log showed above 

however, bash got stuck
after some time the ssh connection was lost
and when i again logged in i saw a vmcore





sosreport and vmcore at http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/nchilaka/bug.1499865/

Comment 12 Nag Pavan Chilakam 2018-05-09 08:14:13 UTC

have tried this 4 times on latest kernel ie  3.10.0-862.el7.x86_64 and didnt hit the crash again, hence moving to verified

glusterfs-server-3.12.2-8.el7rhgs.x86_64

Comment 14 errata-xmlrpc 2018-09-04 06:44:14 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607