Description of problem: Reported by Mario Mikocevic. RHCS with one gfs2 shared on three nodes. Oops-ed while cp-ing from ext3 to gfs2 on one node. Version-Release number of selected component (if applicable): kernel-2.6.18-164.10.1.el5 gfs2-utils-0.1.62-1.el5 cman-2.0.115-1.el5_4.9 lvm2-cluster-2.02.46-8.el5 How reproducible: Always Steps to Reproduce: 1. Setup 3 node RHCS with - /dev/mapper/VolGroup01-LogVol02 on /var/www type gfs2 (rw,noatime,hostdata=jid=0:id=109:first=1,quota=on) 2. cp some data from ext3 to gfs2 Actual results: list_del corruption. next->prev should be ffff8106d401f000, but was 0000000000000000 ----------- [cut here ] --------- [please bite here ] --------- Kernel BUG at lib/list_debug.c:70 invalid opcode: 0000 [1] SMP last sysfs file: /devices/pci0000:00/0000:00:04.0/0000:17:00.0/0000:18:0a.0/0000:1f:00.0/host1/rport-1:0-4/tar get1:0:3/1:0:3:3/state CPU 2 Modules linked in: lock_dlm gfs2 dlm configfs 8021q bonding ipv6 xfrm_nalgo crypto_api ip_conntrack_ftp ip_con ntrack_netbios_ns ipt_LOG ipt_REJECT xt_tcpudp xt_state ip_conntrack nfnetlink iptable_filter ip_tables x_tabl es video hwmon backlight sbs i2c_ec i2c_core button battery asus_acpi acpi_memhotplug ac parport_pc lp parport hpilo sg bnx2 ide_cd pcspkr e1000e serio_raw cdrom dm_raid45 dm_message dm_region_hash dm_mem_cache dm_round_ robin dm_multipath scsi_dh dm_snapshot dm_zero dm_mirror dm_log dm_mod usb_storage ata_piix libata cciss shpch p qla2xxx scsi_transport_fc sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd Pid: 0, comm: swapper Not tainted 2.6.18-164.10.1.el5 #1 RIP: 0010:[<ffffffff80151a44>] [<ffffffff80151a44>] list_del+0x48/0x71 RSP: 0018:ffff81082ff6bc40 EFLAGS: 00010082 RAX: 0000000000000058 RBX: ffff8106d401f000 RCX: ffffffff80309c28 RDX: ffffffff80309c28 RSI: 0000000000000000 RDI: ffffffff80309c20 RBP: ffff81082d584a40 R08: ffffffff80309c28 R09: 0000000000000001 R10: 0000000000000000 R11: 0000000000000080 R12: ffff81082cb0e080 R13: ffff8106d401fb00 R14: 0000000000000023 R15: 0000000000000000 FS: 0000000000000000(0000) GS:ffff81082ff20e40(0000) knlGS:0000000000000000 CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b CR2: 00000000193b0128 CR3: 00000008160f2000 CR4: 00000000000006e0 Process swapper (pid: 0, threadinfo ffff81082ff66000, task ffff81082ff21080) Stack: ffff8106d401f000 ffffffff800daa96 ffff81082ff6bcc0 0000003c00000001 ffff81082cd89818 000000000000003c ffff81082cd89800 ffff81082d584a40 0000000000000000 ffff81082cb0e080 ffff81082d083a00 ffffffff800dac58 Call Trace: <IRQ> [<ffffffff800daa96>] free_block+0xb5/0x143 [<ffffffff800dac58>] cache_flusharray+0x74/0xa3 [<ffffffff80007684>] kmem_cache_free+0x1c2/0x1dd [<ffffffff8827034a>] :dm_mod:dec_pending+0x134/0x18e [<ffffffff88270588>] :dm_mod:clone_endio+0xbf/0xce [<ffffffff88270588>] :dm_mod:clone_endio+0xbf/0xce [<ffffffff8002cde2>] __end_that_request_first+0x23c/0x5bf [<ffffffff88079fde>] :scsi_mod:scsi_end_request+0x27/0xcd [<ffffffff8807a1d2>] :scsi_mod:scsi_io_completion+0x14e/0x324 [<ffffffff88078c8b>] :scsi_mod:scsi_delete_timer+0x12/0x59 [<ffffffff880a7802>] :sd_mod:sd_rw_intr+0x252/0x28c [<ffffffff8807a467>] :scsi_mod:scsi_device_unbusy+0x67/0x81 [<ffffffff80037cfc>] blk_done_softirq+0x5f/0x6d [<ffffffff8001235a>] __do_softirq+0x89/0x133 [<ffffffff8005e2fc>] call_softirq+0x1c/0x28 [<ffffffff8006cb20>] do_softirq+0x2c/0x85 [<ffffffff8006c9a8>] do_IRQ+0xec/0xf5 [<ffffffff8005722b>] mwait_idle+0x0/0x4a [<ffffffff8005d615>] ret_from_intr+0x0/0xa <EOI> [<ffffffff80057261>] mwait_idle+0x36/0x4a [<ffffffff8004943c>] cpu_idle+0x95/0xb8 [<ffffffff8007708a>] start_secondary+0x498/0x4a7 Code: 0f 0b 68 89 53 2b 80 c2 46 00 48 8b 13 48 8b 43 08 48 89 42 RIP [<ffffffff80151a44>] list_del+0x48/0x71 RSP <ffff81082ff6bc40> <0>Kernel panic - not syncing: Fatal exception Expected results: cp succeeds Additional info: I cannot reproduce that with quota=off.
Statement: This issue did not affect the version of Linux kernel as shipped with Red Hat Enterprise Linux 3, 4 and Red Hat Enterprise MRG as they did not include support for the GFS2 file system. A future kernel update in Red Hat Enterprise Linux 5 will address this issue.
is there any actionable info on this issue yet?
(In reply to comment #5) > is there any actionable info on this issue yet? This issue will be addressed in Red Hat Enterprise Linux 5 soon. If you are unable to wait for an update to be released, please contact Red Hat Support and request for a hotfix. Thanks.
i'm not really concerned about where this issue stands with respect to rhel. i'm looking for patches that we can apply to the debian kernels. thanks.
The upstream patch is here: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=7e619bc3e6252dc746f64ac3b486e784822e9533
Acknowledgements: Red Hat would like to thank Mario Mikocevic for responsibly reporting this issue.
Deleted Technical Notes Contents. Old Contents: A buffer overflow flaw was found in the Global File System 2 (GFS2) implementation in the Linux kernel. A quota could be written past the end of a memory page, causing memory corruption and leaving the quota stored on disk in an invalid state. A user with write access to a GFS2 filesystem could trigger this flaw to cause a kernel crash (denial of service). Whether or not this occurs depends on the uid/gid of the quota being written and only when quotas are set to on or account.
This issue has been addressed in following products: Red Hat Enterprise Linux 5 Via RHSA-2010:0504 https://rhn.redhat.com/errata/RHSA-2010-0504.html