Bug 811042

Summary: [abrt] kernel: BUG: unable to handle kernel paging request at ffffffff79178ff0
Product: [Fedora] Fedora Reporter: mbkraetz
Component: kernelAssignee: Eric Sandeen <esandeen>
Status: CLOSED CANTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 16CC: gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Unspecified   
Whiteboard: abrt_hash:a59608bd37e949c436ee9fe69c8d966a4aac019a
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-07-16 22:10:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description mbkraetz 2012-04-10 00:01:54 UTC 
libreport version: 2.0.8
abrt_version:   2.0.7
cmdline:        BOOT_IMAGE=/vmlinuz-3.3.1-3.fc16.x86_64 root=/dev/mapper/vg_server-lv_root ro rd.md=0 rd.dm=0 KEYTABLE=us quiet SYSFONT=latarcyrheb-sun16 rd.lvm.lv=vg_server/lv_root rhgb rd.luks=0 LANG=en_US.UTF-8 rd.lvm.lv=vg_server/lv_swap
comment:        I was moving a large quantity of files from one physical drive to another 130GB. 
event_log:      2012-04-10-00:00:51> Smolt profile successfully saved
kernel:         3.3.1-3.fc16.x86_64
reason:         BUG: unable to handle kernel paging request at ffffffff79178ff0
time:           Mon 09 Apr 2012 07:56:03 PM EDT

backtrace:
:BUG: unable to handle kernel paging request at ffffffff79178ff0
:IP: [<ffffffff79178ff0>] 0xffffffff79178fef
:PGD 1c07067 PUD 0 
:Oops: 0010 [#1] SMP 
:CPU 0 
:Modules linked in: xfs btrfs zlib_deflate libcrc32c nls_utf8 udf vfat fat ppdev parport_pc lp parport fuse be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 fcoe libfcoe cxgb3i libcxgbi cxgb3 mdio ib_iser 8021q garp stp llc libfc scsi_transport_fc rdma_cm scsi_tgt ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip6t_REJECT ip6t_ipv6header nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables nf_conntrack_ftp nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd sp5100_tco soundcore snd_page_alloc shpchp r8169 serio_raw joydev uinput i2c_piix4 edac_core edac_mce_amd microcode k10temp mii firewire_ohci firewire_core ata_generic pata_acpi crc_itu_t pata_atiixp wmi usb_storage radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: scs
:i_wait_scan]
:Pid: 2203, comm: nautilus Not tainted 3.3.1-3.fc16.x86_64 #1 Gigabyte Technology Co., Ltd. GA-880GM-UD2H/GA-880GM-UD2H
:RIP: 0010:[<ffffffff79178ff0>]  [<ffffffff79178ff0>] 0xffffffff79178fef
:RSP: 0018:ffff8803e887bcb0  EFLAGS: 00010246
:RAX: 00000000ffffffff RBX: ffffea000cf9ba80 RCX: 00000000ffffffe0
:RDX: 0000000000000200 RSI: 0000000000000000 RDI: ffffea000cf9ba80
:RBP: ffff8803e887bcb8 R08: 0000000000000000 R09: 0000000000000001
:R10: ffffea000cf9ce5c R11: 000000000000000d R12: 0000000000000000
:R13: ffff8803fd772ea0 R14: 0000000000000000 R15: 0000000000000000
:FS:  00007f302bfff700(0000) GS:ffff88042fc00000(0000) knlGS:0000000000000000
:CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
:CR2: ffffffff79178ff0 CR3: 00000003fc64e000 CR4: 00000000000006f0
:DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
:DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
:Process nautilus (pid: 2203, threadinfo ffff8803e887a000, task ffff8803f8584590)
:Stack:
: ffffffff8117bf2e ffff8803e887bce8 ffffffff81121795 ffffffffffffffff
: ffffea000cf9ba80 ffff8803fd772ea0 ffff8803fd772ea0 ffff8803e887bd08
: ffffffff8112e866 0000000000000000 ffffffffffffffff ffff8803e887bde8
:Call Trace:
: [<ffffffff8117bf2e>] ? mem_cgroup_uncharge_cache_page+0x1e/0x30
: [<ffffffff81121795>] delete_from_page_cache+0x55/0x80
: [<ffffffff8112e866>] truncate_inode_page+0x66/0xa0
: [<ffffffff8112ea0a>] truncate_inode_pages_range+0x16a/0x4c0
: [<ffffffff811be1b8>] ? fsnotify+0x1f8/0x290
: [<ffffffff8112ed75>] truncate_inode_pages+0x15/0x20
: [<ffffffff8120308e>] ext4_evict_inode+0x10e/0x490
: [<ffffffff8119aabf>] evict+0x9f/0x1a0
: [<ffffffff8119acc5>] iput+0x105/0x200
: [<ffffffff8118fff3>] do_unlinkat+0x153/0x1c0
: [<ffffffff810d322c>] ? __audit_syscall_entry+0xcc/0x310
: [<ffffffff810d3846>] ? __audit_syscall_exit+0x3d6/0x410
: [<ffffffff81191456>] sys_unlink+0x16/0x20
: [<ffffffff815fbfe9>] system_call_fastpath+0x16/0x1b
:Code:  Bad RIP value.
:RIP  [<ffffffff79178ff0>] 0xffffffff79178fef
: RSP <ffff8803e887bcb0>
:CR2: ffffffff79178ff0

smolt_data:
:
:
:General
:=================================
:UUID: efc097aa-0637-4197-9b39-76a1b373e290
:OS: Fedora release 16 (Verne)
:Default run level: Unknown
:Language: en_US.UTF-8
:Platform: x86_64
:BogoMIPS: 6428.59
:CPU Vendor: AuthenticAMD
:CPU Model: AMD Phenom(tm) II X2 555 Processor
:CPU Stepping: 3
:CPU Family: 16
:CPU Model Num: 4
:Number of CPUs: 2
:CPU Speed: 3200
:System Memory: 15545
:System Swap: 17599
:Vendor: Gigabyte Technology Co., Ltd.
:System: GA-880GM-UD2H 
:Form factor: Desktop
:Kernel: 3.3.1-3.fc16.x86_64
:SELinux Enabled: 1
:SELinux Policy: targeted
:SELinux Enforce: Enforcing
:MythTV Remote: Unknown
:MythTV Role: Unknown
:MythTV Theme: Unknown
:MythTV Plugin: 
:MythTV Tuner: -1
:
:
:Devices
:=================================
:(4098:17296:5208:45058) pci, ahci, STORAGE, GA-MA770-DS3rev2.0 Motherboard
:(4130:38409:4130:38401) pci, pcieport, PCI/PCI, RS780/RS880 PCI to PCI bridge (PCIE port 5)
:(4098:17302:5208:20484) pci, ehci_hcd, USB, SB7x0/SB8x0/SB9x0 USB EHCI Controller
:(4130:38401:4130:38401) pci, None, HOST/PCI, RS880 Host Bridge
:(4130:38402:4130:38402) pci, None, PCI/PCI, RS780/RS880 PCI to PCI bridge (int gfx)
:(4130:4612:0:0) pci, None, HOST/PCI, Family 10h Processor Link Control
:(4172:32804:5208:4096) pci, firewire_ohci, FIREWIRE, GA-EP45-DS5 Motherboard
:(4130:4609:0:0) pci, None, HOST/PCI, Family 10h Processor Address Map
:(4130:4608:0:0) pci, None, HOST/PCI, Family 10h Processor HyperTransport Configuration
:(4130:4611:0:0) pci, k10temp, HOST/PCI, Family 10h Processor Miscellaneous Control
:(4130:4610:0:0) pci, None, HOST/PCI, Family 10h Processor DRAM Controller
:(4332:33128:5208:57344) pci, r8169, ETHERNET, GA-EP45-DS5 Motherboard
:(4098:17308:5208:20482) pci, pata_atiixp, STORAGE, SB7x0/SB8x0/SB9x0 IDE Controller
:(4098:17285:5208:17285) pci, None, SERIAL, GA-MA770-DS3rev2.0 Motherboard
:(4098:17309:4098:17309) pci, None, PCI/ISA, SB7x0/SB8x0/SB9x0 LPC host controller
:(4098:17283:5208:41218) pci, snd_hda_intel, MULTIMEDIA, SBx00 Azalia (Intel HDA)
:(4098:17305:5208:20484) pci, ohci_hcd, USB, SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
:(4098:17284:0:0) pci, None, PCI/PCI, SBx00 PCI to PCI Bridge
:(4098:38677:5208:53248) pci, radeon, VIDEO, RS880 [Radeon HD 4250]
:(4098:38671:5208:38415) pci, snd_hda_intel, MULTIMEDIA, RS880 Audio Device [Radeon HD 4200]
:(4098:17303:5208:20484) pci, ohci_hcd, USB, SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
:(4098:17304:5208:20484) pci, ohci_hcd, USB, SB7x0 USB OHCI1 Controller
:(4098:17302:5208:20484) pci, ehci_hcd, USB, SB7x0/SB8x0/SB9x0 USB EHCI Controller
:(4098:17304:5208:20484) pci, ohci_hcd, USB, SB7x0 USB OHCI1 Controller
:(4098:17303:5208:20484) pci, ohci_hcd, USB, SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
:
:
:Filesystem Information
:=================================
:device mtpt type bsize frsize blocks bfree bavail file ffree favail
:-------------------------------------------------------------------
:/dev/mapper/vg_server-lv_root / ext4 4096 4096 13092026 11463409 11332387 3276800 3096984 3096984
:/dev/sda1 /boot ext4 1024 1024 508745 409796 384196 128016 127780 127780
:/dev/mapper/vg_server-lv_home /home ext4 4096 4096 226215966 190926011 179603848 56614912 56578533 56578533
:/dev/sde3 WITHHELD vfat UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN
:/dev/sde2 WITHHELD vfat UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN
:/dev/sde1 WITHHELD vfat UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN UNKNOWN
:/dev/sr0 WITHHELD udf 2048 2048 4129176 0 0 47 0 0
:
Comment 1 Josh Boyer 2012-04-10 00:27:19 UTC 
Eric, anything you have seen before?
Comment 2 Eric Sandeen 2012-04-10 22:39:09 UTC 
nope...
Comment 3 Dave Jones 2012-07-16 22:05:46 UTC 
that ffffffff79178ff0 address is bugging me.
It's below the start of the kernel text, so it looks like we've jumped off to some crazy address indirectly from delete_from_page_cache.

Almost as if we had a mapping->a_ops->freepage corruption.

I wonder if that corruption was the result of some bit-flip, and it was supposed to be ffffffffa9178ff0 for example (which would put it in module space).

that would be a multi-bit flip though, going from %1010 to %0111, which seems astronomically low odds.


Can you still reproduce this on 3.4 ?
Comment 4 Dave Jones 2012-07-16 22:10:04 UTC 
actually, looking at this and your other trace in 811885, it really looks like there's some kind of low level hardware problem here, which could be any number of things.

I don't think this is a software fault. Which would also explain why we haven't had similar reports.