Additional info: reporter: libreport-2.1.10 WARNING: CPU: 4 PID: 266 at lib/list_debug.c:29 __list_add+0x65/0xc0() r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded list_add corruption. next->prev should be prev (ffffffff81c98690), but was ffff8800d8c5e480. (next=ffffffff81c98690). r8169 0000:05:00.0: can't disable ASPM; OS doesn't have ASPM control pcieport 0000:00:1c.5: driver skip pci_set_master, fix it! Modules linked in: usblp(+) r8169(+) rfkill lpc_ich i2c_i801 serio_raw uinput mii snd_page_alloc drm mfd_core snd_timer snd i2c_core mei_me mei soundcore shpchp firewire_ohci firewire_core crc_itu_t video wmi CPU: 4 PID: 266 Comm: systemd-udevd Not tainted 3.12.6-200.fc19.x86_64 #1 Hardware name: System manufacturer System Product Name/P8P67, BIOS 3602 11/01/2012 0000000000000009 ffff8800da2f7aa8 ffffffff81662651 ffff8800da2f7af0 ffff8800da2f7ae0 ffffffff810691dd ffff880407f68de0 ffffffff81c98690 ffffffff81c98690 0000000000000000 ffffffffa0120360 ffff8800da2f7b40 Call Trace: [<ffffffff81662651>] dump_stack+0x45/0x56 [<ffffffff810691dd>] warn_slowpath_common+0x7d/0xa0 [<ffffffff8106924c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff8131a9a5>] __list_add+0x65/0xc0 [<ffffffff8132fd5f>] pinctrl_get+0x2df/0x470 [<ffffffff8132ff35>] devm_pinctrl_get+0x45/0x80 [<ffffffff81417deb>] pinctrl_bind_pins+0x3b/0x220 [<ffffffff813ff4ca>] driver_probe_device+0x6a/0x3a0 [<ffffffff813ff8d3>] __driver_attach+0x93/0xa0 [<ffffffff813ff840>] ? __device_attach+0x40/0x40 [<ffffffff813fd3a3>] bus_for_each_dev+0x63/0xa0 [<ffffffff813fef3e>] driver_attach+0x1e/0x20 [<ffffffff813feab0>] bus_add_driver+0x200/0x2d0 [<ffffffff813fff14>] driver_register+0x64/0xf0 [<ffffffff8147c242>] usb_register_driver+0x82/0x160 [<ffffffffa013c000>] ? 0xffffffffa013bfff [<ffffffffa013c01e>] usblp_driver_init+0x1e/0x1000 [usblp] [<ffffffff810020f2>] do_one_initcall+0xf2/0x1a0 [<ffffffff81054533>] ? set_memory_nx+0x43/0x50 [<ffffffff810d687d>] load_module+0x1b9d/0x2640 [<ffffffff810d2a30>] ? store_uevent+0x40/0x40 [<ffffffff810d7496>] SyS_finit_module+0x86/0xb0 [<ffffffff816716a9>] system_call_fastpath+0x16/0x1b Potential duplicate: bug 1051907
Created attachment 849022 [details] File: dmesg
Does this happen on every boot? If you blacklist the r8169 driver does the backtrace go away?
Good Morning Mr. Boyer. I'm only an italian end user of fedora, not a programmer, but I love free os. So... I hope to help you to make the system work better. This problem occurred only that time. Your automatic system has alerted me and I decided to reboot pc for safety. All worked well but I decided to send this abrt alert because I think that there is some problems though I thought they where related to network manager, not to hardware. I know that driver of my hardware is unable to use it at 1Gb/s speed but there is no problem in normal use (my pc isn't directly connected to other computers) ***** My hardware from lshw: product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10EC:8168] vendor: Realtek Semiconductor Co., Ltd. [10EC] bus info: pci@0000:05:00.0 logical name: p5p1 version: 06 serial: f4:6d:04:ad:4e:79 size: 100Mbit/s capacity: 1Gbit/s width: 64 bits clock: 33MHz capabilities: Power Management, Message Signalled Interrupts, PCI Express, MSI-X, Vital Product Data, bus mastering, PCI capabilities listing, ethernet, Physical interface, twisted pair, Media Independant Interface, 10Mbit/s, 10Mbit/s (full duplex), 100Mbit/s, 100Mbit/s (full duplex), 1Gbit/s, 1Gbit/s (full duplex), Auto-negotiation configuration: autonegotiation: on broadcast: yes driver: r8169 driverversion: 2.3LK-NAPI duplex: full firmware: rtl8168e-3_0.0.4 03/27/12 ip: 10.0.0.5 latency: 0 link: yes multicast: yes port: MII speed: 100Mbit/s resources: irq: 60 ioport: d000(size=256) memory: f2104000-f2104fff memory: f2100000-f2103fff ****** But in recent times I have problems: I can connect to the network (the router) only after a long time. Start the pc after a few hours after the last use means that, when I login in my work session, the old connection dies and the new connection takes a long time to be born. From my dmesg (last boot) Jan 13 17:20:59 AsusCasa NetworkManager[876]: <info> (p5p1): carrier is OFF Jan 13 17:20:59 AsusCasa NetworkManager[876]: <info> (p5p1): new Ethernet device (driver: 'r8169' ifindex: 2) Jan 13 17:20:59 AsusCasa NetworkManager[876]: <info> (p5p1): exported as /org/freedesktop/NetworkManager/Devices/0 Jan 13 17:20:59 AsusCasa NetworkManager[876]: <info> (p5p1): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2] Jan 13 17:20:59 AsusCasa NetworkManager[876]: <info> (p5p1): bringing up device. ************** Jan 13 17:20:59 Asus kernel: [ 11.248630] r8169 0000:05:00.0 p5p1: link down ***************** ********************** Jan 13 17:25:01 Asus kernel: [ 253.292585] r8169 0000:05:00.0 p5p1: link up Four minutes !!! But after this alert I don't know if there is a driver problem, so I send alert. About this problem, at the time my last "dmesg" write this words about r8169 (also in dmsg of abrt) but it works. Jan 13 17:20:58 AsusCasa kernel: [ 6.257351] type=1403 audit(1389630054.441:3): policy loaded auid=4294967295 ses=4294967295 Jan 13 17:20:58 AsusCasa kernel: [ 6.841524] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 Jan 13 17:20:58 AsusCasa kernel: [ 6.846626] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded Jan 13 17:20:58 AsusCasa kernel: [ 6.846631] r8169 0000:05:00.0: can't disable ASPM; OS doesn't have ASPM control Jan 13 17:20:58 AsusCasa kernel: [ 6.846634] pcieport 0000:00:1c.5: driver skip pci_set_master, fix it! Jan 13 17:20:58 AsusCasa kernel: [ 6.846987] r8169 0000:05:00.0 eth0: RTL8168evl/8111evl at 0xffffc90005bee000, f4:6d:04:ad:4e:79, XID 0c900800 IRQ 60 Jan 13 17:20:58 AsusCasa kernel: [ 6.846988] r8169 0000:05:00.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko] If I can be helpful in some way ..... Hi Alberto Chiodi
Description of problem: Simple boot the system Version-Release number of selected component: kernel Additional info: reporter: libreport-2.1.10 cmdline: BOOT_IMAGE=/vmlinuz-3.12.7-200.fc19.x86_64 root=/dev/mapper/VG_System-LV_System ro rd.md=0 rd.dm=0 rd.lvm.lv=VG_System/LV_Auslagerung1 rd.lvm.lv=VG_System/LV_System rd.luks=0 LANG=de_DE.UTF-8 SYSFONT=True KEYTABLE=de-latin1 rhgb quiet crashkernel=128M kernel: 3.12.7-200.fc19.x86_64 runlevel: unknown type: Kerneloops Truncated backtrace: WARNING: CPU: 4 PID: 421 at lib/list_debug.c:29 __list_add+0x65/0xc0() list_add corruption. next->prev should be prev (ffffffff81c986d0), but was ffff8800d6400660. (next=ffffffff81c986d0). Modules linked in: snd_hda_intel(+) snd_hda_codec snd_hwdep rfkill snd_seq snd_seq_device snd_pcm joydev usblp e1000e(+) snd_page_alloc ptp snd_timer shpchp snd pps_core iTCO_wdt iTCO_vendor_support x86_pkg_temp_thermal coretemp kvm_intel i2c_i801 soundcore lpc_ich mfd_core kvm mei_me mei tpm_tis crct10dif_pclmul crc32_pclmul tpm tpm_bios crc32c_intel ghash_clmulni_intel microcode binfmt_misc uinput usb_storage i915 i2c_algo_bit drm_kms_helper drm firewire_ohci firewire_core crc_itu_t i2c_core video CPU: 4 PID: 421 Comm: systemd-udevd Not tainted 3.12.7-200.fc19.x86_64 #1 Hardware name: /DQ77MK, BIOS MKQ7710H.86A.0064.2013.1003.1058 10/03/2013 0000000000000009 ffff8807efaa1ac8 ffffffff81666f2b ffff8807efaa1b10 ffff8807efaa1b00 ffffffff8106921d ffff8807f16c1060 ffffffff81c986d0 ffffffff81c986d0 0000000000000000 ffffffffa038e960 ffff8807efaa1b60 Call Trace: [<ffffffff81666f2b>] dump_stack+0x45/0x56 [<ffffffff8106921d>] warn_slowpath_common+0x7d/0xa0 [<ffffffff8106928c>] warn_slowpath_fmt+0x4c/0x50 [<ffffffff8131b115>] __list_add+0x65/0xc0 [<ffffffff813305bf>] pinctrl_get+0x2df/0x470 [<ffffffff81330795>] devm_pinctrl_get+0x45/0x80 [<ffffffff8141ce6b>] pinctrl_bind_pins+0x3b/0x220 [<ffffffff8140454a>] driver_probe_device+0x6a/0x3a0 [<ffffffff81404953>] __driver_attach+0x93/0xa0 [<ffffffff814048c0>] ? __device_attach+0x40/0x40 [<ffffffff81402423>] bus_for_each_dev+0x63/0xa0 [<ffffffff81403fbe>] driver_attach+0x1e/0x20 [<ffffffff81403b30>] bus_add_driver+0x200/0x2d0 [<ffffffffa039b168>] ? ftrace_define_fields_azx_get_position+0xcc/0xcc [snd_hda_intel] [<ffffffff81404f94>] driver_register+0x64/0xf0 [<ffffffffa039b168>] ? ftrace_define_fields_azx_get_position+0xcc/0xcc [snd_hda_intel] [<ffffffff8133fcfc>] __pci_register_driver+0x4c/0x50 [<ffffffffa039b186>] azx_driver_init+0x1e/0xe98 [snd_hda_intel] [<ffffffff810020f2>] do_one_initcall+0xf2/0x1a0 [<ffffffff81054533>] ? set_memory_nx+0x43/0x50 [<ffffffff810d693d>] load_module+0x1b9d/0x2640 [<ffffffff810d2af0>] ? store_uevent+0x40/0x40 [<ffffffff810d7556>] SyS_finit_module+0x86/0xb0 [<ffffffff81675ee9>] system_call_fastpath+0x16/0x1b
It happens only random, not at every boot. And in my case I don't have any Realtek LAN chip.
*** Bug 1051907 has been marked as a duplicate of this bug. ***
Created attachment 854892 [details] pinctrl_protect_list_add.patch Proposed fix for this bug.
I launched kernel build with a patch here: http://koji.fedoraproject.org/koji/taskinfo?taskID=6448820 Please test it (when it finish to compile).
Hm, try to install the testing kernel fails. yum update --enablerepo=updates-testing kernel-3.12.8-200.bz1051918.fc19 no package kernel-3.12.8-200.bz1051918.fc19 available.
Frank, Stanislaw's test kernel has not hit updates-testing. You need to download it from the URL at comment 8. If you can confirm the fix it'll help pushing it into an official update. cheers, Michele
It fails because yum think that an it is an older kernel. :( LANG=C yum update http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-headers-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-tools-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-tools-libs-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-tools-libs-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/perf-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/python-perf-3.12.8-200.bz1051918.fc19.x86_64.rpm http://kojipkgs.fedoraproject.org//work/tasks/8821/6448821/kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64.rpm Loaded plugins: changelog, fastestmirror, langpacks, refresh-packagekit kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm | 30 MB 00:00:26 Examining /var/tmp/yum-root-0ILrOg/kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-3.12.8-200.bz1051918.fc19.x86_64 Package kernel.x86_64 not installed, cannot update it. Run yum install to install it instead. Marking /var/tmp/yum-root-0ILrOg/kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm as an update to kernel-3.12.6-200.fc19.x86_64 Marking /var/tmp/yum-root-0ILrOg/kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm as an update to kernel-3.12.7-200.fc19.x86_64 kernel-headers-3.12.8-200.bz1051918.fc19.x86_64.rpm | 898 kB 00:00:01 Examining /var/tmp/yum-root-0ILrOg/kernel-headers-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-headers-3.12.8-200.bz1051918.fc19.x86_64 /var/tmp/yum-root-0ILrOg/kernel-headers-3.12.8-200.bz1051918.fc19.x86_64.rpm: does not update installed package. kernel-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm | 8.5 MB 00:00:08 Examining /var/tmp/yum-root-0ILrOg/kernel-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-devel-3.12.8-200.bz1051918.fc19.x86_64 Package kernel-devel not installed, cannot update it. Run yum install to install it instead. kernel-tools-3.12.8-200.bz1051918.fc19.x86_64.rpm | 105 kB 00:00:00 Examining /var/tmp/yum-root-0ILrOg/kernel-tools-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-tools-3.12.8-200.bz1051918.fc19.x86_64 Package kernel-tools not installed, cannot update it. Run yum install to install it instead. kernel-tools-libs-3.12.8-200.bz1051918.fc19.x86_64.rpm | 44 kB 00:00:00 Examining /var/tmp/yum-root-0ILrOg/kernel-tools-libs-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-tools-libs-3.12.8-200.bz1051918.fc19.x86_64 Package kernel-tools-libs not installed, cannot update it. Run yum install to install it instead. kernel-tools-libs-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm | 41 kB 00:00:00 Examining /var/tmp/yum-root-0ILrOg/kernel-tools-libs-devel-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-tools-libs-devel-3.12.8-200.bz1051918.fc19.x86_64 Package kernel-tools-libs-devel not installed, cannot update it. Run yum install to install it instead. perf-3.12.8-200.bz1051918.fc19.x86_64.rpm | 608 kB 00:00:01 Examining /var/tmp/yum-root-0ILrOg/perf-3.12.8-200.bz1051918.fc19.x86_64.rpm: perf-3.12.8-200.bz1051918.fc19.x86_64 Package perf not installed, cannot update it. Run yum install to install it instead. python-perf-3.12.8-200.bz1051918.fc19.x86_64.rpm | 99 kB 00:00:00 Examining /var/tmp/yum-root-0ILrOg/python-perf-3.12.8-200.bz1051918.fc19.x86_64.rpm: python-perf-3.12.8-200.bz1051918.fc19.x86_64 Package python-perf not installed, cannot update it. Run yum install to install it instead. kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64.rpm | 2.0 MB 00:00:02 Examining /var/tmp/yum-root-0ILrOg/kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64.rpm: kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64 Package kernel-modules-extra.x86_64 not installed, cannot update it. Run yum install to install it instead. Marking /var/tmp/yum-root-0ILrOg/kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64.rpm as an update to kernel-modules-extra-3.12.6-200.fc19.x86_64 Marking /var/tmp/yum-root-0ILrOg/kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64.rpm as an update to kernel-modules-extra-3.12.7-200.fc19.x86_64 Resolving Dependencies --> Running transaction check ---> Package kernel.x86_64 0:3.12.8-200.bz1051918.fc19 will be installed ---> Package kernel-modules-extra.x86_64 0:3.12.8-200.bz1051918.fc19 will be installed --> Finished Dependency Resolution Changes in packages about to be updated: --> Running transaction check ---> Package kernel.x86_64 0:3.12.6-200.fc19 will be erased ---> Package kernel-modules-extra.x86_64 0:3.12.6-200.fc19 will be erased --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================================= Package Arch Version Repository Size ================================================================================================================================================= Installing: kernel x86_64 3.12.8-200.bz1051918.fc19 /kernel-3.12.8-200.bz1051918.fc19.x86_64 131 M kernel-modules-extra x86_64 3.12.8-200.bz1051918.fc19 /kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64 8.1 M Removing: kernel x86_64 3.12.6-200.fc19 @updates 131 M kernel-modules-extra x86_64 3.12.6-200.fc19 @updates 8.1 M Transaction Summary ================================================================================================================================================= Install 2 Packages Remove 2 Packages Total size: 139 M Is this ok [y/d/N]: y Downloading packages: Running transaction check Running transaction test Transaction check error: package kernel-3.12.8-200.fc19.x86_64 (which is newer than kernel-3.12.8-200.bz1051918.fc19.x86_64) is already installed package kernel-modules-extra-3.12.8-200.fc19.x86_64 (which is newer than kernel-modules-extra-3.12.8-200.bz1051918.fc19.x86_64) is already installed Error Summary -------------
You can download package by hand and install it using "rpm -ivh kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm". If it still complains you can use rpm --force option.
rpm -ivh --oldpackage kernel-3.12.8-200.bz1051918.fc19.x86_64.rpm should work just fine.
Since we don't have test results let's apply the patch and see if problem still happen on updated kernel. I posted patch here: http://marc.info/?l=linux-kernel&m=139150130201826&w=2
*** Bug 1045750 has been marked as a duplicate of this bug. ***
*** Bug 1047482 has been marked as a duplicate of this bug. ***
Josh, please apply fix for this bug (I forgot to cc you previously). Patch is currently in Linus' tree: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=7b320cb1ed2dbd2c5f2a778197baf76fd6bf545a
*** Bug 1047648 has been marked as a duplicate of this bug. ***
Applied, thanks!
kernel-3.13.3-200.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/kernel-3.13.3-200.fc20
kernel-3.12.11-200.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/kernel-3.12.11-200.fc19
This was hitting us on our Intel 2600GZ4GC platforms, failure every 30-50 reboots. Putting the mutex around the list op fixed it for us (added it as a patch to 3.12.10-200), thanks! Reading back in the kernel history is a commit removing the acquisition of that lock with a fairly complex justification, I'm kind of worried there may be other MP issues with the list mgmt in this driver now.
Package kernel-3.12.11-200.fc19: * should fix your issue, * was pushed to the Fedora 19 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing kernel-3.12.11-200.fc19' as soon as you are able to, then reboot. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-2556/kernel-3.12.11-200.fc19 then log in and leave karma (feedback).
kernel-3.13.3-201.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.