Bug 545599

Summary: lirc_imon.ko crashes with new kernel
Product: [Fedora] Fedora Reporter: Tom Horsley <horsley1953>
Component: kernelAssignee: Jarod Wilson <jarodwilson>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 12CC: dougsland, gansalmon, itamar, jarodwilson, kernel-maint
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-2.6.32.10-90.fc12 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-17 14:50:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tom Horsley 2009-12-08 22:09:33 UTC 
Description of problem:

Just updated to kernel-2.6.31.6-162.fc12.x86_64, and when I rebooted, a
kernel backtrace flashed across the screen too fast to see and then
the system hung at "setting hostname".

This is the same symptom I've had previously when the lirc_imon driver
couldn't deal with my (apparently incredibly old) imon device:

Bus 004 Device 002: ID 0aa8:8001 TriGem Computer, Inc. TG_iMON

I renamed the lirc_imon.ko module while booted in a working kernel,
and then was able to boot again with no problem into 2.6.31.6-162.

Version-Release number of selected component (if applicable):
kernel-2.6.31.6-162.fc12.x86_64

How reproducible:
I only tried booting once, but if the past is any guide, it would
happen every time. I'm going to update my 32 bit fedora 12 partition
soon, and I'll see if it winds up doing the same thing.

Steps to Reproduce:
1.yum update
2.reboot
3.
  
Actual results:
hung system during boot

Expected results:
booted system

Additional info:

bug 459523 and bug 520008 are probably earlier versions of this same
problem (which is how I've come to recognize it :-).
Comment 1 Tom Horsley 2009-12-08 22:25:00 UTC 
Yep, the 2.6.31.6-162 PAE i686 kernel also crashes the same way.
Comment 2 Tom Horsley 2009-12-11 01:34:25 UTC 
And the new 2.6.31.6-166 kernels also crash. The walkback was still mostly
on the screen in the 1686 version, and I wrote down some (but definitely
not all) of the info I saw:

EIP is at send_packet+0x1a/0x1ba [lirc_imon]
Process modprobe
Call trace: imon_set_ir_protocol
imon_probe
usb_probe_interface
driver_probe_device
__driver_attach
...
imon_init
...
Comment 3 Tom Horsley 2009-12-11 01:44:00 UTC 
OK, came up with the idea of renaming the lirc_imon module back to original
name and manually doing a modprobe and was able to record the stdout/stderr
and /var/log/messages spew (which seems to be the same as the boot time
spew). This is on x86_64 kernel:

[root@zooty lirc]# mv NOTlirc_imon.koNOT lirc_imon.ko
[root@zooty lirc]# modprobe lirc_imon

Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:Oops: 0000 [#1] SMP 

Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:last sysfs file: /sys/module/lirc_dev/initstate
Killed
[root@zooty lirc]# 
Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:Stack:

Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:Call Trace:

Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:Code: 48 89 e5 41 55 41 54 53 48 83 ec 08 0f 1f 44 00 00 83 bf a0 00 00 00 00 48 89 fb 0f 85 97 00 00 00 48 8b 8f 80 00 00 00 48 8b 37 <0f> b6 41 02 8b 16 0f b6 49 06 c1 e2 08 c1 e0 0f 09 c2 48 8b 87 

Message from syslogd@zooty at Dec 10 20:36:09 ...
 kernel:CR2: 0000000000000002

From /var/log/messages:

Dec 10 20:36:09 zooty kernel: Process modprobe (pid: 2489, threadinfo ffff88021a126000, task ffff88021a191780)
Dec 10 20:36:09 zooty kernel: Stack:
Dec 10 20:36:09 zooty kernel: ffff88022ca43800 ffff8802336bb230 ffff88022ca43800 ffff8802336bb200
Dec 10 20:36:09 zooty kernel: <0> ffff88021a127c78 ffffffffa0448879 8600000000000000 00000000c195e6a6
Dec 10 20:36:09 zooty kernel: <0> ffff880233ce7898 ffff88022ca43800 0000000000000000 ffff8802336bb200
Dec 10 20:36:09 zooty kernel: Call Trace:
Dec 10 20:36:09 zooty kernel: [<ffffffffa0448879>] imon_set_ir_protocol+0x11c/0x1f1 [lirc_imon]
Dec 10 20:36:09 zooty kernel: [<ffffffffa044a47c>] imon_probe+0xe5b/0x114b [lirc_imon]
Dec 10 20:36:09 zooty kernel: [<ffffffff81150642>] ? __sysfs_add_one+0x34/0x8e
Dec 10 20:36:09 zooty kernel: [<ffffffff81302b55>] usb_probe_interface+0x143/0x1f4
Dec 10 20:36:09 zooty kernel: [<ffffffff812abb1c>] driver_probe_device+0xdb/0x1fb
Dec 10 20:36:09 zooty kernel: [<ffffffff812abc99>] __driver_attach+0x5d/0x81
Dec 10 20:36:09 zooty kernel: [<ffffffff812abc3c>] ? __driver_attach+0x0/0x81
Dec 10 20:36:09 zooty kernel: [<ffffffff812aafba>] bus_for_each_dev+0x53/0x88
Dec 10 20:36:09 zooty kernel: [<ffffffff812ab8bb>] driver_attach+0x1e/0x20
Dec 10 20:36:09 zooty kernel: [<ffffffff812ab578>] bus_add_driver+0xf7/0x257
Dec 10 20:36:09 zooty kernel: [<ffffffff812abf84>] driver_register+0x9d/0x10e
Dec 10 20:36:09 zooty kernel: [<ffffffff813028b7>] usb_register_driver+0x93/0xfa
Dec 10 20:36:09 zooty kernel: [<ffffffffa0452000>] ? imon_init+0x0/0x4e [lirc_imon]
Dec 10 20:36:09 zooty kernel: [<ffffffffa045202c>] imon_init+0x2c/0x4e [lirc_imon]
Dec 10 20:36:09 zooty kernel: [<ffffffff8100a069>] do_one_initcall+0x5e/0x162
Dec 10 20:36:09 zooty kernel: [<ffffffff8107bd5a>] sys_init_module+0xd6/0x20c
Dec 10 20:36:09 zooty kernel: [<ffffffff81011cf2>] system_call_fastpath+0x16/0x1b
Dec 10 20:36:09 zooty kernel: Code: 48 89 e5 41 55 41 54 53 48 83 ec 08 0f 1f 44 00 00 83 bf a0 00 00 00 00 48 89 fb 0f 85 97 00 00 00 48 8b 8f 80 00 00 00 48 8b 37 <0f> b6 41 02 8b 16 0f b6 49 06 c1 e2 08 c1 e0 0f 09 c2 48 8b 87 
Dec 10 20:36:09 zooty kernel: RIP  [<ffffffffa0448461>] send_packet+0x2c/0x246 [lirc_imon]
Dec 10 20:36:09 zooty kernel: RSP <ffff88021a127c18>
Dec 10 20:36:09 zooty kernel: CR2: 0000000000000002
Dec 10 20:36:09 zooty kernel: ---[ end trace 5c3926942f01303e ]---
Comment 4 Tom Horsley 2009-12-27 03:39:31 UTC 
Just a note that kernel 2.6.31.9-174.fc12.x86_64 still crashes the same way.
Comment 5 Tom Horsley 2010-01-22 22:29:27 UTC 
The new 2.6.31.12-174.2.3.fc12.x86_64 is still crashing in lirc_imon.ko.
Comment 6 Tom Horsley 2010-02-20 13:28:42 UTC 
And the latest 2.6.31.12-174.2.19.fc12.x86_64 still crashes. Is there
anyone out there working on lirc drivers anymore? Is my old model device
just dead forever? Should I blacklist it so this stops happening
on every kernel update?
Comment 7 Tom Horsley 2010-02-22 02:34:25 UTC 
Yet another new kernel appeared a few days after the last one, but
2.6.31.12-174.2.22.fc12.x86_64 still has the lirc_imon.ko crash.
Comment 8 Jarod Wilson 2010-02-25 19:31:37 UTC 
Hadn't seen this bz until just now. This should be fixed in the 2.6.32.x builds in updates-testing. If not, its definitely fixed in my lirc git tree, and I just need to update the Fedora lirc patches...
Comment 9 Tom Horsley 2010-03-06 19:31:59 UTC 
Just got the released kernel update to:

kernel-2.6.32.9-67.fc12.x86_64

lirc_imon.ko still crashes during udev with this kernel.
Comment 10 Tom Horsley 2010-03-15 00:23:26 UTC 
kernel-2.6.32.9-70.fc12.x86_64 still has the lirc_imon.ko crash
(just to keep this bug updated :-).
Comment 11 Jarod Wilson 2010-03-15 01:52:21 UTC 
Apologies for the radio silence. I should have an imon driver update coming shortly. Once I get it patched into the kernel and a build pushed through the build system, I'll give you a heads up (hoping to get that done tonight or tomorrow).
Comment 12 Tom Horsley 2010-03-15 17:22:04 UTC 
Not a problem. I don't actually use the IR controller anyway. The only reason
I haven't blacklisted the driver is so I can report when it makes my system
crash :-).
Comment 13 Jarod Wilson 2010-03-15 18:29:30 UTC 
Okay, so I'm in need of the current backtrace w/-70. There's no way its 100% the same thing as what's in comment #3, because imon_set_ir_protocol is referenced in that trace, and its no longer in the lirc_imon driver, so my theory as to what was crashing has to be thrown out the window. :)
Comment 14 Tom Horsley 2010-03-15 21:36:16 UTC 
OK, I renamed lirc_imon.ko back in the -70 kernel modules dir, then did a
modprobe for it:

[root@zooty lirc]# modprobe lirc_imon
Killed
[root@zooty lirc]# 
Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:Oops: 0002 [#1] SMP 

Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:last sysfs file: /sys/module/lirc_dev/initstate

Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:Stack:

Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:Call Trace:

Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:Code: 56 41 55 41 54 53 48 83 ec 78 0f 1f 44 00 00 48 c7 c6 d0 b2 1e a0 48 8d 47 30 49 89 fd 48 89 45 c0 e8 84 9b 14 e1 48 85 c0 74 0d <c7> 04 25 08 00 00 00 00 00 00 00 eb 07 c7 40 08 01 00 00 00 49 

Message from syslogd@zooty at Mar 15 17:27:29 ...
 kernel:CR2: 0000000000000008

This showed up in /var/log/messages at the same time:

MarMar 15 17:27:29 zooty kernel: lirc_imon: Driver for SoundGraph iMON MultiMedia IR/Display, v0.8
Mar 15 17:27:29 zooty kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
Mar 15 17:27:29 zooty kernel: IP: [<ffffffffa01e9b28>] imon_probe+0x32/0x8ae [lirc_imon]
Mar 15 17:27:29 zooty kernel: PGD 884d2067 PUD 884fe067 PMD 0 
Mar 15 17:27:29 zooty kernel: Oops: 0002 [#1] SMP 
Mar 15 17:27:29 zooty kernel: last sysfs file: /sys/module/lirc_dev/initstate
Mar 15 17:27:29 zooty kernel: CPU 2 
Mar 15 17:27:29 zooty kernel: Modules linked in: lirc_imon(+) ppp_async crc_ccitt ppp_generic slhc serpent twofish_x86_64 twofish_common cryptd aes_x86_64 aes_generic xts gf128mul dm_crypt fuse nfsd lockd nfs_acl auth_rpcgss exportfs hwmon_vid coretemp sunrpc cpufreq_ondemand acpi_cpufreq freq_table bridge stp llc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 kvm_intel kvm uinput snd_hda_codec_realtek snd_ca0106 snd_hda_intel snd_ac97_codec snd_hda_codec snd_usb_audio ac97_bus snd_seq usblp snd_pcm snd_timer i2c_i801 iTCO_wdt iTCO_vendor_support e1000e snd_usb_lib snd_rawmidi snd_seq_device snd_hwdep snd snd_page_alloc soundcore serio_raw ppdev parport_pc parport joydev uvcvideo videodev v4l1_compat v4l2_compat_ioctl32 lirc_dev ata_generic pata_acpi dm_multipath pata_jmicron radeon ttm drm_kms_helper drm i2c_algo_bit i2c_core [last unloaded: microcode]
Mar 15 17:27:29 zooty kernel: Pid: 9464, comm: modprobe Not tainted 2.6.32.9-70.fc12.x86_64 #1 TP43D2-A7
Mar 15 17:27:29 zooty kernel: RIP: 0010:[<ffffffffa01e9b28>]  [<ffffffffa01e9b28>] imon_probe+0x32/0x8ae [lirc_imon]
Mar 15 17:27:29 zooty kernel: RSP: 0018:ffff8800a9425ca8  EFLAGS: 00010282
Mar 15 17:27:29 zooty kernel: RAX: ffffffffa01eb2d0 RBX: ffff880233772430 RCX: ffffffffa01eb1b0
Mar 15 17:27:29 zooty kernel: RDX: 0000000000000003 RSI: ffffffffa01eb2d0 RDI: ffff88023737d000
Mar 15 17:27:29 zooty kernel: RBP: ffff8800a9425d48 R08: ffffffff00000000 R09: ffff880224a72510
Mar 15 17:27:29 zooty kernel: R10: 0000000000000000 R11: ffff8800a9425dc8 R12: ffff880233772400
Mar 15 17:27:29 zooty kernel: R13: ffff880233772400 R14: ffffffffa01eb218 R15: ffff88023737d000
Mar 15 17:27:29 zooty kernel: FS:  00007fd4c4d7b700(0000) GS:ffff880008f00000(0000) knlGS:0000000000000000
Mar 15 17:27:29 zooty kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 15 17:27:29 zooty kernel: CR2: 0000000000000008 CR3: 0000000037b36000 CR4: 00000000000426e0
Mar 15 17:27:29 zooty kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 15 17:27:29 zooty kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 15 17:27:29 zooty kernel: Process modprobe (pid: 9464, threadinfo ffff8800a9424000, task ffff8800ab12dd80)
Mar 15 17:27:29 zooty kernel: Stack:
Mar 15 17:27:29 zooty kernel: ffff8800a9425cf8 ffffffff81177e6f ffff8802310ba730 0000000000000246
Mar 15 17:27:29 zooty kernel: <0> ffff8800a9425cf8 ffffffff81454e15 ffff8802310ba730 ffff8800852fd410
Mar 15 17:27:29 zooty kernel: <0> 0000000000000001 ffff88023737d000 ffff8800a9425d08 ffffffff81333bd7
Mar 15 17:27:29 zooty kernel: Call Trace:
Mar 15 17:27:29 zooty kernel: [<ffffffff81177e6f>] ? sysfs_add_one+0x21/0xf3
Mar 15 17:27:29 zooty kernel: [<ffffffff81454e15>] ? mutex_lock+0x29/0x50
Mar 15 17:27:29 zooty kernel: [<ffffffff81333bd7>] ? usb_pm_unlock+0x15/0x17
Mar 15 17:27:29 zooty kernel: [<ffffffff81333ee0>] ? usb_autopm_do_device+0xc4/0xcf
Mar 15 17:27:29 zooty kernel: [<ffffffff813348fd>] usb_probe_interface+0x14f/0x207
Mar 15 17:27:29 zooty kernel: [<ffffffff812da492>] driver_probe_device+0xea/0x217
Mar 15 17:27:29 zooty kernel: [<ffffffff812da61c>] __driver_attach+0x5d/0x81
Mar 15 17:27:29 zooty kernel: [<ffffffff812da5bf>] ? __driver_attach+0x0/0x81
Mar 15 17:27:29 zooty kernel: [<ffffffff812d98a4>] bus_for_each_dev+0x53/0x88
Mar 15 17:27:29 zooty kernel: [<ffffffff812da222>] driver_attach+0x1e/0x20
Mar 15 17:27:29 zooty kernel: [<ffffffff812d9e62>] bus_add_driver+0xf7/0x25d
Mar 15 17:27:29 zooty kernel: [<ffffffff812da91c>] driver_register+0x9d/0x10e
Mar 15 17:27:29 zooty kernel: [<ffffffff81334651>] usb_register_driver+0x93/0xfa
Mar 15 17:27:29 zooty kernel: [<ffffffffa004a000>] ? imon_init+0x0/0x4e [lirc_imon]
Mar 15 17:27:29 zooty kernel: [<ffffffffa004a02c>] imon_init+0x2c/0x4e [lirc_imon]
Mar 15 17:27:29 zooty kernel: [<ffffffff8100a069>] do_one_initcall+0x5e/0x159
Mar 15 17:27:29 zooty kernel: [<ffffffff8108cf2e>] sys_init_module+0xd6/0x234
Mar 15 17:27:29 zooty kernel: [<ffffffff81011d32>] system_call_fastpath+0x16/0x1b
Mar 15 17:27:29 zooty kernel: Code: 56 41 55 41 54 53 48 83 ec 78 0f 1f 44 00 00 48 c7 c6 d0 b2 1e a0 48 8d 47 30 49 89 fd 48 89 45 c0 e8 84 9b 14 e1 48 85 c0 74 0d <c7> 04 25 08 00 00 00 00 00 00 00 eb 07 c7 40 08 01 00 00 00 49 
Mar 15 17:27:29 zooty kernel: RIP  [<ffffffffa01e9b28>] imon_probe+0x32/0x8ae [lirc_imon]
Mar 15 17:27:29 zooty kernel: RSP <ffff8800a9425ca8>
Mar 15 17:27:29 zooty kernel: CR2: 0000000000000008
Mar 15 17:27:29 zooty kernel: ---[ end trace 3469681d788827f7 ]---
Comment 15 Jarod Wilson 2010-03-16 17:32:09 UTC 
Okay, I see what's going wrong now, and should have a test build for you to try out soonish that ought to remedy this problem...
Comment 16 Jarod Wilson 2010-03-16 19:18:34 UTC 
A 2.6.32.10-78.fc12 kernel build with patches that I believe should fix things is working its way through the build system right now.

http://koji.fedoraproject.org/koji/taskinfo?taskID=2057058

Once its done, download links for the resulting binaries should be available there.
Comment 17 Jarod Wilson 2010-03-16 21:33:10 UTC 
Build just finished, please give it a try and let me know if it is indeed fixed...
Comment 18 Tom Horsley 2010-03-16 23:59:04 UTC 
Hey! Not only does the kernel boot without crashing, but if I run xmode2, I
see perfectly believable graphs show up when I press a remote button. Looks
like it is indeed fixed. Thanks.
Comment 19 Jarod Wilson 2010-03-17 14:50:51 UTC 
Excellent, good to hear. Closing the bug, next updates kernel that gets pushed should carry the fix.
Comment 20 Fedora Update System 2010-03-23 14:56:00 UTC 
kernel-2.6.32.10-90.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/kernel-2.6.32.10-90.fc12
Comment 21 Fedora Update System 2010-03-30 02:23:41 UTC 
kernel-2.6.32.10-90.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.