Bug 499651

This bug appears to have been reported against 'rawhide' during the Fedora 11 development cycle.
Changing version to '11'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

me too. About 50% of my resumes hang on Fedora 11.
I got the above trace, but it not restricted to that path.
I can't find my traces in kerneloops.org as it's not searchable !

Same here, T43, 
two traces during suspend/resume:

BUG: sleeping function called from invalid context at kernel/workqueue.c:440
in_atomic(): 0, irqs_disabled(): 1, pid: 12053, name: pm-suspend
Pid: 12053, comm: pm-suspend Not tainted 2.6.29.5-186.fc11.i686.PAE #1
Call Trace:
 [<c042e4ee>] __might_sleep+0xe5/0xea
 [<c0443f9a>] flush_work+0x1f/0x96
 [<c071675c>] ? _spin_unlock_irqrestore+0x27/0x3d
 [<c044417f>] ? __queue_work+0x2f/0x34
 [<c0444240>] work_on_cpu+0x41/0x4b
 [<c04437e6>] ? do_work_for_cpu+0x0/0x17
 [<f8808454>] ? do_drv_read+0x0/0x45 [acpi_cpufreq]
 [<f8808391>] get_cur_val+0x9f/0xc4 [acpi_cpufreq]
 [<f8808412>] get_cur_freq_on_cpu+0x5c/0x9e [acpi_cpufreq]
 [<c0672bff>] ? cpufreq_cpu_get+0x78/0x9b
 [<c0672deb>] cpufreq_suspend+0xa7/0x11d
 [<c05ed6da>] sysdev_suspend+0x42/0x15b
 [<c0458463>] suspend_devices_and_enter+0xf3/0x186
 [<c0458662>] enter_state+0x142/0x19d
 [<c0458755>] state_store+0x98/0xac
 [<c04586bd>] ? state_store+0x0/0xac
 [<c05630b5>] kobj_attr_store+0x16/0x22
 [<c04e667d>] sysfs_write_file+0xc9/0xf4
 [<c04e65b4>] ? sysfs_write_file+0x0/0xf4
 [<c04a8b39>] vfs_write+0x95/0xf4
 [<c04a8c54>] sys_write+0x4c/0x70
 [<c040945f>] sysenter_do_call+0x12/0x34
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
Back to C!
Extended CMOS year: 2000
Force enabled HPET at resume
------------[ cut here ]------------
WARNING: at kernel/hrtimer.c:625 hres_timers_resume+0x34/0x4a() (Not tainted)
Hardware name: 266874G
hres_timers_resume() called with IRQs enabled!Modules linked in: fuse sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq dm_multipath uinput arc4 ecb snd_intel8x0 snd_ac97_codec ath5k ac97_bus mac80211 snd_pcm thinkpad_acpi snd_timer iTCO_wdt iTCO_vendor_support snd cfg80211 yenta_socket hwmon video soundcore rsrc_nonstatic tg3 i2c_i801 usb_storage snd_page_alloc output ata_generic pata_acpi sha256_generic cbc aes_i586 aes_generic dm_crypt radeon drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan]
Pid: 12053, comm: pm-suspend Not tainted 2.6.29.5-186.fc11.i686.PAE #1
Call Trace:
 [<c0435266>] warn_slowpath+0x7c/0xa4
 [<c0449bb3>] ? ktime_get_ts+0x4f/0x53
 [<c0419594>] ? lapic_next_event+0x18/0x1c
 [<c044f86e>] ? clockevents_program_event+0xe1/0xf0
 [<c04506fd>] ? tick_dev_program_event+0x47/0xb5
 [<c04507cb>] ? tick_program_event+0x26/0x2e
 [<c044fd1c>] ? tick_notify+0x2e8/0x2f7
 [<c041f5d3>] ? hpet_resume_counter+0x0/0x51
 [<c0718a3b>] ? notifier_call_chain+0x26/0x48
 [<c0449514>] hres_timers_resume+0x34/0x4a
 [<c044d8bc>] timekeeping_resume+0x130/0x138
 [<c05ed571>] __sysdev_resume+0x19/0x3d
 [<c05ed5bb>] sysdev_resume+0x26/0x59
 [<c0458482>] suspend_devices_and_enter+0x112/0x186
 [<c0458662>] enter_state+0x142/0x19d
 [<c0458755>] state_store+0x98/0xac
 [<c04586bd>] ? state_store+0x0/0xac
 [<c05630b5>] kobj_attr_store+0x16/0x22
 [<c04e667d>] sysfs_write_file+0xc9/0xf4
 [<c04e65b4>] ? sysfs_write_file+0x0/0xf4
 [<c04a8b39>] vfs_write+0x95/0xf4
 [<c04a8c54>] sys_write+0x4c/0x70
 [<c040945f>] sysenter_do_call+0x12/0x34
---[ end trace 100d0dfbfd82ef5d ]---

My laptop is an inspiron 6000 BTW.
Looks like I can grep /var/log/messages to see what's submitted to kerneloops.org:

http://www.kerneloops.org/submitresult.php?number=455828
http://www.kerneloops.org/submitresult.php?number=455108

I updated kernel from rawhide (2.6.29.4 -> 2.6.30) and the problem disappears.
However that new kernel has other issues precluding its use.
Note also that both new and old kernels occasionally suspend immediately again after a resume which is slightly annoying.

Same here. 

This issue seems related with cpufreq_ondemand kernel module. oops, when loaded, no oops, when not loaded.

Kernel 2.6.30 fixes other issues, too
e.g. https://bugzilla.redhat.com/show_bug.cgi?id=463110
https://bugzilla.redhat.com/show_bug.cgi?id=502270

but it looks like kernel 2.6.30 needs some fixes:
Regression since 2.6.29:
https://bugzilla.redhat.com/show_bug.cgi?id=497636

Maybe a fix for this? 
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=b14893a62c73af0eca414cfed505b8c09efc613c#patch1

Good catch! For me I also need to unload the acpi_cpufreq module though.

So before suspend, I do `/etc/init.d/cpuspeed stop; rmmod acpi_cpufreq` all is
OK.
However the caveat with that is that frequency scaling doesn't work once
acpi_cpufreq has been unloaded. It's auto-reloaded with /etc/init.d/cpuspeed
start, but scaling doesn't subsequently work.

As for the suspend immediately happening after a resume, that only happens when
I suspend using the Fn+Standby keyboard combo. If I close the lid and open it,
it does not happen.

Tried kernel-2.6.29.5-191.fc11 from Fedora 11 testing. Still same issue.
I checked and it did _not_ have the fix mentioned in comment #6.
So I rebuilt the cpufreq_ondemand module with that change and ...
still the same issue.

Note, I've not noticed a power increase actually by totally disabling frequency scaling as described above. Some even suggest that running the CPU at full speed can now take less power: http://mjg59.livejournal.com/88608.html

http://www.kerneloops.org/submitresult.php?number=523665

T43 hanging in suspend as well here.

Removing the acpi_cpufreq module fixes my crash on second resume as well.

I'm just chiming in to let everyone know that the advice to stop my beloved cpuspeed service and then rmmod acpi_cpufreq does allow my Dell Latitude D410 to properly suspend to ram and survive the subsequent resume without staring me in the face with a black lit up screen and no response to any input.  I kinda miss being able to suspend to ram (suspend to disk is about as slow as booting up cold and causes my bluetooth mouse to not function without first having to restart the bluetooth service and sometimes even that doesn't work).

Does anyone know if I permanently disable the cpuspeed service and don't use acpi_cpufreq if the CPU will be booted in top speed and left that way while it runs even if the CPU is idle?  That would kinda suck...I had an older D400 that I used to use cpuspeed on and it used to suspend and resume like clockwork.  But it died a horrible death still running Fedora 9 and never got a chance to see Fedora 10 or 11 to see if this same bug would have affected that same machine (I'm led to believe it would since its basically the same laptop with just slightly newer/faster components).

(In reply to comment #10)
> http://www.kerneloops.org/submitresult.php?number=523665
> 
> T43 hanging in suspend as well here.  

I removed acpi_cpufreq as well, and was able to suspend / resume multiple times without failure. But the laptop gets quite hot without cpufreq enabled :(

My laptop gets too hot as well.  So I created a little script that removes the module, does the suspend, then restarts cpuspeed on resume.

#!/bin/bash
service cpuspeed stop
modprobe -r acpi_cpufreq
pm-suspend
service cpuspeed start

Thanks for that useful tip Samuel. I'll use that as I've noticed my fan comming on more and battery lasting less when cpufreq is not running. So how common is this issue? What is the Highest Common Factor between our systems?

Mine is a Dell inspiron 6000 with the following hardware related modules loaded:

firewire_ohci          
snd_intel8x0           
snd_ac97_codec         
sdhci_pci               
ipw2200               
iTCO_wdt               
b44                    
mmc_core               
iTCO_vendor_support     
dell_laptop             
ata_generic             
pata_acpi               
i915

Can that script by Samuel Sieb be somehow integrated into the normal suspend configs/scripts?  I'm almost sure it can be, but I've never needed to tweak that aspect of Fedora before so I'm not too familiar with customizing the suspend or power management configs in Fedora.

Oh and is there a difference between using 'rmmod acpi_cpufreq' vs. 'modprobe -r acpi_cpufreq'?  I've been using the former and the script specifies the latter, which I was not aware you could do.

I think the difference is that modprobe will also remove any other dependent modules that aren't being used.  It's just habit for me now.  I think the following script should integrate with the normal suspend system if you put it in /etc/pm/sleep.d and make it executable:

#!/bin/bash
if [ "$1" == "suspend" ]; then
  service cpuspeed stop
  modprobe -r acpi_cpufreq
fi
if [ "$1" == "resume" ]; then
  service cpuspeed start
fi

Call the script 00-cpuspeed.sh just to be safe.

Thanks, I didn't know it was just as easy as how cron works (should have known).  If all it takes is just to create such a script and placing it in that folder making sure to make it executable, I'll give it a shot tonight and try it out.

Thanks for the example script and advice on the modprobe -f and rmmod differences, really appreciate it.  Am looking forward however, to a kernel fix for this annoying bug :D

(In reply to comment #20)
> Thanks, I didn't know it was just as easy as how cron works (should have
> known).  If all it takes is just to create such a script and placing it in that
> folder making sure to make it executable, I'll give it a shot tonight and try
> it out.
> 
> Thanks for the example script and advice on the modprobe -f and rmmod
> differences, really appreciate it.  Am looking forward however, to a kernel fix
> for this annoying bug :D  

+1 a very annoying bug. The workaround works great as a stopgap measure though. Thanks

Just wanted to chime in to say, that the latest F11 32bit PAE kernel (I dunno if to call it i386 or i586 anymore, most definitely isn't i686 I believe)... pushed to updates, has apparently NOT resolved the failed to resume problem.  And as a matter of fact I believe it has introduced a major glitch with my audio that prevents smooth uninterrupted audio playback from Amarok and flash video playback from within Firefox 3.5.  This is only worse :(  I might just have to remove the current kernel reverting back to the old one and blacklist the current kernel or something...cause this is even worse.  I can live without working suspend and resume, not the end of the world, but sound that crashes some apps or otherwise is unstable is just unacceptable.

Reilly I can confirm the sound issues with the latest kernel. Glitches and loosing volume controls. Let's not mention it again in this bug though ;)

No problem...I just mentioned it mostly to let everyone know that this bug was apparently not resolved in this kernel from what I could see, and that the possible regressions mentioned earlier in this bug (comment #5 by Matthias Runge) appeared to be true and did make it through to a released updated kernel...

I removed the workaround and updated to the 2.6.30.5 that was pushed to f11, and the issue appears to be fixed.

can others confirm this fix?

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

(In reply to comment #26)
> can others confirm this fix?

I've updated to the errata kernel (2.6.30.5-43.fc11), and suspending/resuming works now.  (CPU frequency scaling even works after suspending/resuming, as well.)

(I have a Sony TX650 with an Intel 1.2 GHz Pentium M)

Second oops from comment 3 still occur on suspend with kernel 2.6.30.5-43.fc11.


Hardware is Thinkpad Z60t

I actually saw that my Dell Latitude D410 started suspending a resuming without that workaround from a previous kernel, but it will only do this successfully once, then it still dies on subsequent suspends to ram or disk.  I don't call this permanently fixed.  However, the reason it crashes on the second time around, I honestly can't say its because of the cpu_freq thing, might be...have had no time to check.

I'll see if I can do some testing and checking of logs or whatever to see what I can find out.  Thanks guys.

Oh one other thing, I think the stuttering sound issue was also fixed in this kernel or a recent one...I have to do a more thorough stress test though. :)

2.6.30.5-43 has fixed the issue on my pentium-m
Cheers.

Created attachment 365770 [details]
kernel log output during oops and crash

As of kernel 2.6.30.8-64.fc11.i686.PAE I am still experiencing kernel oops and outright crashes when trying to resume from either suspend to ram and/or hibernate.  I've attached the relevant output hoping this helps in tracking down what this long standing issue is.

As of kernel 2.6.31.5-127.fc12.i686.PAE I have no more issues in that area.

I know it doesn't matter any more now that F12 is out, but with the latest F11 i686-PAE kernel on a Dell Latitude-D410:

Nov 18 19:05:40 Latitude-D410 kernel: ------------[ cut here ]------------      
Nov 18 19:05:40 Latitude-D410 kernel: WARNING: at kernel/smp.c:289 smp_call_function_single+0x54/0xe5() (Not tainted)                                           
Nov 18 19:05:40 Latitude-D410 kernel: Hardware name: Latitude D410                                                                                              
Nov 18 19:05:40 Latitude-D410 kernel: Modules linked in: fuse rfcomm bridge stp llc bnep sco l2cap irnet ppp_generic slhc irtty_sir sir_dev ircomm_tty ircomm sunrpc ip6t_REJECT ip6t_ipv6header nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq dm_multipath uinput snd_intel8x0 snd_intel8x0m snd_ac97_codec ac97_bus snd_pcm snd_timer yenta_socket snd rsrc_nonstatic iTCO_wdt iTCO_vendor_support soundcore ipw2200 libipw btusb bluetooth snd_page_alloc tg3 dell_laptop lib80211 irda rfkill crc_ccitt joydev pcspkr dcdbas ata_generic pata_acpi xts gf128mul aes_i586 aes_generic dm_crypt i915 drm i2c_algo_bit video output i2c_core [last unloaded: scsi_wait_scan]                                 
Nov 18 19:05:40 Latitude-D410 kernel: Pid: 2490, comm: pm-suspend Not tainted 2.6.30.9-96.fc11.i686.PAE #1                                                      
Nov 18 19:05:40 Latitude-D410 kernel: Call Trace:                               
Nov 18 19:05:40 Latitude-D410 kernel: [<c043564e>] warn_slowpath_common+0x70/0x87                                                                               
Nov 18 19:05:40 Latitude-D410 kernel: [<c045444e>] ? smp_call_function_single+0x54/0xe5                                                                         
Nov 18 19:05:40 Latitude-D410 kernel: [<f994f4da>] ? do_drv_read+0x0/0x41 [acpi_cpufreq]                                                                        
Nov 18 19:05:40 Latitude-D410 kernel: [<c0435677>] warn_slowpath_null+0x12/0x15 
Nov 18 19:05:40 Latitude-D410 kernel: [<c045444e>] smp_call_function_single+0x54/0xe5                                                                           
Nov 18 19:05:40 Latitude-D410 kernel: [<f994f2e3>] get_cur_val+0xa1/0xc6 [acpi_cpufreq]                                                                         
Nov 18 19:05:40 Latitude-D410 kernel: [<f994f364>] get_cur_freq_on_cpu+0x5c/0x9e [acpi_cpufreq]                                                                 
Nov 18 19:05:40 Latitude-D410 kernel: [<c0694a69>] ? cpufreq_cpu_get+0x78/0x9b  
Nov 18 19:05:40 Latitude-D410 kernel: [<c0694c55>] cpufreq_suspend+0xa7/0x11d   
Nov 18 19:05:40 Latitude-D410 kernel: [<c041820c>] ? apic_read+0x14/0x16        
Nov 18 19:05:40 Latitude-D410 kernel: [<c06062d6>] sysdev_suspend+0xd1/0x2b4    
Nov 18 19:05:40 Latitude-D410 kernel: [<c04583f9>] suspend_devices_and_enter+0xef/0x184                                                                         
Nov 18 19:05:40 Latitude-D410 kernel: [<c04585fa>] enter_state+0x142/0x19d      
Nov 18 19:05:40 Latitude-D410 kernel: [<c04586ed>] state_store+0x98/0xac        
Nov 18 19:05:40 Latitude-D410 kernel: [<c0458655>] ? state_store+0x0/0xac       
Nov 18 19:05:40 Latitude-D410 kernel: [<c05710ad>] kobj_attr_store+0x16/0x22    
Nov 18 19:05:40 Latitude-D410 kernel: [<c04f0f03>] sysfs_write_file+0xc1/0xec   
Nov 18 19:05:40 Latitude-D410 kernel: [<c04f0e42>] ? sysfs_write_file+0x0/0xec  
Nov 18 19:05:40 Latitude-D410 kernel: [<c04b2c27>] vfs_write+0x85/0xe4          
Nov 18 19:05:40 Latitude-D410 kernel: [<c04b2d24>] sys_write+0x40/0x62          
Nov 18 19:05:40 Latitude-D410 kernel: [<c0408474>] sysenter_do_call+0x12/0x28   
Nov 18 19:05:40 Latitude-D410 kernel: ---[ end trace b35877c0acf96684 ]---

Just an FYI, this hasn't manifested itself ever since I upgrade to F12 i686 on the same laptop.  Don't know what we'd want to do with this bug now, as I don't have any more running F11 systems (just one F10 which is overdue to be upgraded to F12 as it is).

This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.