Bug 620955
Summary: | Fedora 12 kernels since 2.6.32.16-141 cause performance problems | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | John Fitzgibbon <john_fitzgibbon> |
Component: | kernel | Assignee: | Kernel Maintainer List <kernel-maint> |
Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | medium | Docs Contact: | |
Priority: | low | ||
Version: | 12 | CC: | anton, dougsland, gansalmon, itamar, jonathan, kernel-maint, madhu.chinakonda, Marc.Herbert+rhzilla, roebela |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | kernel-2.6.32.19-163.fc12 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2010-08-23 22:02:03 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
John Fitzgibbon
2010-08-03 20:39:17 UTC
Can you run oprofile and see where the kernel is using that time? http://docs.fedoraproject.org/en-US/Fedora/12/html/Deployment_Guide/ch-oprofile.html (In reply to comment #1) > Can you run oprofile and see where the kernel is using that time? > > http://docs.fedoraproject.org/en-US/Fedora/12/html/Deployment_Guide/ch-oprofile.html Would like to help, but it'll be a while before I've time to RTFM... any chance you could help me out with a quick cheat sheet on what *exactly* you'd like me to run/collect/post? You need the kernel-debuginfo and oprofile packages installed to profile the kernel. Load the kernel module and start the daemon: opcontrol --init Run a profiling session: opcontrol --reset opcontrol --start --vmlinux=/usr/lib/debug/lib/modules/$(uname -r)/vmlinux [ run workload ] opcontrol --stop opreport --threshold 1 -l /usr/lib/debug/lib/modules/$(uname -r)/vmlinux Stop the daemon when done: opcontrol --shutdown Chuck, thanks for the mini-howto. I profiled one good run of the tests for comparison, then happened to catch a particularly bad test run after messing about in firefox for a bit, (though I exited firefox before the test run). Good test run, (elapsed = 33 seconds): # opreport --threshold 1 -l /usr/lib/debug/lib/modules/$(uname -r)/vmlinux Overflow stats not available CPU: Pentium M (P6 core), speed 1866 MHz (estimated) Counted CPU_CLK_UNHALTED events (clocks processor is not halted, and not in a thermal trip) with a unit mask of 0x00 (No unit mask) count 100000 samples % symbol name 74472 20.7922 get_page_from_freelist 15532 4.3365 read_hpet 14791 4.1296 page_fault 8733 2.4382 mls_compute_sid 7680 2.1442 system_call 6429 1.7949 __might_sleep 6363 1.7765 do_select 6217 1.7358 list_del 4737 1.3225 mem_cgroup_charge_common 4708 1.3144 copy_user_highpage 4541 1.2678 copy_from_user 3807 1.0629 handle_mm_fault Bad test run, (elapsed = 1450 seconds): # opreport --threshold 1 -l /usr/lib/debug/lib/modules/$(uname -r)/vmlinux Overflow stats not available CPU: Pentium M (P6 core), speed 1866 MHz (estimated) Counted CPU_CLK_UNHALTED events (clocks processor is not halted, and not in a thermal trip) with a unit mask of 0x00 (No unit mask) count 100000 samples % symbol name 747550 4.4549 do_select 588017 3.5042 get_page_from_freelist 464146 2.7660 __might_sleep 391223 2.3315 read_hpet 326714 1.9470 sched_clock_local 257557 1.5349 handle_mm_fault 251698 1.5000 softlockup_tick 243364 1.4503 core_sys_select 238071 1.4188 do_page_fault 233153 1.3895 test_ti_thread_flag 217087 1.2937 syscall_trace_enter 212461 1.2661 unmap_vmas 209578 1.2490 ktime_get 205824 1.2266 hrtimer_interrupt 201451 1.2005 restore_nocheck 193820 1.1550 __rcu_pending 186830 1.1134 kmap_atomic_prot 185022 1.1026 rb_get_reader_page 183781 1.0952 page_address 177655 1.0587 update_curr 174139 1.0378 copy_from_user FYI: Looking through other bugs, I came across bug #619396 -- sounds like a very similar problem, (slow performance, in X in that bug), with the same kernels, (2.6.32.16-141 and later), on the same h/w, (Thinkpad T43). Maybe something specific to the h/w, (graphics, maybe)? Can you add "nomodeset" to the kernel boot options and boot to runlevel 3 to run your tests and see if the problem goes away? (In reply to comment #5) > FYI: Looking through other bugs, I came across bug #619396 -- sounds like a > very similar problem, (slow performance, in X in that bug), with the same > kernels, (2.6.32.16-141 and later), on the same h/w, (Thinkpad T43). Maybe > something specific to the h/w, (graphics, maybe)? That problem only happens after a suspend/resume cycle. (In reply to comment #6) > Can you add "nomodeset" to the kernel boot options and boot to runlevel 3 to > run your tests and see if the problem goes away? Yes, left the tests running in a loop at runlevel 3 with nomodeset and performance is steady. I missed one step in the directions for profiling. You need to boot the machine with kernel option "idle=poll" to get accurate profiles. This will cause the machine to run hotter, so don't leave it that way. Also, are there a large number of interrupts happening? Check /proc/interrupts and see if anything looks unusual. It looks like I dropped some DRM fixes in 2.6.34.16-141 . Can you try 2.6.32.18-158.rc1.fc12 when it finishes building in koji? (In reply to comment #10) > It looks like I dropped some DRM fixes in 2.6.34.16-141 . Can you try > 2.6.32.18-158.rc1.fc12 when it finishes building in koji? I'll need to test for a day or two to be 100% sure, but that seems to do the trick. Thanks. (In reply to comment #11) > I'll need to test for a day or two to be 100% sure. Fix looks good. Feel free to close. Thanks again. kernel-2.6.32.19-162.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/kernel-2.6.32.19-162.fc12 kernel-2.6.32.19-163.fc12 has been submitted as an update for Fedora 12. http://admin.fedoraproject.org/updates/kernel-2.6.32.19-163.fc12 *** Bug 625215 has been marked as a duplicate of this bug. *** kernel-2.6.32.19-163.fc12 has been pushed to the Fedora 12 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with su -c 'yum --enablerepo=updates-testing update kernel'. You can provide feedback for this update here: http://admin.fedoraproject.org/updates/kernel-2.6.32.19-163.fc12 (In reply to comment #16) > If you want to test the update, you can install it with > su -c 'yum --enablerepo=updates-testing update kernel'. Done:- no problems with this version either. > You can provide feedback for this update here: > http://admin.fedoraproject.org/updates/kernel-2.6.32.19-163.fc12 Done. kernel-2.6.32.19-163.fc12 has been pushed to the Fedora 12 stable repository. If problems still persist, please make note of it in this bug report. |