Bug 468802

Summary: (shuttle m/b + e8400 cpu) Lack of CPU Frequency Scaling Support.
Product: [Fedora] Fedora Reporter: Maciej Żenczykowski <zenczykowski>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED CANTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 14CC: dwmw2, elad, gansalmon, itamar, jmedefind, jonathan, kernel-maint, madhu.chinakonda, petr.tuma, zenczykowski
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-08-26 13:10:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
acpi dump from affected machine none

Description Maciej Żenczykowski 2008-10-28 04:51:07 UTC
Kernel version 2.6.26.7-86.fc9.x86_64 (from koji, and various previous fc9 kernels from koji, from install and from updates) - untested with non-64 bit kernels.

How reproducible: Always

Steps to Reproduce:

1. Shuttle SG33G5M Deluxe 'Box' (w/ motherboard) with Intel Core 2 Duo CPU (E8400)

http://us.shuttle.com/barebone/Models/sg33g5m_deluxe.html

with latest BIOS (SG33SM17) from http://global.shuttle.com/download03.jsp?PI=784

2. EIST enabled in BIOS and explicitly reported as such during POST.

3. acpi-cpufreq module fails to load
  
Actual results:

# uname -a
Linux zeus.lan 2.6.26.7-86.fc9.x86_64 #1 SMP Wed Oct 22 22:54:13 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

# cat /proc/cpuinfo 
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
stepping        : 6
cpu MHz         : 3002.916
cache size      : 6144 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 10
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm
bogomips        : 6009.16
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

(repeat for second core)

# modprobe acpi-cpufreq
FATAL: Error inserting acpi_cpufreq (/lib/modules/2.6.26.7-86.fc9.x86_64/kernel/arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.ko): No such device

(nothing in dmesg)

# ( cd /lib/modules/2.6.26.7-86.fc9.x86_64; find | egrep cpufreq; )
./kernel/arch/x86/kernel/cpu/cpufreq
./kernel/arch/x86/kernel/cpu/cpufreq/acpi-cpufreq.ko
./kernel/arch/x86/kernel/cpu/cpufreq/powernow-k8.ko
./kernel/drivers/cpufreq
./kernel/drivers/cpufreq/cpufreq_ondemand.ko
./kernel/drivers/cpufreq/cpufreq_conservative.ko
./kernel/drivers/cpufreq/cpufreq_stats.ko
./kernel/drivers/cpufreq/cpufreq_powersave.ko
./kernel/drivers/cpufreq/freq_table.ko

Expected results:

The module should load, or other modules (speedstep-centrino? p4-clockmod?) should be present for fallback.

Additional info:

This works fine on my macbook pro 3,1 laptop, so I'm guessing the problem is some sort of BIOS issue (corrupt/missing ACPI tables or something).  However I'm at a loss as to what kernel command line parameters to pass if any, and how to go about debugging/fixing this...

Comment 1 Maciej Żenczykowski 2008-11-11 07:57:04 UTC
Created attachment 323155 [details]
acpi dump from affected machine

Attaching acpidump from affected machine - from brief examination and comparison with another machine on which this works, this maybe missing ACPI sections.

# uname -a
Linux zeus.lan 2.6.27.4-19.fc9.x86_64 #1 SMP Thu Oct 30 19:30:01 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

Comment 2 Bug Zapper 2009-06-10 03:06:20 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 3 Maciej Żenczykowski 2009-06-13 06:57:18 UTC
I've upgraded the original machine to Fedora 10 and installed Fedora 11 on a second nearly identical machine.

Issue is present on:
* Shuttle SG33G5M Deluxe w/ Intel Core 2 Duo 3GHz [E8400] & Fedora 9/10
* Shuttle SG33G5M Deluxe w/ Intel Core 2 Quad 2.83GHz [Q9550S] & Fedora 10/11
* Shuttle SG33G5         w/ Intel Core 2 Duo 3GHz [E8400] & Fedora 11

Based on an analysis of the ACPI tables (lack of proper entries) this is all caused by a lack of proper BIOS support (running newest BIOS as of 2 weeks ago on both machines, there has been a BIOS update issued for one of them since then, but according to the notes it's AHCI related).  In general the quality of the BIOS is so-so, since the SG33G5M Deluxe also appears to slightly misroute IRQs 18 & 19 (working on fixing that in my copious spare time).

I can 'force' a manual voltage/frequency change by twiddling MSR 199 with wrmsr.  I haven't yet attempted to manually patch up the acpi tables, but it looks like this should be doable, if very hacky and inconvenient.

I think it would also be possible to get p4-clockmod to support these cpus, however my first attempts at a patch were not properly multi-core aware.

Comment 4 Petr Tuma 2010-03-21 16:25:25 UTC
Maciej, I have the same issue (Shuttle SG33G5 with BIOS 119 and E7400 CPU), seeing that your comments have been posted some time ago, I wanted to ask if you had any luck resolving this (or, if you would know whether the BIOS at least supports some older CPU properly). Thanks, Petr.

Comment 5 Maciej Żenczykowski 2010-03-22 22:51:14 UTC
I'll get back to you on this one, I was actually working on exactly this issue over the weekend (I finally got fed up with it once more and decided to work on getting the correct fix instead of the two workarounds I've been using: one for cpu speed, one for irqs being misrouted; and I believe I'm close).

*** STANDARD DISCLAIMER: NO GUARANTEE THIS WON'T FRY YOUR CPU, YOUR MOTHERBOARD OR YOUR BRAINS ***

That said I've been using this on the following 2 machines (cpu swap at one point): SG33G5M Deluxe + Intel Core 2 Quad Q9550S; SG33G5M Deluxe + Intel Core 2 Duo E8400; SG33G5 + Intel Core 2 Duo E8400 for way over a year with no issues, and I've inspected the configuration [both via RDMSR and via ACPI table dumps] of another couple machines [Macbook Pro 3,1 + 4,1 laptops, Desktop with Intel Core 2 Proc, machine with Intel Atom cpu, etc.], and it all adds up, further searching around on the internet really reinforces this opinion: however, only parts of this are actually documented by Intel in the publicly available docs, I'd assume more is available to BIOS writers under NDA.

That said, the further you move away from my CPUs, the more you risk it not working or working badly.  I would guess 'Intel Core 2' would be probably safe.  This is definitely Intel-only, and requires a relatively recent CPU (ie. with EIST [Intel Enhanced SpeedStep Technology] support, you can check the intel sSpec finder for that, ie. http://processorfinder.intel.com/details.aspx?sSpec=SLGAE for my Q9550S cpu).


Could you run:

sudo yum install msr-tools

which will install rdmsr/wrmsr command line utils, and then:

for MSR in 198 199 1A0; do for CPU in `sed -rn 's@^processor[\t]: ([0-9]+)$@\1@p' /proc/cpuinfo`; do echo -n "CPU${CPU} ${MSR}: "; sudo rdmsr -0 -X -p ${CPU} 0x${MSR}; done; done

which should output current EIST (Enhanced Intel SpeedStep) configuration.

Here's what I get:

CPU0 198: 0616481C06000616
CPU1 198: 0616481C06000616
CPU2 198: 0616481C06000616
CPU3 198: 0616481C06000616
CPU0 199: 0000000000000616
CPU1 199: 0000000000000616
CPU2 199: 0000000000000616
CPU3 199: 0000000000000616
CPU0 1A0: 0000004062972489
CPU1 1A0: 0000004062972489
CPU2 1A0: 0000004062972489
CPU3 1A0: 0000004062972489

You will note a quad core cpu, and that the configuration of all cores is currently identical.

The value in MSR 0x198 [PERF_STATE] can be interpreted as 8 bytes:

06 16 48 1C 06 00 06 16
== -- -- -- -- -- -- -- this is the minimum FID
-- == -- -- -- -- -- -- this is the minimum VID
-- -- == -- -- -- -- -- this is the maximum FID
-- -- -- == -- -- -- -- this is the maximum VID
-- -- -- -- == -- -- -- unsure
-- -- -- -- -- == -- -- unsure
-- -- -- -- -- -- == -- this is the current FID
-- -- -- -- -- -- -- == this is the current VID

The value in MSR 0x199 [PERF_CTL] is the currently desired FID+VID (most bits are just 0).

The value in MSR 0x1A0 [MISC_ENABLE] is a bitfield, bit 17 is EIST enable and thus should be 1.  00000040629'7'2489 -> bit 17 is the lowest bit of the '7', thus the '7' being an odd hex digit means EIST is enabled.

You can force a transition by writing a new value into MSR 0x199 and verifying the state transition by looking in MSR 0x198, and verifying the CPU Voltage changes using lm_sensors

$ sensors | egrep 'CPU:'
CPU:         +1.10 V  (min =  +1.07 V, max =  +1.23 V)   

[if you don't have lm_sensors configured correctly this may just show up as in0... I've posted my sensors.conf on the lm_sensors wiki, so you should be able to fetch it from there]

For example, "sudo wrmsr -p 0 0x199 0x0616" forces CPU0 to transition to FID=0x06 VID=0x16, while "sudo wrmsr -p 3 0x199 0x481C" forces CPU3 to transition to FID=0x48 VID=0x1C.

You will probably discover that transitioning some cpus pulls other cpus along for the ride.  For me CPUs 0 & 2 are tied, and CPUs 1 & 3 are tied.  They also share L2 cache.  (The Q9550S is a single socket, 4 core cpu, but internally it's two die, each die with it's own 2 cores and it's own shared 6MB of L2 cache).  For a Core 2 Duo, I would guess both cores share L2 cache and transition together.  I would expect that the frequency of the two dies on a two-die quad core can run at different frequencies, but I don't think the motherboard can support different voltages, so you probably end up running with the higher voltage.  Please note, that I haven't really done significant testing of transitioning just a single core, I've pretty much always transitioned all cores at once.

What is FID?
(all examples based on my Q9550S with default FSB of 333)
- small values of FID is just the frequency multiplier.
  ie. FID=0x06 -> 6 * FSB frequency = 6 * 333 MHz = 2000 MHz
  ie. FID=0x07 -> 7 * FSB = 2333 MHz
  ie. FID=0x08 -> 8 * FSB = 2666 MHz
- if the FID has the 0x40 bit set, then the multiplier is 0.5 higher
  ie. FID=0x47 -> 7.5 * FSB = 2500 MHz
  ie. FID=0x48 -> 8.5 * FSB = 2833 MHz
- I have also seen a FID=0x88 on a Macbook Pro, since this is a mobile CPU, and this is the minimum power state, this seems to imply dropping the FSB by some factor as well.  (because it's normally a 2400 MHz = 9 * 266 cpu (with FID=0x09 at full speed), which at FID=0x88 runs at 800 MHz, which implies it probably ends up being 8 * 100.  With the FID=0x88 the VID is also lower than the minimum VID you would otherwise expect.

What is VID?
- this appears to be a simple integer, lower means lower voltage, higher means higher voltage
- I've seen rumours of Voltage = 700 mV + 12.5 * VID for Core CPUs
  and 800 mV + 12.5 * VID for Core 2 CPUs
- the 800+12.5*VID seems to be reasonably close to what I see from sensors...
- at VID=0x16 I should thus see 800+12.5*0x16 = 800 + 12.5 * 22 = 1075 mV,
  what I see is around ~1.088 V under full load
- at VID=0x1C I should thus see 800+12.5*0x1C = 800 + 12.5 * 28 = 1150 mV,
  what I see is around ~1.152 under full load
[However variations without load are much greater, values of VIDs between 0x17..0x1B fit less correctly, and the resolution of the sensors utilities is merely 16 mV (ie. it never shows a value that doesn't divide evenly by 16mV)]

Now, you should now have enough information to switch your cpus to minimum FID/VID and maximum FID/VID.

It gets more rocky with values in between.

Thankfully it seems writing junk to 0x199 doesn't hurt anything... it just gets ignored and the CPU picks something sane.

However you can still pick lower or higher VIDs for a given FID, and the CPU won't complain.  Ie. you may end up under-volting or over-volting the CPU for a specific FID if you don't pick the right VID.  Thankfully, the CPU has to be capable of dealing with the maximum VID anyway, since the transition from a high FID/VID to a low FID/VID is explicitly known to first lower the FID, and only then lower the VID (with a transition from low to high, first increasing VID, and only afterwards increasing the FID).  The reason for this is to maintain CPU stability - you do not want to run the CPU under-volted - it might make computational mistakes, while running over-volted merely burns more power.  While it is true that extreme over-volting could burn the CPU out; remember: we're keeping within the bounds the CPU is giving us, so we're never going beyond what the CPU is willing to deal with anyway - this does assume you don't have some over-volting configured in the BIOS in order to overclock.

I will point out that at this point we're getting even more fuzzy, than we've been up to here...  Before now we weren't really guessing... now we're going to start guessing.

First of all, it seems you can't get a FID of 0x46 (ie. 6.5) - it just doesn't work (please verify).  Some CPUs may not support 0.5 multipliers at all.
Since my min is FID/VID 06/16 and max is FID/VID 48/1C.
I thus potentially have FIDs: 06 07 47 08 48 and VIDs 16 17 18 19 1A 1B 1C.
I have 5 FIDs, 7 VIDs - so they almost match up.
Furthermore since I skipped FID 46, I kind of have 6 FIDs.
Since undervolting is bad for stability, I decided to skip the extra FID as early as possible, thus I get
FID=06 VID=16 minimum 2000 MHz state
       VID=17 skip extra VID, better earlier than later
FID=46 VID=18 skip because of bad FID
FID=07 VID=19 2333 MHz
FID=47 VID=1A 2500 MHz
FID=08 VID=1B 2666 MHz
FID=48 VID=1C maximum 2833 MHz state.

This suggests that my values are 0x0616, 0x4619, 0x471A, 0x081B, 0x481C.
And they do all indeed work, with voltage measurements according to sensors which roughly match up.

Obviously I'm really lucky here... Q9550"S" is a low-power cpu, as a result of which the maximum vid (1C) is very low.  [I've heard rumours this is actually an Intel Core 2 Quad Extreme CPU from the binning process which has had it's maximum FID/VID locked to a lower value]

Since the maximum VID is low, there are few VIDs to choose from and it's hard to go wrong...

If you have few potential FIDs, but many many VIDs you probably want to roughly deal them out evenly, remembering to skip any additional VIDs earlier rather than later.  For example with FIDs 06 07 47 08 and VIDs 16..22 I would probably pick 0616 071C 471F 0822.

Ultimately I don't really think there's much danger of permanent damage here... you'll either overvolt, and burn a little too much power, or undervolt and have an unstable system.  You can test for stability by forcing the CPU to a specific FID/VID and running mprime benchmark torture test for 12 hours.

Okay, enough for today, next installment, will hopefully include information on how to fix the ACPI tables to get this to work via the in-kernel drivers.

Comment 6 Bug Zapper 2010-04-27 12:18:00 UTC
This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 7 Maciej Żenczykowski 2010-04-28 03:39:05 UTC
As this is pretty much a BIOS bug, it is indeed very much so present in F12, and will likely continue to be present in F13.  However, good news is that I now have a patched BIOS which appears to make frequency scaling work (still non final form though).

Comment 8 jmedefind 2010-08-23 01:50:35 UTC
(In reply to comment #7)
> As this is pretty much a BIOS bug, it is indeed very much so present in F12,
> and will likely continue to be present in F13.  However, good news is that I
> now have a patched BIOS which appears to make frequency scaling work (still non
> final form though).

Maciej,

Do you have any plans of releasing your patched BIOS?

Thanks,
Jonathan

Comment 9 Bug Zapper 2010-11-04 11:43:55 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 10 Maciej Żenczykowski 2010-12-04 06:20:13 UTC
I should really document how to frickin' fix this.

Comment 11 Elad Alfassa 2011-05-04 18:01:05 UTC
Moving to kernel (was 0xFFFF)



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 12 Elad Alfassa 2011-05-04 18:02:53 UTC
Opps, I forgot to actually select kernel in the dropdown list. sorry about that.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Comment 13 Josh Boyer 2011-08-26 13:10:02 UTC
This is a rather old bug and it seems to have been resolved as a BIOS issue.  Closing out.