Bug 503038

Summary: Dell 1855: ipmitool sensors stopped reporting temperatures after upgrade to OpenIPMI-1.4.14-1.4E.25 and kernel-smp-2.6.9-89.ELsmp
Product: Red Hat Enterprise Linux 4 Reporter: Ingvar Hagelund <ingvar>
Component: OpenIPMIAssignee: Jan Safranek <jsafrane>
Status: CLOSED CURRENTRELEASE QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: high    
Version: 4.9CC: azelinka, jwest, mgoodwin, moshiro, rvokal, syeghiay, tao
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 636854 (view as bug list) Environment:
Last Closed: 2012-06-14 20:54:31 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 485811, 634972, 636854    
Attachments:
Description Flags
zipped params, as requested none

Description Ingvar Hagelund 2009-05-28 13:16:03 UTC 
Description of problem:

On Dell 1855 blades, after upgrading from kernel-smp-2.6.9-78.0.22.EL to kernel-smp-2.6.9-89.EL, and OpenIPMI-1.4.14-1.4E.20 to OpenIPMI-1.4.14-1.4E.25, ipmitool stopped reporting temperatures.


Version-Release number of selected component (if applicable):

kernel-smp-2.6.9-89.EL
OpenIPMI-1.4.14-1.4E.25


How reproducible:

Always


Steps to Reproduce:

1. On a Dell 1855 running 2.6.9-78.0.22.ELsmp and OpenIPMI-1.4.14-1.4E.20, 'ipmitool sensors' reports temperature:

# ipmitool sensor | grep degrees
Temp             | na         | degrees C  | na    | na        | na        | na        | 85.000    | 90.000    | na        
Temp             | na         | degrees C  | na    | na        | na        | na        | 85.000    | 90.000    | na        
Mem Temp 1       | 26.000     | degrees C  | ok    | na        | 5.000     | 10.000    | 75.000    | 80.000    | na        
Mem Temp 2       | 30.000     | degrees C  | ok    | na        | 5.000     | 10.000    | 75.000    | 80.000    | na        

2. Upgrade all errata including rebooting to latest kernel 2.6.9-89.ELsmp. Run 'ipmitool sensor' again.

3. 'ipmitool sensor' does no longer report temperatures
  

Actual results:

# ipmitool sensor | grep degrees
CPU1 Temp        | na         | degrees C  | na    | na        | na        | na        | na        | na        | na        
CPU2 Temp        | na         | degrees C  | na    | na        | na        | na        | na        | na        | na        
Mem Temp 1       | na         | degrees C  | na    | na        | na        | na        | na        | na        | na        
Mem Temp 2       | na         | degrees C  | na    | na        | na        | na        | na        | na        | na        


Expected results:

ipmitool should report sensor temperatures


Additional info:

At first glance, other ipmitool functionality seems to work as expected.
Also tested on Dell 1955 blades. Not reproducable there.

These blades are in production, so it's a bit awkward to do more testing than this. I have a couple of 1855 blades that are not yet upgraded to rhel-4.9, so I can run a few commands on those if you respond quickly :-)
Comment 1 Jan Safranek 2009-06-01 14:08:16 UTC 
Please note that Bugzilla is not support tool, visit redhat.com/support to get the service you pay for.


However, I reproduced the bug on local Dell 1855 system. It's caused by this change in source files:

http://ipmitool.cvs.sourceforge.net/viewvc/ipmitool/ipmitool/lib/ipmi_sensor.c?r1=1.32&r2=1.33

I have not found any specific info why the code was changed, I'll dig more when I have more time...
Comment 2 Jan Safranek 2009-06-02 07:13:50 UTC 
ipmitool upstream is aware of the problem, but there is no solution yet:

http://sourceforge.net/mailarchive/forum.php?thread_name=C2866F9FC4CB034EB51A633DF16859860553CE1E%40ssbarcelone.teknor.com&forum_name=ipmitool-devel

It seems to be weirdness of some HW, the BMC reports a sensor is owned by particular slave board (blade?), but it does not bridge sensor reading commands to it.

As a workaround, you can try to query detailed information about sensors using '-vv' option to get address of the sensor owner:

# ipmitool -vv sensor list
...
...
...
Sending request to IPMB target @ 0xc0
Error reading event status for sensor #07: Invalid command

'0xc0' is the address you are interested in. And then ask directly appropriate owner using '-m' and '-t' options:

# ipmitool -m 0xc0 -t 0xc0 sensor list
CPU1 Temp        | 36.000     | degrees C  | ok    | na        | 5.000     | 10.000    | 120.000   | 125.000   | na        
Daughter Card    | na         | discrete   | na    | na        | na        | na        | na        | na        | na        
CPU2 Temp        | 35.000     | degrees C  | ok    | na        | 5.000     | 10.000    | 120.000   | 125.000   | na        
...
Comment 5 Ingvar Hagelund 2009-06-02 08:57:55 UTC 
Except the fact that the address to the sensor varies among the blades, and has to be hardcoded for each blade, this workaround works well.

Ingvar
Comment 6 Ingvar Hagelund 2009-06-03 07:37:15 UTC 
Semi-related: Same hardware, same upgrade: The output of
 ipmitool chassis status
has changed. A minor change, but still interesting:

--- /var/tmp/chassis-status.ref 2007-02-07 22:02:08.000000000 +0100
+++ /var/tmp/chassis-status	2009-06-03 08:17:01.000000000 +0200
@@ -16,4 +16,4 @@
 Sleep Button Disabled: false
 Diag Button Disabled : false
 Reset Button Disabled: false
-Power Button Disabled: true
+Power Button Disabled: false
Comment 7 Jan Safranek 2009-06-03 15:03:52 UTC 
(In reply to comment #6)
> Semi-related: Same hardware, same upgrade: The output of
>  ipmitool chassis status
> has changed. A minor change, but still interesting:
> 
> -Power Button Disabled: true
> +Power Button Disabled: false  

This is actually a bugfix, see http://sourceforge.net/mailarchive/message.php?msg_name=!%26!AAAAAAAAAAAYAAAAAAAAAPEZLKEHAu5BjaF%2FT5KKLfzCgAAAEAAAAHT6lwjpWYNFv%2BfM9eo9CXgBAAAAAA%3D%3D%40bull.net
Comment 8 Jan Safranek 2009-10-19 11:00:37 UTC 
*** Bug 526409 has been marked as a duplicate of this bug. ***
Comment 15 Mark Goodwin 2010-02-09 03:16:21 UTC 
Created attachment 389658 [details]
zipped params, as requested