Bug 1447454
| Summary: | getVdsHardwareInfo fails with IndexError | ||
|---|---|---|---|
| Product: | [oVirt] vdsm | Reporter: | Jillian Morgan <penguin.wrangler> |
| Component: | SuperVDSM | Assignee: | Yaniv Bronhaim <ybronhei> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Lukas Svaty <lsvaty> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 4.19.10 | CC: | bugs, igoihman, lsvaty, oourfali, penguin.wrangler, stirabos, yanqzhan |
| Target Milestone: | ovirt-4.1.3 | Flags: | rule-engine:
ovirt-4.1+
lsvaty: testing_ack+ |
| Target Release: | 4.19.17 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-07-06 13:09:14 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Irit, can you please take a look? Hi Ian,Thanks for the report and the detailed suggestion. You posted vdsm-python-4.19.10.1-1.fc24.noarch - so I understand you're using ovirt-4.1 cluster over fedora 24 Lets figure the cause of the issue before catching all IndexErrors that can exist in this code. If the code is wrong , we should fix it. If dmidecode output was changed, we need to be aligned. VDSM uses this output to report it, and to define its uuid. So its quite important not to ignore it if something changes. Please paste "dmidecode" output that you get over this machine. it never happened on my f24 env so we need to see what field is missing and why Though I must say that you're quite right about reading without using - those parts can be dropped from the utility 'bios', 'cache', 'processor', 'chassis', 'memory' but before dropping this code (that probably will also fix the issue presented here) lets understand what changed in dmidecode or is different in your env that caused that glitch Well this isn't a new problem. This started happening way back after upgrading the hosts from Fedora 22 to Fedora 23, and has been present ever since. So far it has only been a log nuisance, but now while trying to migrate from standalone to hosted engine, the error is blocking the deployment process, so this finally inspired me to debug the problem.
I produced a simple 2-liner to run with 'python -m pdb' to trace the exact point where it fails:
from dmidecodeUtil import getHardwareInfoStructure
x=getHardwareInfoStructure()
But the result makes no sense to me. Upon "step"-ing line by line through the functions, and seeing that it successfully handles 'system', 'bios', 'cache', 'processor', and 'chassis' dictionaries, it finally fails while processing the 'memory' dictionary.
The exact failure is:
Traceback (most recent call last):
File "/usr/lib64/python2.7/pdb.py", line 1314, in main
pdb._runscript(mainpyfile)
File "/usr/lib64/python2.7/pdb.py", line 1233, in _runscript
self.run(statement)
File "/usr/lib64/python2.7/bdb.py", line 400, in run
exec cmd in globals, locals
File "<string>", line 1, in <module>
File "test.py", line 2, in <module>
x=getHardwareInfoStructure()
File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 488, in __call__
value = self.func(*args)
File "dmidecodeUtil.py", line 54, in getHardwareInfoStructure
dmiInfo = getAllDmidecodeInfo()
File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 488, in __call__
value = self.func(*args)
File "dmidecodeUtil.py", line 48, in getAllDmidecodeInfo
myLeafDict[k] = __leafDict(getattr(dmidecode, k)())
File "dmidecodeUtil.py", line 35, in __leafDict
ret.update(__leafDict(v))
File "dmidecodeUtil.py", line 35, in __leafDict
ret.update(__leafDict(v))
File "dmidecodeUtil.py", line 33, in __leafDict
for k, v in d.iteritems():
IndexError: list assignment index out of range
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py(33)__leafDict()
-> for k, v in d.iteritems():
(Pdb) pp d
{'Array Handle': '0x002b',
'AssetTag': 'FBANK 0 DIMMB2 AssetTag',
'Bank Locator': 'BANK 0',
'Data Width': '64 bit',
'Error Information Handle': 'No Error',
'Form Factor': 'SODIMM',
'Locator': 'DIMMB2',
'Manufacturer': 'Toshiba',
'Part Number': '9965527-021.A00LF',
'Serial Number': '66340727',
'Set': None,
'Size': '8192 MB',
'Speed': '1600 MHz (ns)',
'Total Width': '64 bit',
'Type': 'DDR3FBD2',
'Type Detail': [None,
None,
None,
None,
None,
None,
'Synchronous',
None,
None,
None,
None,
None]}
(Pdb) pp k
'Array Handle'
(Pdb) pp v
'0x002b'
However, if I create another simple script which takes that exact dictionary d as above, and call __leafdict(d) on it, it returns successfully (though the returned result dictionary is kinda useless due to the way is collapses all the nested memory info down to a single-level key-value list so most of the information is lost). I can't figure out why it is failing when used in superVdsm, but works in my simple test script with the same input.
For reference, here is the full output of "dmidecode" (dmidecode-3.0-6.fc24.x86_64) on one of the affected systems:
[root@bang ~]# dmidecode
# dmidecode 3.0
Getting SMBIOS data from sysfs.
SMBIOS 2.8 present.
53 structures occupying 2189 bytes.
Table at 0x7F4D8000.
Handle 0x0000, DMI type 0, 24 bytes
BIOS Information
Vendor: American Megatrends Inc.
Version: 1.1a
Release Date: 08/27/2015
Address: 0xF0000
Runtime Size: 64 kB
ROM Size: 6144 kB
Characteristics:
PCI is supported
BIOS is upgradeable
BIOS shadowing is allowed
Boot from CD is supported
Selectable boot is supported
BIOS ROM is socketed
EDD is supported
5.25"/1.2 MB floppy services are supported (int 13h)
3.5"/720 kB floppy services are supported (int 13h)
3.5"/2.88 MB floppy services are supported (int 13h)
Print screen service is supported (int 5h)
8042 keyboard services are supported (int 9h)
Serial services are supported (int 14h)
Printer services are supported (int 17h)
ACPI is supported
USB legacy is supported
BIOS boot specification is supported
Targeted content distribution is supported
UEFI is supported
BIOS Revision: 5.6
Handle 0x0001, DMI type 1, 27 bytes
System Information
Manufacturer: Supermicro
Product Name: A1SAi
Version: 123456789
Serial Number: 123456789
UUID: Not Settable
Wake-up Type: Power Switch
SKU Number: 081315D9
Family: SMC X10
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: Supermicro
Product Name: A1SAi
Version: 123456789
Serial Number: NM16BS005323
Asset Tag: To be filled by O.E.M.
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: To be filled by O.E.M.
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0
Handle 0x0003, DMI type 3, 25 bytes
Chassis Information
Manufacturer: To Be Filled By O.E.M.
Type: Expansion Chassis
Lock: Not Present
Version: To Be Filled By O.E.M.
Serial Number: To Be Filled By O.E.M.
Asset Tag: To Be Filled By O.E.M.
Boot-up State: Safe
Power Supply State: Safe
Thermal State: Safe
Security Status: None
OEM Information: 0x00000000
Height: Unspecified
Number Of Power Cords: 1
Contained Elements: 1
Power Supply (1)
SKU Number: To be filled by O.E.M.
Handle 0x0004, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1A1
Internal Connector Type: None
External Reference Designator: PS2Mouse
External Connector Type: PS/2
Port Type: Mouse Port
Handle 0x0005, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1A1
Internal Connector Type: None
External Reference Designator: Keyboard
External Connector Type: PS/2
Port Type: Keyboard Port
Handle 0x0006, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A1
Internal Connector Type: None
External Reference Designator: TV Out
External Connector Type: Mini Centronics Type-14
Port Type: Other
Handle 0x0007, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A2A
Internal Connector Type: None
External Reference Designator: COM A
External Connector Type: DB-9 male
Port Type: Serial Port 16550A Compatible
Handle 0x0008, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2A2B
Internal Connector Type: None
External Reference Designator: Video
External Connector Type: DB-15 female
Port Type: Video Port
Handle 0x0009, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J3A1
Internal Connector Type: None
External Reference Designator: USB1
External Connector Type: Access Bus (USB)
Port Type: USB
Handle 0x000A, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J3A1
Internal Connector Type: None
External Reference Designator: USB2
External Connector Type: Access Bus (USB)
Port Type: USB
Handle 0x000B, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J3A1
Internal Connector Type: None
External Reference Designator: USB3
External Connector Type: Access Bus (USB)
Port Type: USB
Handle 0x000C, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9A1 - TPM HDR
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x000D, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9C1 - PCIE DOCKING CONN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x000E, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2B3 - CPU FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x000F, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6C2 - EXT HDMI
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0010, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J3C1 - GMCH FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0011, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1D1 - ITP
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0012, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9E2 - MDC INTPSR
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0013, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9E4 - MDC INTPSR
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0014, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9E3 - LPC HOT DOCKING
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0015, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9E1 - SCAN MATRIX
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0016, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J9G1 - LPC SIDE BAND
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0017, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J8F1 - UNIFIED
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0018, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J6F1 - LVDS
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x0019, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2F1 - LAI FAN
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x001A, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J2G1 - GFX VID
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x001B, DMI type 8, 9 bytes
Port Connector Information
Internal Reference Designator: J1G6 - AC JACK
Internal Connector Type: Other
External Reference Designator: Not Specified
External Connector Type: None
Port Type: Other
Handle 0x001C, DMI type 9, 17 bytes
System Slot Information
Designation: Slot1 PCI-E 2.0 X8
Type: x8 PCI Express
Current Usage: In Use
Length: Long
ID: 0
Characteristics:
3.3 V is provided
Opening is shared
PME signal is supported
Bus Address: 0000:00:03.0
Handle 0x001D, DMI type 10, 6 bytes
On Board Device Information
Type: Video
Status: Enabled
Description: To Be Filled By O.E.M.
Handle 0x001E, DMI type 11, 5 bytes
OEM Strings
String 1: To Be Filled By O.E.M.
Handle 0x001F, DMI type 12, 5 bytes
System Configuration Options
Option 1: To Be Filled By O.E.M.
Handle 0x0020, DMI type 32, 20 bytes
System Boot Information
Status: No errors detected
Handle 0x0021, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard IGD
Type: Video
Status: Enabled
Type Instance: 1
Bus Address: 0000:00:02.0
Handle 0x0022, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard LAN
Type: Ethernet
Status: Enabled
Type Instance: 1
Bus Address: 0000:00:19.0
Handle 0x0023, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard 1394
Type: Other
Status: Enabled
Type Instance: 1
Bus Address: 0000:03:1c.2
Handle 0x0024, DMI type 38, 18 bytes
IPMI Device Information
Interface Type: KCS (Keyboard Control Style)
Specification Version: 2.0
I2C Slave Address: 0x10
NV Storage Device: Not Present
Base Address: 0x0000000000000CA2 (I/O)
Register Spacing: Successive Byte Boundaries
Handle 0x0025, DMI type 7, 19 bytes
Cache Information
Socket Designation: L1-Cache
Configuration: Enabled, Not Socketed, Level 1
Operational Mode: Write Back
Location: Internal
Installed Size: 448 kB
Maximum Size: 448 kB
Supported SRAM Types:
Synchronous
Installed SRAM Type: Synchronous
Speed: Unknown
Error Correction Type: Single-bit ECC
System Type: Instruction
Associativity: 8-way Set-associative
Handle 0x0026, DMI type 7, 19 bytes
Cache Information
Socket Designation: L2-Cache
Configuration: Enabled, Not Socketed, Level 2
Operational Mode: Write Back
Location: Internal
Installed Size: 4096 kB
Maximum Size: 4096 kB
Supported SRAM Types:
Synchronous
Installed SRAM Type: Synchronous
Speed: Unknown
Error Correction Type: Single-bit ECC
System Type: Unified
Associativity: 16-way Set-associative
Handle 0x0027, DMI type 4, 42 bytes
Processor Information
Socket Designation: CPU0
Type: Central Processor
Family: Atom
Manufacturer: Intel(R) Corporation
ID: D8 06 04 00 FF FB EB BF
Signature: Type 0, Family 6, Model 77, Stepping 8
Flags:
FPU (Floating-point unit on-chip)
VME (Virtual mode extension)
DE (Debugging extension)
PSE (Page size extension)
TSC (Time stamp counter)
MSR (Model specific registers)
PAE (Physical address extension)
MCE (Machine check exception)
CX8 (CMPXCHG8 instruction supported)
APIC (On-chip APIC hardware supported)
SEP (Fast system call)
MTRR (Memory type range registers)
PGE (Page global enable)
MCA (Machine check architecture)
CMOV (Conditional move instruction supported)
PAT (Page attribute table)
PSE-36 (36-bit page size extension)
CLFSH (CLFLUSH instruction supported)
DS (Debug store)
ACPI (ACPI supported)
MMX (MMX technology supported)
FXSR (FXSAVE and FXSTOR instructions supported)
SSE (Streaming SIMD extensions)
SSE2 (Streaming SIMD extensions 2)
SS (Self-snoop)
HTT (Multi-threading)
TM (Thermal monitor supported)
PBE (Pending break enabled)
Version: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz
Voltage: 1.6 V
External Clock: 100 MHz
Max Speed: 2600 MHz
Current Speed: 2400 MHz
Status: Populated, Enabled
Upgrade: Other
L1 Cache Handle: 0x0025
L2 Cache Handle: 0x0026
L3 Cache Handle: Not Provided
Serial Number: Not Specified
Asset Tag: ProcessorInfo_ASSET_TAG
Part Number: Not Specified
Core Count: 8
Core Enabled: 8
Thread Count: 8
Characteristics:
64-bit capable
Handle 0x002A, DMI type 15, 73 bytes
System Event Log
Area Length: 65535 bytes
Header Start Offset: 0x0000
Header Length: 16 bytes
Data Start Offset: 0x0010
Access Method: Memory-mapped physical 32-bit address
Access Address: 0xFFA50000
Status: Valid, Not Full
Change Token: 0x00000001
Header Format: Type 1
Supported Log Type Descriptors: 25
Descriptor 1: Single-bit ECC memory error
Data Format 1: Multiple-event handle
Descriptor 2: Multi-bit ECC memory error
Data Format 2: Multiple-event handle
Descriptor 3: Parity memory error
Data Format 3: None
Descriptor 4: Bus timeout
Data Format 4: None
Descriptor 5: I/O channel block
Data Format 5: None
Descriptor 6: Software NMI
Data Format 6: None
Descriptor 7: POST memory resize
Data Format 7: None
Descriptor 8: POST error
Data Format 8: POST results bitmap
Descriptor 9: PCI parity error
Data Format 9: Multiple-event handle
Descriptor 10: PCI system error
Data Format 10: Multiple-event handle
Descriptor 11: CPU failure
Data Format 11: None
Descriptor 12: EISA failsafe timer timeout
Data Format 12: None
Descriptor 13: Correctable memory log disabled
Data Format 13: None
Descriptor 14: Logging disabled
Data Format 14: None
Descriptor 15: System limit exceeded
Data Format 15: None
Descriptor 16: Asynchronous hardware timer expired
Data Format 16: None
Descriptor 17: System configuration information
Data Format 17: None
Descriptor 18: Hard disk information
Data Format 18: None
Descriptor 19: System reconfigured
Data Format 19: None
Descriptor 20: Uncorrectable CPU-complex error
Data Format 20: None
Descriptor 21: Log area reset/cleared
Data Format 21: None
Descriptor 22: System boot
Data Format 22: None
Descriptor 23: End of log
Data Format 23: None
Descriptor 24: OEM-specific
Data Format 24: OEM-specific
Descriptor 25: OEM-specific
Data Format 25: OEM-specific
Handle 0x002B, DMI type 16, 23 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: Single-bit ECC
Maximum Capacity: 64 GB
Error Information Handle: Not Provided
Number Of Devices: 4
Handle 0x002C, DMI type 19, 31 bytes
Memory Array Mapped Address
Starting Address: 0x00000000000
Ending Address: 0x007FFFFFFFF
Range Size: 32 GB
Physical Array Handle: 0x002B
Partition Width: 1
Handle 0x002D, DMI type 17, 34 bytes
Memory Device
Array Handle: 0x002B
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: SODIMM
Set: None
Locator: DIMMA1
Bank Locator: BANK 0
Type: DDR3
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 1600 MHz
Manufacturer: Toshiba
Serial Number: 68181210
Asset Tag: FBANK 0 DIMMA1 AssetTag
Part Number: 9965527-021.A00LF
Rank: 2
Configured Clock Speed: 1600 MHz
Handle 0x002E, DMI type 20, 35 bytes
Memory Device Mapped Address
Starting Address: 0x00000000000
Ending Address: 0x001FFFFFFFF
Range Size: 8 GB
Physical Device Handle: 0x002D
Memory Array Mapped Address Handle: 0x002C
Partition Row Position: Unknown
Handle 0x002F, DMI type 17, 34 bytes
Memory Device
Array Handle: 0x002B
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: SODIMM
Set: None
Locator: DIMMA2
Bank Locator: BANK 0
Type: DDR3
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 1600 MHz
Manufacturer: Toshiba
Serial Number: 66342426
Asset Tag: FBANK 0 DIMMA2 AssetTag
Part Number: 9965527-021.A00LF
Rank: 2
Configured Clock Speed: 1600 MHz
Handle 0x0030, DMI type 20, 35 bytes
Memory Device Mapped Address
Starting Address: 0x00200000000
Ending Address: 0x003FFFFFFFF
Range Size: 8 GB
Physical Device Handle: 0x002F
Memory Array Mapped Address Handle: 0x002C
Partition Row Position: Unknown
Handle 0x0031, DMI type 17, 34 bytes
Memory Device
Array Handle: 0x002B
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: SODIMM
Set: None
Locator: DIMMB1
Bank Locator: BANK 0
Type: DDR3
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 1600 MHz
Manufacturer: Toshiba
Serial Number: 68181510
Asset Tag: FBANK 0 DIMMB1 AssetTag
Part Number: 9965527-021.A00LF
Rank: 2
Configured Clock Speed: 1600 MHz
Handle 0x0032, DMI type 20, 35 bytes
Memory Device Mapped Address
Starting Address: 0x00400000000
Ending Address: 0x005FFFFFFFF
Range Size: 8 GB
Physical Device Handle: 0x0031
Memory Array Mapped Address Handle: 0x002C
Partition Row Position: Unknown
Handle 0x0033, DMI type 17, 34 bytes
Memory Device
Array Handle: 0x002B
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: SODIMM
Set: None
Locator: DIMMB2
Bank Locator: BANK 0
Type: DDR3
Type Detail: Synchronous Unbuffered (Unregistered)
Speed: 1600 MHz
Manufacturer: Toshiba
Serial Number: 66340727
Asset Tag: FBANK 0 DIMMB2 AssetTag
Part Number: 9965527-021.A00LF
Rank: 2
Configured Clock Speed: 1600 MHz
Handle 0x0034, DMI type 20, 35 bytes
Memory Device Mapped Address
Starting Address: 0x00600000000
Ending Address: 0x007FFFFFFFF
Range Size: 8 GB
Physical Device Handle: 0x0033
Memory Array Mapped Address Handle: 0x002C
Partition Row Position: Unknown
Handle 0x0035, DMI type 13, 22 bytes
BIOS Language Information
Language Description Format: Long
Installable Languages: 1
en|US|iso8859-1
Currently Installed Language: en|US|iso8859-1
Handle 0x0036, DMI type 127, 4 bytes
End Of Table
For clarity, here are a failing, and a succeeding test program. I cannot tell what the functional difference is between them:
test_fail.py:
-------------
from dmidecodeUtil import __leafDict
import dmidecode
a = dmidecode
b = getattr(a,'memory')()
c = __leafDict(b)
print c
# python test_fail.py
Traceback (most recent call last):
File "test_fail.py", line 5, in <module>
c = __leafDict(b)
File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 35, in __leafDict
ret.update(__leafDict(v))
File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 35, in __leafDict
ret.update(__leafDict(v))
File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 33, in __leafDict
for k, v in d.iteritems():
IndexError: list assignment index out of range
But if I use the debugger, break before the call to __leafDict, and capture the dictionary returned in b, then create:
test_good.py:
--------------
from dmidecodeUtil import __leafDict
#import dmidecode
#a = dmidecode
#b = getattr(a,'memory')()
b={'0x0033': {'dmi_type': 17, 'data': {'Part Number': '9965527-021.A00LF', 'Type': 'DDR3FBD2', 'Set': None, 'Data Width': '64 bit', 'AssetTag': 'FBANK 0 DIMMB2 AssetTag', 'Speed': '1600 MHz (ns)', 'Bank Locator': 'BANK 0', 'Error Information Handle': 'No Error', 'Locator': 'DIMMB2', 'Serial Number': '66340727', 'Total Width': '64 bit', 'Type Detail': [None, None, None, None, None, None, 'Synchronous', None, None, None, None, None], 'Manufacturer': 'Toshiba', 'Size': '8192 MB', 'Form Factor': 'SODIMM', 'Array Handle': '0x002b'}, 'dmi_handle': '0x0033', 'dmi_size': 34}, '0x002b': {'dmi_type': 16, 'data': {'Maximum Capacity': '64 GB', 'Number Of Devices': 4, 'Use': 'System Memory', 'Error Information Handle': 'Not Provided', 'Error Correction Type': 'Single-bit ECC', 'Location': 'System Board Or Motherboard'}, 'dmi_handle': '0x002b', 'dmi_size': 23}, '0x002d': {'dmi_type': 17, 'data': {'Part Number': '9965527-021.A00LF', 'Type': 'DDR3FBD2', 'Set': None, 'Data Width': '64 bit', 'AssetTag': 'FBANK 0 DIMMA1 AssetTag', 'Speed': '1600 MHz (ns)', 'Bank Locator': 'BANK 0', 'Error Information Handle': 'No Error', 'Locator': 'DIMMA1', 'Serial Number': '68181210', 'Total Width': '64 bit', 'Type Detail': [None, None, None, None, None, None, 'Synchronous', None, None, None, None, None], 'Manufacturer': 'Toshiba', 'Size': '8192 MB', 'Form Factor': 'SODIMM', 'Array Handle': '0x002b'}, 'dmi_handle': '0x002d', 'dmi_size': 34}, '0x0031': {'dmi_type': 17, 'data': {'Part Number': '9965527-021.A00LF', 'Type': 'DDR3FBD2', 'Set': None, 'Data Width': '64 bit', 'AssetTag': 'FBANK 0 DIMMB1 AssetTag', 'Speed': '1600 MHz (ns)', 'Bank Locator': 'BANK 0', 'Error Information Handle': 'No Error', 'Locator': 'DIMMB1', 'Serial Number': '68181510', 'Total Width': '64 bit', 'Type Detail': [None, None, None, None, None, None, 'Synchronous', None, None, None, None, None], 'Manufacturer': 'Toshiba', 'Size': '8192 MB', 'Form Factor': 'SODIMM', 'Array Handle': '0x002b'}, 'dmi_handle': '0x0031', 'dmi_size': 34}, '0x002f': {'dmi_type': 17, 'data': {'Part Number': '9965527-021.A00LF', 'Type': 'DDR3FBD2', 'Set': None, 'Data Width': '64 bit', 'AssetTag': 'FBANK 0 DIMMA2 AssetTag', 'Speed': '1600 MHz (ns)', 'Bank Locator': 'BANK 0', 'Error Information Handle': 'No Error', 'Locator': 'DIMMA2', 'Serial Number': '66342426', 'Total Width': '64 bit', 'Type Detail': [None, None, None, None, None, None, 'Synchronous', None, None, None, None, None], 'Manufacturer': 'Toshiba', 'Size': '8192 MB', 'Form Factor': 'SODIMM', 'Array Handle': '0x002b'}, 'dmi_handle': '0x002f', 'dmi_size': 34}}
c = __leafDict(b)
print c
Then __leafDict works as expected and I get the result c printed:
{'Use': 'System Memory', 'Location': 'System Board Or Motherboard', 'Type': 'DDR3FBD2', 'Array Handle': '0x002b', 'Number Of Devices': 4, 'Serial Number': '68181510', 'Total Width': '64 bit', 'Form Factor': 'SODIMM', 'dmi_size': 34, 'Manufacturer': 'Toshiba', 'Data Width': '64 bit', 'Part Number': '9965527-021.A00LF', 'AssetTag': 'FBANK 0 DIMMB1 AssetTag', 'Bank Locator': 'BANK 0', 'Error Correction Type': 'Single-bit ECC', 'dmi_handle': '0x0031', 'dmi_type': 17, 'Maximum Capacity': '64 GB', 'Set': None, 'Error Information Handle': 'No Error', 'Locator': 'DIMMB1', 'Type Detail': [None, None, None, None, None, None, 'Synchronous', None, None, None, None, None], 'Speed': '1600 MHz (ns)', 'Size': '8192 MB'}
What's going on?
Hi Ian, I couldn't reproduce it on my fedora 24. Could you check what python-dmidecode version you're using? The version of python-dmidecode is the current release in F24: python-dmidecode-3.12.2-3.fc24.x86_64 I tested the same version and couldn't reproduce the issue on my setup. Anyway, I've added the fix you proposed in order to prevent such failures in the future and the unused verbs will be removed from the code. Thanks again for reporting this issue. I see this fix has been slated for release in ovirt-4.2. Seems like a very simple and straightforward bug fix to be, and not a new feature or a big change requiring extensive verification. I was a bit surprised that the latest 4.1.2 release didn't incorporate the fix, thus it reverted my patched files to the non-working versions. Any chance this could be bumped up to the 4.1.z stream? Target ovirt-4.1.3? *** Bug 1456121 has been marked as a duplicate of this bug. *** verified with SanityOnly in vdsm-4.19.17-1.el7ev.x86_64 patch is present command works successfully |
Description of problem: getVdsHardwareInfo fails as follows: (and causes oVirt hosted-engine-setup to fail) MainProcess|jsonrpc/5::ERROR::2017-05-02 13:29:18,100::supervdsmServer::97::SuperVdsm.Se rverCallback::(wrapper) Error in getHardwareInfo Traceback (most recent call last): File "/usr/share/vdsm/supervdsmServer", line 95, in wrapper res = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_api/hwinfo.py", line 29, in getH ardwareInfo return getHardwareInfoStructure() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 488, in __call__ value = self.func(*args) File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 52, in getHardware InfoStructure dmiInfo = getAllDmidecodeInfo() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 488, in __call__ value = self.func(*args) File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 46, in getAllDmide codeInfo myLeafDict[k] = __leafDict(getattr(dmidecode, k)()) File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 34, in __leafDict ret.update(__leafDict(v)) File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 34, in __leafDict ret.update(__leafDict(v)) File "/usr/lib/python2.7/site-packages/vdsm/dmidecodeUtil.py", line 32, in __leafDict for k, v in d.iteritems(): IndexError: list assignment index out of range Version-Release number of selected component (if applicable): vdsm-python-4.19.10.1-1.fc24.noarch But I also still see the same code unmodified in the master branch at https://github.com/oVirt/vdsm/blame/master/lib/vdsm/dmidecodeUtil.py How reproducible: Assuming 100%, happens on all 3 nodes of a 3-node oVirt cluster Steps to Reproduce: 1. vdsClient -s 0 getVdsHardwareInfo 2. 3. Actual results: getVdsHardwareInfo fails Expected results: getVdsHardwareInfo suceeds and returns the requested hardware info Additional info: The error is that the __leafDict() function in dmidecodeUtil.py does not have a try/catch block to trap the IndexError. I don't care if the root cause fault is actually somewhere in the dmidecode module or not, this vdsm module should trap the error and deal with it gracefully. Worse, in this case it stems from the getHardwareInfoStructure() method, which only returns basic information about the "system" from DMI, while it uses the getAllDmidecodeInfo() method which queries ALL of the DMI info, even though it only cares about the "system" key from the returned dict. While in my case the fault occurs during the processing of the "memory" section, it is completely irrelevant because getHardwareInfoStructure() doesn't even use the returned memory info. Here is the fix: Add a try/catch block in __leafDict() so that it looks like this: def __leafDict(d): try: ret = {} for k, v in d.iteritems(): if isinstance(v, dict): ret.update(__leafDict(v)) else: ret[k] = v return ret except IndexError: return {} This fixes the error in vdsm of not handling something odd about the returned DMI info.