Bug 1366309 - ironic introspection fails on GCE node
Summary: ironic introspection fails on GCE node
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic-inspector
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Dmitry Tantsur
QA Contact: Raviv Bar-Tal
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-11 14:51 UTC by Jason Montleon
Modified: 2016-08-11 15:17 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-11 15:17:45 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
journal from failed gce node introspection (110.64 KB, text/x-vhdl)
2016-08-11 14:51 UTC, Jason Montleon
no flags Details

Description Jason Montleon 2016-08-11 14:51:29 UTC
Created attachment 1190070 [details]
journal from failed gce node introspection

Description of problem:
Node instrospection fails with:
ironic-inspector inspection failed: The following required parameters are missing: ['memory_mb']  

Version-Release number of selected component (if applicable):
openstack-ironic-api-4.2.2-4.el7ost.noarch
openstack-ironic-common-4.2.2-4.el7ost.noarch
openstack-ironic-conductor-4.2.2-4.el7ost.noarch
openstack-ironic-inspector-2.2.5-2.el7ost.noarch
python-ironicclient-0.8.1-1.el7ost.noarch
python-ironic-inspector-client-1.2.0-6.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. Set up a Ravello environment backed by GCE
2. Set up a director
3. Attempt to introspect a node

Actual results:
Instrospection fails

Expected results:
Introspection succeeds

Additional info:
I can see in the attached journal:
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.866 440 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): dmidecode --type 17 | grep Size execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:230
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.909 440 DEBUG oslo_concurrency.processutils [-] CMD "dmidecode --type 17 | grep Size" returned: 0 in 0.043s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:260
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.914 440 DEBUG ironic_python_agent.utils [-] Execution completed, command line is "dmidecode --type 17 | grep Size" execute /usr/lib/python2.7/site-packages/ironic_python_agent/utils.py:65
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.915 440 DEBUG ironic_python_agent.utils [-] Command stdout is: "        Size: No Module Installed
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: " execute /usr/lib/python2.7/site-packages/ironic_python_agent/utils.py:66
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.916 440 DEBUG ironic_python_agent.utils [-] Command stderr is: "" execute /usr/lib/python2.7/site-packages/ironic_python_agent/utils.py:67
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:23.919 440 DEBUG root [-] One memory slot is empty get_memory /usr/lib/python2.7/site-packages/ironic_python_agent/hardware.py:492

--------------------------------------
Further down memory_mb is listed as 0:
--------------------------------------
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:24.308 440 INFO ironic_python_agent.inspector [-] value for cpus is 4
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:24.309 440 INFO ironic_python_agent.inspector [-] value for local_gb is 99
Aug 11 09:38:24 localhost.localdomain ironic-python-agent[440]: 2016-08-11 09:38:24.309 440 INFO ironic_python_agent.inspector [-] value for memory_mb is 0

Comment 2 Dmitry Tantsur 2016-08-11 14:52:48 UTC
Thanks for the detailed report! Could you please execute 'dmidecode --type 17' on one of such hardware nodes and paste the output?

Comment 3 Jason Montleon 2016-08-11 14:56:19 UTC
# dmidecode --type 17
# dmidecode 2.12-dmifs
SMBIOS 2.8 present.

Handle 0x1100, DMI type 17, 40 bytes
Memory Device
	Array Handle: 0x1000
	Error Information Handle: Not Provided
	Total Width: Unknown
	Data Width: Unknown
	Size: No Module Installed
	Form Factor: DIMM
	Set: None
	Locator: DIMM 0
	Bank Locator: Not Specified
	Type: RAM
	Type Detail: Other
	Speed: Unknown
	Manufacturer: QEMU
	Serial Number: Not Specified
	Asset Tag: Not Specified
	Part Number: Not Specified
	Rank: Unknown
	Configured Clock Speed: Unknown
	Minimum Voltage:  Unknown
	Maximum Voltage:  Unknown
	Configured Voltage:  Unknown


I tried some other values and did see:
# dmidecode --type 16
# dmidecode 2.12-dmifs
SMBIOS 2.8 present.

Handle 0x1000, DMI type 16, 23 bytes
Physical Memory Array
	Location: Other
	Use: System Memory
	Error Correction Type: Multi-bit ECC
	Maximum Capacity: 16 GB
	Error Information Handle: Not Provided
	Number Of Devices: 1

Comment 4 Dmitry Tantsur 2016-08-11 14:58:46 UTC
Wow, that's strange indeed... Unfortunately, maximum capacity != real capacity. I'll see what we can do... In the meanwhile, could you please provide the exact model of the hardware?

Comment 5 Jason Montleon 2016-08-11 15:05:49 UTC
It is a Google Compute Engine VM. I've CC'd jsaucier as he may be able to provide more information about what exactly was specified. 

I understand there may not be much that can be done. From what has been explained to me it is possible to specify different backing services in Ravello (AWS vs GCE, etc.) and it seems in at least some setups there has been success deploying an overcloud.

Comment 6 Dmitry Tantsur 2016-08-11 15:13:49 UTC
Oh, right, I didn't recognize GCE abbreviation initially. I am sorry, but I don't think we can help here much. We need several utilities working, dmidecode including. I do think you might just skip introspection, populating the required fields manually. Or, indeed, you may get more luck with AWS, I've heard some folks were experimenting with it. Please let me know if I can help you in some other way.

Comment 7 Jean-Francois Saucier 2016-08-11 15:15:55 UTC
The hardware is a VM push by Ravello to Google Cloud Engine. Normally, the VM have :

4 CPUs
16GB of memory
100GB of disk
2 network interface (one for provisioning, one for IPMI)


Note You need to log in before you can comment on or make changes to this bug.