Hide Forgot
Created attachment 1323267 [details] ramdisk-journal-output Description of problem: OpenStack Introspection fails to complete, with previous IPA images, it completes just fine. From the inspector logs : fd:fe:05:24:40', u'3c:fd:fe:05:24:42'] for node look up find_node /usr/lib/python2.7/site-packages/ironic_inspector/node_cache.py:805 2017-09-06 17:23:26.544 12628 INFO ironic_inspector.utils [-] Could not find a node for attributes {'bmc_address': u'0.0.0.0', 'mac': [u'f4:52:14:33:ea:61', u'2c:44:fd:96:4e:50', u'2c:44:fd:96:4e:51', u'2c:44:fd:96:4e:52', u'2c:44:fd:96:4e:53', u'3c:fd:fe:05:24:40', u'3c:fd:fe:05:24:42']} 2017-09-06 17:23:26.545 12628 INFO ironic_inspector.process [-] [node: MAC 2c:44:fd:96:4e:51] Ramdisk logs were stored in file unknown_20170906-212326.544429.tar.gz 2017-09-06 17:23:26.545 12628 ERROR ironic_inspector.utils [-] [node: MAC 2c:44:fd:96:4e:51] The following failures happened during running pre-processing hooks: Look up error: Could not find a node for attributes {'bmc_address': u'0.0.0.0', 'mac': [u'f4:52:14:33:ea:61', u'2c:44:fd:96:4e:50', u'2c:44:fd:96:4e:51', u'2c:44:fd:96:4e:52', u'2c:44:fd:96:4e:53', u'3c:fd:fe:05:24:40', u'3c:fd:fe:05:24:42']} 2017-09-06 17:23:26.545 12628 DEBUG ironic_inspector.main [-] Returning error to client: The following failures happened during running pre-processing hooks: Look up error: Could not find a node for attributes {'bmc_address': u'0.0.0.0', 'mac': [u'f4:52:14:33:ea:61', u'2c:44:fd:96:4e:50', u'2c:44:fd:96:4e:51', u'2c:44:fd:96:4e:52', u'2c:44:fd:96:4e:53', u'3c:fd:fe:05:24:40', u'3c:fd:fe:05:24:42']} error_response /usr/lib/python2.7/site-packages/ironic_inspector/main.py:67 Version-Release number of selected component (if applicable): rhosp-director-images-ipa-12.0-20170830.1.el7ost.noarch How reproducible: 100% Actual results: In description Expected results: Introspection to complete. Additional info:
Potential blocker - regression in ironic-python-agent inspection code.
Joe, do you have an access to a similar server with an OS? Could you please run the following commands for me: sudo ipmitool lan print for i in $(seq 1 16); do sudo ipmitool lan print $i | grep "IP Address"; done I'm trying to figure out which IPMI channel your machines are using. Our code has an assumption that a valid channel cannot follow an invalid one - I suspect it's wrong for you.
[root@overcloud-compute-1 heat-admin]# ipmitool lan print Set in Progress : Set Complete Auth Type Support : Auth Type Enable : Callback : : User : : Operator : : Admin : : OEM : IP Address Source : DHCP Address IP Address : 10.12.8.144 Subnet Mask : 255.255.248.0 MAC Address : b4:b5:2f:ef:86:0e SNMP Community String : BMC ARP Control : ARP Responses Enabled, Gratuitous ARP Disabled Default Gateway IP : 10.12.15.254 802.1q VLAN ID : Disabled 802.1q VLAN Priority : 0 Cipher Suite Priv Max : Not Available Bad Password Threshold : Not Available [root@overcloud-compute-1 heat-admin]# for i in $(seq 1 16); do sudo ipmitool lan print $i | grep "IP Address"; done Invalid channel: 1 IP Address Source : DHCP Address IP Address : 10.12.8.144 Invalid channel: 3 Invalid channel: 4 Invalid channel: 5 Invalid channel: 6 Invalid channel: 7 Invalid channel: 8 Invalid channel: 9 Invalid channel: 10 Invalid channel: 11 Invalid channel: 12 Invalid channel: 13 Invalid channel: 14 Invalid channel: 15 Invalid channel: 16
As I expected. Thanks Joe!
Related BZ filed a while ago against OSP 10 https://bugzilla.redhat.com/show_bug.cgi?id=1430201 Atleast, some of the error message looks the same. Dmitry, could you confirm if this patch also fixes that and should be backported to 10?
Hi! This issues are not related.
Thanks Dmitry!
Hi Joe, do you think you could test this with the package that includes the patch? Thanks!
Hi Joe, Do you think you can test this out so we can get it verified? Thanks!
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462
Removing the needinfo as this is Closed.