Bug 1535685
| Summary: | Introspection fails because LLDP discovery assumes the Chassis ID to be a MAC address | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Joachim von Thadden <j.thadden> | |
| Component: | openstack-ironic-inspector | Assignee: | Bob Fournier <bfournie> | |
| Status: | CLOSED ERRATA | QA Contact: | mlammon | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 12.0 (Pike) | CC: | bfournie, j.thadden, mburns, slinaber | |
| Target Milestone: | beta | Keywords: | Triaged | |
| Target Release: | 13.0 (Queens) | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | openstack-ironic-inspector-7.2.0-0.20180217115157.a8d621f.el7ost | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1544873 (view as bug list) | Environment: | ||
| Last Closed: | 2018-06-27 13:42:25 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1544873 | |||
I think the exception is being raised here: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/types.py#L323 in Ironic (not inspector). The local_link_connection requires a MAC. If the type in the Chassid ID TLV is not MAC type (4) then the TLV shouldn't be processed by the ironic-inspector plugin. I'm not sure why it is, or the type is actually 4 and the value is erroneously an IP. Is it possible to get saved introspection data for this node using: openstack baremetal introspection data save <node_uuid>? Also can you please provide the rpm info for ironic and ironic-inspector and the ironic-inspector.log and ironic-conductor.log files? Thanks. And anyway, a false entry in the chassis id should never lead to a aborted introspection! OK, thanks. It looks like the likely way this occurred is an invalid LLDP packet from the switch - i.e. subtype identifying that the field is a mac address, but the field actually containing an IP address. But its not possible to know for certain without the introspection data. The intent of the local_link_connection is that it would be used for physical network mapping so if its not a MAC the mapping could be incorrect. I agree that failing introspection in this case is harsh though. Looks like problem can occur if switch correctly reports Chassis ID as something other than a mac address. Change to use Gerrit ID for stable/queens Install osp 13 latest puddle: 2018-05-07.2
This cannot be verified using specific hardware (not available), however we can verify that the new code is in place and RPM which we do here:
Env:
openstack-ironic-inspector-7.2.1-0.20180409163359.2435d97.el7ost.noarch
In /usr/lib/python2.7/site-packages/ironic_inspector/plugins/local_link_connection.py
# Only accept mac address for chassis ID
if (item == SWITCH_ID_ITEM_NAME and
not netutils.is_valid_mac(value)):
LOG.info("Skipping switch_id since it's not a MAC: %s", value,
node_info=node_info)
return
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086 |
Description of problem: Introspection fails with Introspection of node 3d68b5f6-0077-43eb-9991-bf5ceb4f1ccd completed. Status:FAILED. Errors:Unexpected exception BadRequest during processing: Expected a MAC address or OpenFlow datapath ID but received 128.0.128.34. (HTTP 400) Version-Release number of selected component (if applicable): This error is probably also in RHOSP10 and 11, but I can not check at the moment. How reproducible: always, if the chassis ID is not a MAC address Steps to Reproduce: 1. start introspection 2. wait for failure Actual results: introspection fails Expected results: introspection succeeds Additional info: Chassis ID is a string field and can be *any* value, also many vendors predefine them with a MAC address. The error is in /usr/lib/python2.7/site-packages/ironic_inspector/plugins/local_link_connection.py in class GenericLocalLinkConnectionHook: if 'mac_address' in chassis_id.subtype: item = SWITCH_ID_ITEM_NAME value = chassis_id.value -> here it is related to LLDP discovery -> and in this case the Chassis ID is not a MAC and the code is searching for that -> the error is that the different types of chassis are not taken into account, see http://www.brocade.com/content/html/en/configuration-guide/NI_05800a_SWITCHING/GUID-19D1919F-4BB1-42B3-A683-CECFF8094D5A.html Quick Fix: If possible just change the Chassis ID to a MAC address for the introspection.