Bug 1535685 - Introspection fails because LLDP discovery assumes the Chassis ID to be a MAC address
Summary: Introspection fails because LLDP discovery assumes the Chassis ID to be a MAC...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ironic-inspector
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: beta
: 13.0 (Queens)
Assignee: Bob Fournier
QA Contact: mlammon
URL:
Whiteboard:
Depends On:
Blocks: 1544873
TreeView+ depends on / blocked
 
Reported: 2018-01-17 21:36 UTC by Joachim von Thadden
Modified: 2018-06-27 13:43 UTC (History)
4 users (show)

Fixed In Version: openstack-ironic-inspector-7.2.0-0.20180217115157.a8d621f.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1544873 (view as bug list)
Environment:
Last Closed: 2018-06-27 13:42:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Launchpad 1748022 None None None 2018-02-07 21:57:01 UTC
OpenStack gerrit 543961 None MERGED Only set switch_id in local_link_connection if it is a mac address 2020-07-14 17:41:54 UTC
Red Hat Product Errata RHEA-2018:2086 None None None 2018-06-27 13:43:05 UTC

Description Joachim von Thadden 2018-01-17 21:36:57 UTC
Description of problem:
Introspection fails with

Introspection of node 3d68b5f6-0077-43eb-9991-bf5ceb4f1ccd completed. Status:FAILED. Errors:Unexpected exception BadRequest during processing: Expected a MAC address or OpenFlow datapath ID but received 128.0.128.34. (HTTP 400)  


Version-Release number of selected component (if applicable):
This error is probably also in RHOSP10 and 11, but I can not check at the moment.

How reproducible:
always, if the chassis ID is not a MAC address

Steps to Reproduce:
1. start introspection
2. wait for failure

Actual results:
introspection fails

Expected results:
introspection succeeds

Additional info:
Chassis ID is a string field and can be *any* value, also many vendors predefine them with a MAC address.

The error is in
/usr/lib/python2.7/site-packages/ironic_inspector/plugins/local_link_connection.py

in class GenericLocalLinkConnectionHook:

     if 'mac_address' in chassis_id.subtype:  
          item = SWITCH_ID_ITEM_NAME  
          value = chassis_id.value  

-> here it is related to LLDP discovery
-> and in this case the Chassis ID is not a MAC and the code is searching for that
-> the error is that the different types of chassis are not taken into account, see http://www.brocade.com/content/html/en/configuration-guide/NI_05800a_SWITCHING/GUID-19D1919F-4BB1-42B3-A683-CECFF8094D5A.html



Quick Fix:
If possible just change the Chassis ID to a MAC address for the introspection.

Comment 1 Bob Fournier 2018-01-18 01:33:56 UTC
I think the exception is being raised here: https://github.com/openstack/ironic/blob/master/ironic/api/controllers/v1/types.py#L323
in Ironic (not inspector). 

The local_link_connection requires a MAC.  If the type in the Chassid ID TLV is not MAC type (4) then the TLV shouldn't be processed by the ironic-inspector plugin. I'm not sure why it is, or the type is actually 4 and the value is erroneously an IP.  

Is it possible to get saved introspection data for this node using:
openstack baremetal introspection data save <node_uuid>?

Also can you please provide the rpm info for ironic and ironic-inspector and the ironic-inspector.log and ironic-conductor.log files?  Thanks.

Comment 3 Joachim von Thadden 2018-01-29 19:24:50 UTC
And anyway, a false entry in the chassis id should never lead to a aborted introspection!

Comment 4 Bob Fournier 2018-01-29 21:25:50 UTC
OK, thanks.  It looks like the likely way this occurred is an invalid LLDP packet from the switch - i.e. subtype identifying that the field is a mac address, but the field actually containing an IP address. But its not possible to know for certain without the introspection data.

The intent of the local_link_connection is that it would be used for physical network mapping so if its not a MAC the mapping could be incorrect.  I agree that failing introspection in this case is harsh though.

Comment 5 Bob Fournier 2018-02-07 21:58:09 UTC
Looks like problem can occur if switch correctly reports Chassis ID as something other than a mac address.

Comment 6 Bob Fournier 2018-02-13 16:42:19 UTC
Change to use Gerrit ID for stable/queens

Comment 10 mlammon 2018-05-09 16:42:19 UTC
Install osp 13 latest puddle: 2018-05-07.2

This cannot be verified using specific hardware (not available), however we can verify that the new code is in place and RPM which we do here:

Env:
openstack-ironic-inspector-7.2.1-0.20180409163359.2435d97.el7ost.noarch

In   /usr/lib/python2.7/site-packages/ironic_inspector/plugins/local_link_connection.py

          # Only accept mac address for chassis ID
            if (item == SWITCH_ID_ITEM_NAME and
                    not netutils.is_valid_mac(value)):
                LOG.info("Skipping switch_id since it's not a MAC: %s", value,
                         node_info=node_info)
                return

Comment 12 errata-xmlrpc 2018-06-27 13:42:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.