Bug 1393008

Summary: LLDP discovery failure may cause the whole introspection to fail due to empty inventory
Product: Red Hat OpenStack Reporter: Ricardo Noriega <rnoriega>
Component: openstack-ironic-python-agentAssignee: Dmitry Tantsur <dtantsur>
Status: CLOSED ERRATA QA Contact: Sergii Nozhka <snozhka>
Severity: high Docs Contact:
Priority: unspecified    
Version: 10.0 (Newton)CC: ddomingo, jschluet, kiran, mburns, slinaber, snozhka
Target Milestone: rcKeywords: Triaged
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-ironic-python-agent-1.5.0-3.el7ost Doc Type: Bug Fix
Doc Text:
This release features more thorough error checking and handling around LLDP discovery. This enhancement prevents malformed packages from failing LLDP discovery; in addition, failed LLDP discovery no longer fails the whole introspection process.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-14 16:31:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ricardo Noriega 2016-11-08 16:45:23 UTC
Description of problem:

During introspection process, the following issue was found:

Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils [-] Error while getting LLDP info: unpack requires a string argument of length 2
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils Traceback (most recent call last):
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 118, in get_lldp_info
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils return _get_lldp_info(interfaces)
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 187, in _get_lldp_info
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils _receive_lldp_packets(s))
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 154, in _receive_lldp_packets
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils return _parse_tlv(pkt)
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 133, in _parse_tlv
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils tlvhdr = struct.unpack('!H', buff[:2])[0]
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils error: unpack requires a string argument of length 2
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils [-] Error while using raw socket: <class 'struct.error'>: unpack requires a string argument of length 2
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils Traceback (most recent call last):
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 118, in get_lldp_info
Nov 08 13:18:21 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils return _get_lldp_info(interfaces)
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 187, in _get_lldp_info
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils _receive_lldp_packets(s))
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 154, in _receive_lldp_packets
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils return _parse_tlv(pkt)
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils File "/usr/lib/python2.7/site-packages/ironic_python_agent/netutils.py", line 133, in _parse_tlv
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils tlvhdr = struct.unpack('!H', buff[:2])[0]
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils error: unpack requires a string argument of length 2
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:21.571 3335 ERROR ironic_python_agent.netutils
Nov 08 13:18:22 localhost.localdomain dhclient[3277]: DHCPDISCOVER on p3p2 to 255.255.255.255 port 67 interval 8 (xid=0x65dcfd2a)
Nov 08 13:18:22 localhost.localdomain kernel: device em1 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p3p1 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p3p2 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p1p3 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p1p4 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p1p2 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain kernel: device p1p1 left promiscuous mode
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root [-] Unexpected error dispatching list_network_interfaces to manager <ironic_python_agent.hardware.GenericHardwareManager object at 0x2a6cfd0>: 'NoneType' object has no attribute 'items'
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root Traceback (most recent call last):
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root File "/usr/lib/python2.7/site-packages/ironic_python_agent/hardware.py", line 1054, in dispatch_to_managers
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root return getattr(manager, method)(*args, **kwargs)
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root File "/usr/lib/python2.7/site-packages/ironic_python_agent/hardware.py", line 535, in list_network_interfaces
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root self._cache_lldp_data(iface_names)
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root File "/usr/lib/python2.7/site-packages/ironic_python_agent/hardware.py", line 477, in _cache_lldp_data
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root for ifname, tlvs in raw_lldp_data.items():
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root AttributeError: 'NoneType' object has no attribute 'items'
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.309 3335 ERROR root
Nov 08 13:18:22 localhost.localdomain ironic-python-agent[3335]: 2016-11-08 13:18:22.310 3335 ERROR ironic_python_agent.utils [-] collector default failed: 'NoneType' object has no attribute 'items'

Version-Release number of selected component (if applicable):

OSP10 beta

Additional info:

As a workaround, removing ipa-collect-lldp=1 from /httpboot/inspector.ipxe does the job to pass introspection.

Comment 1 Dmitry Tantsur 2016-11-11 11:16:59 UTC
*** Bug 1393974 has been marked as a duplicate of this bug. ***

Comment 3 Sergii Nozhka 2016-11-17 14:14:06 UTC
The issue not reproduced with openstack-ironic-python-agent-1.5.0-3.el7ost.

Comment 4 Sergii Nozhka 2016-11-17 14:14:30 UTC
The issue not reproduced with openstack-ironic-python-agent-1.5.0-3.el7ost.

Comment 6 errata-xmlrpc 2016-12-14 16:31:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html