Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1561020

Summary: ironic'Version' object does not support indexing (HTTP 500) can not rebuild cloud
Product: Red Hat OpenStack Reporter: Alex Krzos <akrzos>
Component: python-oslo-utilsAssignee: Victor Stinner <vstinner>
Status: CLOSED NOTABUG QA Contact: Shai Revivo <srevivo>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 12.0 (Pike)CC: akrzos, apevec, lhh
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-28 12:31:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alex Krzos 2018-03-27 13:12:45 UTC
Description of problem:
Attempting to build a new OpenStack overcloud via Tripleo after a cloud had already been built results in error in ironic api.

Stack trace in /var/log/ironic/app.log
2018-03-27 13:05:59.905 104145 ERROR wsme.api [req-4e6fd43b-a0b6-48e3-bbff-3d51b8560d58 b473d94dd5624a96a445bc4f8b99b1d4 2a809d1ef62a472ba81d4eb0f2c3812b - default default] Server-side error: "'Version' object does not support indexing". Detail:
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/wsmeext/pecan.py", line 85, in callfunction
    result = f(self, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/ironic/api/controllers/v1/node.py", line 1581, in get_one
    rpc_node = api_utils.get_rpc_node(node_ident)

  File "/usr/lib/python2.7/site-packages/ironic/api/controllers/v1/utils.py", line 156, in get_rpc_node
    return objects.Node.get_by_uuid(pecan.request.context, node_ident)

  File "/usr/lib/python2.7/site-packages/ironic/objects/node.py", line 207, in get_by_uuid
    node = cls._from_db_object(context, cls(), db_node)

  File "/usr/lib/python2.7/site-packages/ironic/objects/base.py", line 261, in _from_db_object
    if not versionutils.is_compatible(db_version, obj.__class__.VERSION):

  File "/usr/lib/python2.7/site-packages/oslo_utils/versionutils.py", line 45, in is_compatible
    if same_major and (requested_parts[0] != current_parts[0]):

TypeError: 'Version' object does not support indexing


CLI issues captured in gist
https://gist.github.com/akrzos/dcbad7b0521d2422e2fe461d60e43440


The issue appears to be with oslo_utils.

Version-Release number of selected component (if applicable):
Pike (OSP12)
Build: 2018-01-26.2

How reproducible:
Always with this build and orchestration playbooks


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Victor Stinner 2018-03-28 09:46:58 UTC
I guess that the Version object comes from setuptools.

setuptools version 39 introduced a backward incompatible change:

"#296: Removed long-deprecated support for iteration on Version objects as returned by pkg_resources.parse_version. Removed the SetuptoolsVersion and SetuptoolsLegacyVersion names as well. They should not have been used, but if they were, replace with Version and LegacyVersion from packaging.version."

https://github.com/pypa/setuptools/blob/master/CHANGES.rst#v3900

Alex Krzos: Would you be able to provide me your setuptools version? Do you know how setuptools was installed? A workaround would be to downgrade setuptools. I'm surprised that setuptools has been upgraded.

"build a new OpenStack overcloud via Tripleo": it does install RPMs from OSP, no?

Comment 3 Victor Stinner 2018-03-28 09:50:02 UTC
> Do you know how setuptools was installed?

It really looks like the most recent setuptools version has been installed from PyPI, since setuptools version 39 was only released 11 days ago.

Comment 4 Victor Stinner 2018-03-28 10:05:41 UTC
I wrote "setuptools", but the used Python module is "pkg_resources". I'm quite sure that pkg_resources is provided by the python-setuptools package.

I'm interested by the python-setuptools package version, but also to know how it was installed, and if it was recently upgraded: have a look at /var/log/yum.log and/or attach the file to this issue.

Comment 5 Alex Krzos 2018-03-28 12:31:13 UTC
(In reply to Victor Stinner from comment #2)
> 
> Alex Krzos: Would you be able to provide me your setuptools version? Do you
> know how setuptools was installed? A workaround would be to downgrade
> setuptools. I'm surprised that setuptools has been upgraded.
> 
> "build a new OpenStack overcloud via Tripleo": it does install RPMs from
> OSP, no?


setuptools was upgraded by our Browbeat install playbook via pip on the base rhel install.  setuptools 39.0.1 was installed which is causing the issue.  We are pining to an older version of setuptools to fix this.  Since this was external tooling causing the issue we can close this bug.

Thanks Victor for the help!

Comment 6 Victor Stinner 2018-03-28 13:48:02 UTC
Hopefully, oslo.utils 3.28.2 has been released and contains the fix for this bug (support setuptools 39 and newer). OSP 12 is being updated to oslo.utils 3.28.1 which still has the bug: bz#1545607. The fix should be included in the following (automatic) update process.