Bug 1293679

Summary: Rebase python-oslo-versionedobjects to 1.1.0
Product: Red Hat OpenStack Reporter: Lee Yarwood <lyarwood>
Component: python-oslo-versionedobjectsAssignee: RHOS Maint <rhos-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Shai Revivo <srevivo>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 8.0 (Liberty)CC: apevec, dasmith, lhh, lyarwood, vstinner
Target Milestone: ---Keywords: ZStream
Target Release: 8.0 (Liberty)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-20 15:43:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1293607    

Description Lee Yarwood 2015-12-22 16:41:43 UTC
Description of problem:

python-oslo-versionedobjects-0.10.0-1.el7ost.noarch needs to be rebased to 0.13.0 to allow Nova's nova.tests.unit.objects.test_objects.TestRemoteObject tests to pass.

Version-Release number of selected component (if applicable):
python-oslo-versionedobjects-0.10.0-1.el7ost.noarch

How reproducible:
Always

Steps to Reproduce:
1. ./run_tests.sh nova.tests.unit.objects.test_objects.TestRemoteObject

Actual results:

CI failures here :

Easy to reproduce locally by pinning oslo.versionedobjects to 0.10.0 :

# grep oslo.versionedobject requirements.txt 
oslo.versionedobjects==0.10.0
# ./run_tests.sh nova.tests.unit.objects.test_objects.TestRemoteObject
[..]
FAIL: nova.tests.unit.objects.test_objects.TestRemoteObject.test_compat
----------------------------------------------------------------------
Traceback (most recent call last):
testtools.testresult.real._StringException: Empty attachments:
  stdout

pythonlogging:'': {{{
2015-12-22 16:33:29,490 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,491 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,491 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,492 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,492 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,493 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,494 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,494 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,495 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,496 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,496 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,497 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:29,497 ERROR [oslo_messaging._drivers.zmq_driver.zmq_async] ZeroMQ not found!
2015-12-22 16:33:31,416 INFO [248_add_expire_reservations_index] Skipped adding reservations_deleted_expire_idx because an equivalent index already exists.
}}}

stderr: {{{
nova/context.py:181: DeprecationWarning: Using function/method 'oslo_utils.timeutils.strtime()' is deprecated in version '1.6' and will be removed in a future version: use either datetime.datetime.isoformat() or datetime.datetime.strftime() instead
  self, 'timestamp') else None,
/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/pkg_resources/__init__.py:213: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
  stacklevel=1,
}}}

Traceback (most recent call last):
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "nova/tests/unit/objects/test_objects.py", line 942, in test_compat
    self.assertEqual('oldbar', obj.bar)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 350, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'oldbar' != u'bar'


======================================================================
FAIL: nova.tests.unit.objects.test_objects.TestRemoteObject.test_major_version_mismatch
----------------------------------------------------------------------
Traceback (most recent call last):
testtools.testresult.real._StringException: Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "nova/tests/unit/objects/test_objects.py", line 921, in test_major_version_mismatch
    MyObj2.query, self.context)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 422, in assertRaises
    self.assertThat(our_callable, matcher)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: <bound method type.query of <class 'nova.tests.unit.objects.test_objects.MyObj2'>> returned MyObj(bar='bar',created_at=<?>,deleted=<?>,deleted_at=<?>,foo=1,missing=<?>,mutable_default=<?>,readonly=<?>,rel_object=<?>,rel_objects=<?>,updated_at=<?>)


======================================================================
FAIL: nova.tests.unit.objects.test_objects.TestRemoteObject.test_minor_version_greater
----------------------------------------------------------------------
Traceback (most recent call last):
testtools.testresult.real._StringException: Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/mock/mock.py", line 1305, in patched
    return func(*args, **keywargs)
  File "nova/tests/unit/objects/test_objects.py", line 929, in test_minor_version_greater
    MyObj2.query, self.context)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 422, in assertRaises
    self.assertThat(our_callable, matcher)
  File "/home/lyarwood/redhat/devel/pkgs/openstack-nova/.venv/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: <bound method type.query of <class 'nova.tests.unit.objects.test_objects.MyObj2'>> returned MyObj(bar='bar',created_at=<?>,deleted=<?>,deleted_at=<?>,foo=1,missing=<?>,mutable_default=<?>,readonly=<?>,rel_object=<?>,rel_objects=<?>,updated_at=<?>)


Ran 53 tests in 17.043s

FAILED (failures=3)

Expected results:

# rm -rf .venv/
# grep oslo.versionedobject requirements.txt
oslo.versionedobjects==0.13.0
# ./run_tests.sh nova.tests.unit.objects.test_objects.TestRemoteObject
[..]
Ran 53 tests in 16.572s

OK

Additional info:

0.12.0 and 0.11.0 also fail :

# grep oslo.versionedobject requirements.txt
oslo.versionedobjects==0.12.0
# ./run_tests.sh nova.tests.unit.objects.test_objects.TestRemoteObject
[..]
Ran 53 tests in 16.535s

FAILED (failures=3)

Comment 3 Lee Yarwood 2015-12-23 09:16:22 UTC
(In reply to Lee Yarwood from comment #0)
> Description of problem:
> 
> python-oslo-versionedobjects-0.10.0-1.el7ost.noarch needs to be rebased to
> 0.13.0 to allow Nova's nova.tests.unit.objects.test_objects.TestRemoteObject
> tests to pass.

Apologies, rebasing this package isn't possible at present as the upstream stable/liberty branch for the project only contains version 0.10.0. That branch would need to be updated before any rebasing can happen downstream.

For now I've cherry-picked the following changes upstream on to ovo's stable/liberty branch. When added to our 0.10.0-1 version downstream these changes allow Nova's TestRemoteObject tests to pass :

Make class action calls use version manifest
https://review.openstack.org/#/c/260666/

Make tests properly use object_class_action_versions()
https://review.openstack.org/#/c/260667/

The Nova change failing as part of the rebase to stable/liberty in RHBZ#1293607 is :

Fix Nova's indirection fixture override
https://review.openstack.org/#/c/246022/

These TestRemoteObject changes are then backed out via a follow up commit :

Remove the TestRemoteObject class
https://review.openstack.org/#/c/248877/

I can easily drop these changes from the rebase to workaround this but I think we will be hiding a much bigger issue at that point. Reviewing the test logs for both of these changes upstream shows that the latest version of ovo was used to test these stable/liberty changes for Nova and not the corresponding ovo stable/liberty version of 0.10.0 :

http://logs.openstack.org/77/248877/4/check/gate-nova-python27/674d63b/console.html

2015-11-17 21:13:20.747 | oslo.versionedobjects==0.13.0  # git sha d7520a2

http://logs.openstack.org/77/248877/4/check/gate-nova-python27/674d63b/console.html

2015-12-21 15:47:55.861 | oslo.versionedobjects==1.1.0  # git sha 37df384

So our combination of stable/liberty Nova and ovo does not appear to be tested anywhere upstream at present. Do we need a larger rebase of the ovo stable/liberty branch or should we default to using the latest and greatest as is the current practice upstream, even for stable/liberty?

Comment 8 Victor Stinner 2016-06-20 13:24:42 UTC
The failing test (TestRemoteObject.test_compat) was removed from nova in OSP 8 ( change https://code.engineering.redhat.com/gerrit/#/c/64367/ ).

Is it still useful to rebase oslo.versionedobjects in OSP 8? I suggest to keep oslo.versionedobjects 0.10.0-1.el7ost for OSP8, and maybe "cherry pick" some fixes if needed.

Comment 9 Dan Smith 2016-06-20 13:57:37 UTC
I believe this has been fixed by reverting some things upstream that were not supposed to be added to older releases (specifically test dependencies during the migration to the upper-constraints process). Suggest we drop this at this point.

Comment 10 Lee Yarwood 2016-06-20 15:39:04 UTC
Yeah apologies, we actually got around this once a tarball for 12.0.1 was released. IMHO we can close this out now.

Comment 11 Victor Stinner 2016-06-20 15:43:40 UTC
Ok, thanks for the confirmation Lee :-) I close the issue.