Bug 1728646 - Functional tests are failing on RHEL8 with python36 (subunit.parser error with stestr-3)
Summary: Functional tests are failing on RHEL8 with python36 (subunit.parser error wit...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 15.0 (Stein)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: ---
Assignee: Michal Pryc
QA Contact: OSP DFG:Compute
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-10 10:56 UTC by Michal Pryc
Modified: 2023-03-21 19:19 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-30 20:05:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1629047 0 None None None 2019-07-10 11:00:09 UTC
Launchpad 1813147 0 None None None 2019-07-10 11:00:09 UTC
Launchpad 1832164 0 None None None 2019-07-10 11:00:09 UTC

Description Michal Pryc 2019-07-10 10:56:50 UTC
Functional tests on OSP15 with stestr-3 on RHEL8 are failing with subunit error:

ft1.1: subunit.parsertesttools.testresult.real._StringException: Binary content:
  Packet data (application/octet-stream)

Parser Error: {{{Bad checksum - calculated (0x4d492833), stored (0x3031392d)}}}
ft1.2: subunit.parsertesttools.testresult.real._StringException: lost connection during failure report of test 'subunit.parser'


This looks similar to upstream https://bugs.launchpad.net/nova/+bug/1832164, however backporting only this patch to OSP15 is not resolving issue.

The root cause is too much information in Exception causing subunit to fail with that amount of stderr.

In upstream there were number of patches to reduce such stderr deprecation warnings, e.g.:

https://bugs.launchpad.net/nova/+bug/1813147

https://bugs.launchpad.net/nova/+bug/1629047

Comment 1 Michal Pryc 2019-07-11 09:09:31 UTC
Those are top deprecation warning messages:

-----

b'/usr/lib/python3.6/site-packages/paste/deploy/loadwsgi.py:22: DeprecationWarning: Parameters to load are deprecated.  Call .resolve and .require separately.'
b'  return pkg_resources.EntryPoint.parse("x=" + s).load(False)'



----- This one is fixed by codeeng 175704-----

b'/home/zuul/src/code.engineering.redhat.com/nova/nova/db/sqlalchemy/api.py:1879: SADeprecationWarning: The joinedload_all() function is deprecated, and will be removed in a future release.  Please use method chaining with joinedload() instead'



----- Below two can be 'fixed' by downgrading SQLAlchemy to version 1.2.18 -----

b'/usr/lib/python3.6/site-packages/migrate/changeset/databases/visitor.py:76: SADeprecationWarning: The Engine.contextual_connect() method is deprecated.  This method is an artifact of the threadlocal engine strategy which is also to be deprecated.   For explicit connections from an Engine, use the Engine.connect() method.'

-----

b'/usr/lib64/python3.6/site-packages/sqlalchemy/orm/base.py:220: SADeprecationWarning: The Query.with_lockmode() method is deprecated and will be removed in a future release.  Please refer to Query.with_for_update(). '

Comment 2 Michal Pryc 2019-07-11 14:25:06 UTC
There were number of failed runs and from what I can see the common pattern is that for all failed tests the following one was inprogress:


nova.tests.functional.notification_sample_tests.test_instance.TestInstanceNotificationSampleOldAttachFlow.test_instance_action

Comment 3 Michal Pryc 2019-07-11 14:52:56 UTC
Looks like removing test_instance_action causes tests to pass, which means the following upstream fix did not resolve issue completely:

The following commit was trying to fix the issue:
   https://github.com/openstack/nova/commit/93027713ab614b59ebb66f0617f85d43c956427c

Comment 6 Cyril Roelandt 2020-03-23 15:43:06 UTC
We have a similar issue in Glance, and Abhishek Kekane told me that the recently released subunit 1.4.0 fixes this issue.

There is already a patch in RDO to start using this new version in U (https://review.rdoproject.org/r/#/c/26033/).

Comment 7 stchen 2020-09-30 20:05:15 UTC
Closing EOL, OSP 15 has been retired as of Sept 19, 2020


Note You need to log in before you can comment on or make changes to this bug.