Bug 2265822 - Flaky failures in test_timing with python-opentelemetry-contrib 1.23.0/0.44b
Summary: Flaky failures in test_timing with python-opentelemetry-contrib 1.23.0/0.44b
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: python-sentry-sdk
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Roman Inflianskas
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 2265699
TreeView+ depends on / blocked
 
Reported: 2024-02-24 15:02 UTC by Ben Beasley
Modified: 2024-07-24 18:43 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ben Beasley 2024-02-24 15:02:31 UTC
Description of problem:

With python-opentelemetry-contrib 1.23.0/0.44b0[1][2], I’m seeing failures like

_________________________________ test_timing __________________________________
tests/test_metrics.py:112: in test_timing
    (envelope,) = envelopes
E   ValueError: too many values to unpack (expected 1)

in most of my local mock builds, and occasional failures like

____________________________ test_timing_decorator _____________________________
tests/test_metrics.py:180: in test_timing_decorator
    (envelope,) = envelopes
E   ValueError: too many values to unpack (expected 1)

I’m planning to build this update in F40 and F41 soon, but I want to allow enough time for python-sentry-sdk to adapt.

I tried to reproduce the problem in COPR[3], but of all the builds on F40 and F41, only the build on ppc64le on F41 failed[4].

The flakiness and the discrepancy between COPR and local builds seems to support the idea that the problem is one of timing – something is running faster or slower after the python-opentelemetry-contrib than this test expects – rather than a true incompatibility. Although I haven’t been able to reproduce the problem with the currently-packaged python-opentelemetry-contrib 1.22.0/0.43b0, I wonder if it might be possible in certain environments and/or with lower probability.

Version-Release number of selected component (if applicable): 1.39.2-2

How reproducible:

Steps to Reproduce:
1. In mock or in COPR, build [1].
2. In mock or in COPR, using the results of [1], build [2].
3. In mock or in COPR, using the results of [1] and [2], build python-sentry-sdk.

Actual results:

Flaky failures of test_timing and (with lower probability) test_timing_decorator, with frequency depending on environment.

Expected results:

Tests always pass.

Additional info:

If there is no response to this bug, I plan to proceed with the update in one week, on 2024-03-02 or slightly later. For F40, this means the update will not reach stable until the end of the Beta Freeze. If you ask me for more time to investigate, I’m happy to delay the update, although I would still like to have this in F40 before the Final Freeze, scheduled for 2024-04-02.

[1] https://src.fedoraproject.org/rpms/python-opentelemetry/pull-request/12
[2] https://src.fedoraproject.org/rpms/python-opentelemetry-contrib/pull-request/7
[3] https://copr.fedorainfracloud.org/coprs/music/opentelemetry-1.23/packages/
[4] https://copr.fedorainfracloud.org/coprs/music/opentelemetry-1.23/build/7055251/

Comment 1 Ben Beasley 2024-03-04 18:47:21 UTC
I just did a scratch-build of python-sentry-sdk in the same side tag with the opentelemetry/opentelemetry-contrib updates for F41, https://koji.fedoraproject.org/koji/taskinfo?taskID=114462898, and it was successful. So that’s a good sign that this is not too flaky to get successful builds in production Koji at least some of the time.

Comment 2 Roman Inflianskas 2024-07-24 18:43:14 UTC
This stroke again; see: https://src.fedoraproject.org/rpms/python-sentry-sdk/pull-request/10#comment-210847 (from https://kojipkgs.fedoraproject.org/work/tasks/902/120920902/build.log).

I have temporarily disabled these tests in https://src.fedoraproject.org/rpms/python-sentry-sdk/pull-request/10 until they will be reimplemented using mocking time, as described in https://github.com/getsentry/sentry-python/issues/3335.


Note You need to log in before you can comment on or make changes to this bug.