Bug 2154288

Summary:	Unit test failure "nm:ERROR:../src/libnm-client-impl/tests/test-nm-client.c:877:_client_dev_removed: assertion failed (arr->len == 0): (1 == 0)"
Product:	Red Hat Enterprise Linux 9	Reporter:	Thomas Haller <thaller>
Component:	NetworkManager	Assignee:	Thomas Haller <thaller>
Status:	CLOSED UPSTREAM	QA Contact:	Desktop QE <desktop-qa-list>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	9.2	CC:	bgalvani, lrintel, rkhan, sfaye, sukulkar, till
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-02-08 10:04:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Thomas Haller 2022-12-16 14:05:40 UTC

when you run unit tests in gitlab-ci, it's relatively easy to get a failure (feels like 1 out of 80 builds). If you then manually run the pipelines for all distros, you have a relatively high probability to encounter the race. (retry, if you don't).

See for example: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/jobs/33621885

I think this started with https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/8dc4b4701be87eeb9abf7f61de075d728a401b77



I think https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/8dc4b4701be87eeb9abf7f61de075d728a401b77 also introduced a second race in the unit tests. That one is harder to hit. They might be related. See for example:
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/jobs/33527162#L5315

ERROR: test_monitor (__main__.TestNmcli.test_monitor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/NetworkManager/NetworkManager/src/tests/client/test-client.py", line 1226, in f
    func(self)
  File "/builds/NetworkManager/NetworkManager/src/tests/client/test-client.py", line 1233, in f
    func(self)
  File "/builds/NetworkManager/NetworkManager/src/tests/client/test-client.py", line 1917, in test_monitor
    nmc.expect("con-1: connection profile removed")
  File "/usr/lib/python3.11/site-packages/pexpect/spawnbase.py", line 343, in expect
    return self.expect_list(compiled_pattern_list,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pexpect/spawnbase.py", line 372, in expect_list
    return exp.expect_loop(timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pexpect/expect.py", line 181, in expect_loop
    return self.timeout(e)
           ^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/pexpect/expect.py", line 144, in timeout
    raise exc
pexpect.exceptions.TIMEOUT: Timeout exceeded.

Comment 1 Thomas Haller 2023-02-08 09:56:19 UTC

fixed by https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/df228a5f8d4de60951fe689950f016896b5dc383

Comment 2 Thomas Haller 2023-02-08 10:04:18 UTC

closing this one as UPSTREAM.

This bug was about a unit test failure. That unit test failure was in upstream (but arguably also in our RHEL package builds).
Still, it's fixed now, the test is re-enabled and there is no further QA necessary.

(the QA for a unit test is just letting the unit test run -- which it does now all the time, and we will notice when it starts failing again).

Comment 3 Thomas Haller 2023-02-08 10:10:15 UTC

btw, in comment 0, we were talking about two unit test failures.

1)
  "nm:ERROR:../src/libnm-client-impl/tests/test-nm-client.c:877:_client_dev_removed: assertion failed (arr->len == 0): (1 == 0)" (edit)
  fixed by https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/c1d234ce30a301ae9af2462343edf03431608e43

2)
  ERROR: test_monitor (__main__.TestNmcli.test_monitor)
  fixed by https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/df228a5f8d4de60951fe689950f016896b5dc383

(both are fixed, and both tests are running (on upstream `main` and `nm-1-42` branches)