RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2028849 - [NMCI] vlan_create_500_vlans test failure
Summary: [NMCI] vlan_create_500_vlans test failure
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: NetworkManager
Version: 8.6
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: ---
Assignee: Beniamino Galvani
QA Contact: Vladimir Benes
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-03 14:56 UTC by Vladimir Benes
Modified: 2022-05-10 15:32 UTC (History)
8 users (show)

Fixed In Version: NetworkManager-1.36.0-0.7.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-10 14:54:52 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-104711 0 None None None 2021-12-03 14:57:23 UTC
Red Hat Product Errata RHEA-2022:1985 0 None None None 2022-05-10 14:55:07 UTC
freedesktop.org Gitlab NetworkManager NetworkManager merge_requests 1066 0 None merged Fix vlan_create_500_vlans CI test failure 2022-01-26 14:18:49 UTC

Description Vladimir Benes 2021-12-03 14:56:51 UTC 
Description of problem:
I've modified test to be super simple, you can see it in vb/500_vlans

    @rhbz1933041 @rhbz1926599
    @ver+=1.30 @rhelver+=8 @skip_in_kvm @skip_in_centos
    @logging_info_only @restart_if_needed @500_vlans
    @vlan_create_500_vlans
    Scenario: NM - vlan - create 500 vlans
    # Prepare veth pair with the other end in namespace
    # Create 500 (from 10 to 510) vlans on top of eth11p
    # Run dnsmasq inside the namespace to server incoming connections
    * Execute "sh prepare/vlans.sh setup 500"
    # Create 501 profiles which should be autoconnected after a while
    * Execute "for i in $(seq 10 510); do nmcli con add type vlan con-name eth11.$i id $i dev eth11 ethernet.cloned-mac-address random  ipv4.dhcp-timeout 120 ipv4.may-fail no ipv4.dad-timeout 0 ipv6.method disable; sleep 0.25; done"
    Then "502" is visible with command "nmcli  device |grep eth11 |grep ' connected'| wc -l" in "500" seconds
    * Stop NM
    * Execute "for i in $(seq 10 510); do ip link del eth11.$i; done"
    * Reboot
    Then "502" is visible with command "nmcli  device |grep eth11 |grep ' connected'| wc -l" in "500" seconds

it goes well to 260 then it slows down significantly, but around 360 it even starts to go decrease

this are two-second steps of
while nmcli  device |grep ' connected' |grep eth11 |wc -l &&  sleep 2 ; do :;done
0
5
12
19
25
32
38
44
49
55
61
66
72
78
83
89
94
100
105
110
115
120
125
129
134
139
144
147
152
157
161
166
171
175
179
184
188
192
196
200
204
208
212
217
220
225
229
232
237
241
245
249
251
255
258
261
262
264
266
267
270
272
273
275
277
277
279
281
282
283
286
287
288
291
291
293
293
296
297
300
301
302
305
306
308
308
311
313
317
319
321
323
325
327
329
329
329
332
333
334
334
335
336
338
339
345
347
349
345
346
350
352
352
355
356
355
357
361
364
360
360
360
360
363
362
366
364
365
364
363
363
365
366
360
357
357
357
356
355
359
358
356
355
353
350
351
352
353
350
350
356
358
359
356
356
341
338
330



Version-Release number of selected component (if applicable):
NetworkManager-1.36.0-0.2.el8.x86_64

How reproducible:
always
Comment 1 Beniamino Galvani 2021-12-21 17:26:54 UTC 
I ran a script that creates many VLANs and waits for the DHCP
renewals, and I've compared the CPU time (from 'top' output) with
pre-next and current main branch.

The total CPU time after the merge of next branch is at least 20%
higher. From a comparison of callgrind traces, the new code tries to
update the DNS more often because NMDevice emits signal L3CD_CHANGED
at every DHCP renewal, even if nothing really changed.

I think this depends on nm_l3_config_data_equal() also comparing the
lifetimes of addresses and the force-commit flag of
addresses/routes.
Comment 3 Thomas Haller 2022-01-26 21:44:54 UTC 
this rhbz is about a CI test that frequently failed.

On the latest run (with the fix), it passes (https://desktopqe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/beaker-NetworkManager-main-veth-rhel8-upstream/874/artifact/artifacts/report_NetworkManager-ci_Test0479_vlan_create_500_vlans.html)

Marking as Tested.
Comment 4 Thomas Haller 2022-01-26 23:53:18 UTC 
back to new. See https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1066
Comment 5 Thomas Haller 2022-01-26 23:53:38 UTC 
assigned, I mean.
Comment 6 Thomas Haller 2022-01-28 14:50:17 UTC 
(In reply to Thomas Haller from comment #4)
> back to new. See
> https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/
> merge_requests/1066

Resolved. See details here: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/1066#note_1236782
https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/9d60cd2813fc8f9f89e7c0904cc17336a39e4592
Comment 9 Vladimir Benes 2022-02-16 12:03:51 UTC 
NMCI test working as expected.
Comment 12 errata-xmlrpc 2022-05-10 14:54:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (NetworkManager bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:1985
Note You need to log in before you can comment on or make changes to this bug.