Bug 2149851 - Optimize tests to shorten CI times
Summary: Optimize tests to shorten CI times
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Red Hat Enterprise Linux Fast Datapath
Classification: Red Hat
Component: ovn22.12
Version: FDP 22.E
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Ales Musil
QA Contact: Jianlin Shi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-12-01 07:24 UTC by Ales Musil
Modified: 2023-01-13 13:39 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-01-13 13:39:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Tests ordered by duration (151.44 KB, text/plain)
2022-12-01 07:24 UTC, Ales Musil
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FD-2520 0 None None None 2022-12-01 07:31:49 UTC

Description Ales Musil 2022-12-01 07:24:54 UTC
Created attachment 1929005 [details]
Tests ordered by duration

Description of problem:
We have several tests that take a of time. After some measurements I have identified ~15 (~45 considering permutations) tests that together take ~35% of overall test time. 

By optimizing those we could cut the overall CI time at least for the unit tests.

In the attachment is full list of all tests order by the duration.
Down below are tests that are running longer than 5 seconds. 

One thing to keep in mind is that the duration may vary between environments
but it should be proportional.

----------------------------------
Larger than 5 seconds
----------------------------------
1020. northd-parallelization runtime -- ovn-northd -- parallelization=yes (ovn-northd.at:7428): Ok (17.99s)
1021. northd-parallelization runtime -- ovn-northd -- parallelization=no (ovn-northd.at:7428): Ok (17.89s)
399. ACLs on Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:16466): Ok (17.792s)
400. ACLs on Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:16466): Ok (17.649s)
401. ACLs on Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:16466): Ok (17.609s)
402. ACLs on Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:16466): Ok (17.455s)
565. interconnection -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:24498): Ok (15.347s)
566. interconnection -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:24498): Ok (15.271s)
563. interconnection -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:24498): Ok (15.234s)
564. interconnection -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:24498): Ok (15.089s)
1040. Check default drop -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn-northd.at:8189): Ok (12.628s)
1038. Check default drop -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn-northd.at:8189): Ok (12.608s)
1039. Check default drop -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn-northd.at:8189): Ok (12.57s)
1041. Check default drop -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn-northd.at:8189): Ok (12.532s)
119. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:4579): Ok (10.736s)
121. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:4579): Ok (10.692s)
122. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:4579): Ok (10.598s)
120. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:4579): Ok (10.596s)
24. 4-term numeric expression normalization (ovn.at:624): Ok (8.756s)
397. Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:16246): Ok (7.77s)
396. Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:16246): Ok (7.733s)
398. Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:16246): Ok (7.685s)
395. Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:16246): Ok (7.477s)
502. IGMP snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:20995): Ok (5.905s)
501. IGMP snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:20995): Ok (5.834s)
507. MLD snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:21717): Ok (5.715s)
508. MLD snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:21717): Ok (5.692s)
355. options:activation-strategy for logical port -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:15011): Ok (5.687s)
356. options:activation-strategy for logical port -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:15011): Ok (5.607s)
500. IGMP snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:20995): Ok (5.515s)
357. options:activation-strategy for logical port -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:15011): Ok (5.503s)
509. MLD snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:21717): Ok (5.493s)
499. IGMP snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:20995): Ok (5.466s)
358. options:activation-strategy for logical port -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:15011): Ok (5.457s)
510. MLD snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:21717): Ok (5.426s)
27. 5-term numeric expression normalization (ovn.at:642): Ok (5.326s)
19. 3-term numeric expression simplification (ovn.at:570): Ok (5.261s)
678. ARP replies for SNAT external ips -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:28040): Ok (5.184s)
677. ARP replies for SNAT external ips -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:28040): Ok (5.12s)
50. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:2303): Ok (5.045s)
49. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:2303): Ok (5.045s)
47. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:2303): Ok (5.029s)
459. router - check packet length - icmp defrag -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:19051): Ok (5.016s)
1169. ovn-controller - port binding type change handling (ovn-controller.at:534): Ok (5.008s)
Total duration: 413.041s

Comment 1 Ales Musil 2022-12-01 09:19:45 UTC
For some reason the times are completely off for system tests. So we would require different method to analyze them.

Comment 2 Ales Musil 2022-12-15 14:54:43 UTC
After some discussion we will tag those slowest one that don't have obvious "fix" with "slowtest".

Comment 4 Ales Musil 2023-01-13 13:39:00 UTC
All relevant patches have been merged closing as UPSTREAM.


Note You need to log in before you can comment on or make changes to this bug.