Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2149851

Summary: Optimize tests to shorten CI times
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Ales Musil <amusil>
Component: ovn22.12Assignee: Ales Musil <amusil>
Status: CLOSED UPSTREAM QA Contact: Jianlin Shi <jishi>
Severity: medium Docs Contact:
Priority: medium    
Version: FDP 22.ECC: ctrautma, jiji, mmichels
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-13 13:39:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Tests ordered by duration none

Description Ales Musil 2022-12-01 07:24:54 UTC 
Created attachment 1929005 [details]
Tests ordered by duration

Description of problem:
We have several tests that take a of time. After some measurements I have identified ~15 (~45 considering permutations) tests that together take ~35% of overall test time. 

By optimizing those we could cut the overall CI time at least for the unit tests.

In the attachment is full list of all tests order by the duration.
Down below are tests that are running longer than 5 seconds. 

One thing to keep in mind is that the duration may vary between environments
but it should be proportional.

----------------------------------
Larger than 5 seconds
----------------------------------
1020. northd-parallelization runtime -- ovn-northd -- parallelization=yes (ovn-northd.at:7428): Ok (17.99s)
1021. northd-parallelization runtime -- ovn-northd -- parallelization=no (ovn-northd.at:7428): Ok (17.89s)
399. ACLs on Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:16466): Ok (17.792s)
400. ACLs on Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:16466): Ok (17.649s)
401. ACLs on Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:16466): Ok (17.609s)
402. ACLs on Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:16466): Ok (17.455s)
565. interconnection -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:24498): Ok (15.347s)
566. interconnection -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:24498): Ok (15.271s)
563. interconnection -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:24498): Ok (15.234s)
564. interconnection -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:24498): Ok (15.089s)
1040. Check default drop -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn-northd.at:8189): Ok (12.628s)
1038. Check default drop -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn-northd.at:8189): Ok (12.608s)
1039. Check default drop -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn-northd.at:8189): Ok (12.57s)
1041. Check default drop -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn-northd.at:8189): Ok (12.532s)
119. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:4579): Ok (10.736s)
121. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:4579): Ok (10.692s)
122. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:4579): Ok (10.598s)
120. 3 HVs, 3 LS, 3 lports/LS, 1 LR -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:4579): Ok (10.596s)
24. 4-term numeric expression normalization (ovn.at:624): Ok (8.756s)
397. Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:16246): Ok (7.77s)
396. Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:16246): Ok (7.733s)
398. Port Groups -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:16246): Ok (7.685s)
395. Port Groups -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:16246): Ok (7.477s)
502. IGMP snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:20995): Ok (5.905s)
501. IGMP snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:20995): Ok (5.834s)
507. MLD snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:21717): Ok (5.715s)
508. MLD snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:21717): Ok (5.692s)
355. options:activation-strategy for logical port -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:15011): Ok (5.687s)
356. options:activation-strategy for logical port -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:15011): Ok (5.607s)
500. IGMP snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:20995): Ok (5.515s)
357. options:activation-strategy for logical port -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:15011): Ok (5.503s)
509. MLD snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:21717): Ok (5.493s)
499. IGMP snoop/querier/relay -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:20995): Ok (5.466s)
358. options:activation-strategy for logical port -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:15011): Ok (5.457s)
510. MLD snoop/querier/relay -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:21717): Ok (5.426s)
27. 5-term numeric expression normalization (ovn.at:642): Ok (5.326s)
19. 3-term numeric expression simplification (ovn.at:570): Ok (5.261s)
678. ARP replies for SNAT external ips -- ovn-northd -- parallelization=yes -- ovn_monitor_all=no (ovn.at:28040): Ok (5.184s)
677. ARP replies for SNAT external ips -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:28040): Ok (5.12s)
50. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=no -- ovn_monitor_all=no (ovn.at:2303): Ok (5.045s)
49. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=no -- ovn_monitor_all=yes (ovn.at:2303): Ok (5.045s)
47. 3 HVs, 1 LS, 3 lports/HV -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:2303): Ok (5.029s)
459. router - check packet length - icmp defrag -- ovn-northd -- parallelization=yes -- ovn_monitor_all=yes (ovn.at:19051): Ok (5.016s)
1169. ovn-controller - port binding type change handling (ovn-controller.at:534): Ok (5.008s)
Total duration: 413.041s
Comment 1 Ales Musil 2022-12-01 09:19:45 UTC 
For some reason the times are completely off for system tests. So we would require different method to analyze them.
Comment 2 Ales Musil 2022-12-15 14:54:43 UTC 
After some discussion we will tag those slowest one that don't have obvious "fix" with "slowtest".
Comment 3 Ales Musil 2023-01-10 09:45:31 UTC 
slowtest mark: https://patchwork.ozlabs.org/project/ovn/patch/20230110083502.2656502-1-amusil@redhat.com/
some optimizations: https://patchwork.ozlabs.org/project/ovn/list/?series=332641
Comment 4 Ales Musil 2023-01-13 13:39:00 UTC 
All relevant patches have been merged closing as UPSTREAM.