Bug 2004299

Summary: virt-handler goroutine count increases over time
Product: Container Native Virtualization (CNV) Reporter: Sarah Bennert <sbennert>
Component: VirtualizationAssignee: Itamar Holder <iholder>
Status: CLOSED NOTABUG QA Contact: Kedar Bidarkar <kbidarka>
Severity: high Docs Contact:
Priority: high    
Version: 4.8.2CC: acardace, cnv-qe-bugs, dholler, fdeutsch, ipinto, kbidarka, sgott
Target Milestone: ---Keywords: Scale
Target Release: 4.12.0Flags: sbennert: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2004295 Environment:
Last Closed: 2022-10-12 11:02:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2004295, 2004300    
Bug Blocks:    

Description Sarah Bennert 2021-09-14 23:43:52 UTC 
+++ This bug was initially created as a clone of Bug #2004295 +++

Tracking for 4.9.z

Description of problem:
  Opened as a follow-up to https://bugzilla.redhat.com/show_bug.cgi?id=1996660
  Observed an additional slower goroutine leak.

How reproduceible:
  Very reproduceible, must be run for an extended time to observe increase.

Steps to Reproduce:
  Allow for any stabilization between each step.
  While observing goroutines metric on virt-handler,
    1. Create and Start a batch of 100-300 VMs
    2. Delete all VMs in batch
    3. Repeat

Expected Results:
  Goroutines do not increase between VM deployments after initial deployment

Actual Results:
  Over the course of approximately 48 hours, goroutine count increased between 14-28 per virt-handler.
Comment 2 Geetika Kapoor 2022-06-08 11:54:10 UTC 
*** Bug 2004295 has been marked as a duplicate of this bug. ***
Comment 3 Geetika Kapoor 2022-06-08 11:54:47 UTC 
*** Bug 2004300 has been marked as a duplicate of this bug. ***
Comment 4 sgott 2022-08-03 15:24:04 UTC 
removing the target version to cause us to re-triage this -- to see if it still applies.
Comment 5 sgott 2022-08-10 12:26:04 UTC 
Kedar, does this sitll manifest with current releases?
Comment 6 Kedar Bidarkar 2022-09-07 12:35:03 UTC 
Sarah, Do we still see this issue with latest 4.11.0 ?
Comment 7 Itamar Holder 2022-10-12 11:00:51 UTC 
I confirm that this behavior no longer occurs on 4.11 clusters.