Bug 1744534

Summary: virt-operator pod restarts frequently
Product: Container Native Virtualization (CNV) Reporter: Kedar Bidarkar <kbidarka>
Component: VirtualizationAssignee: sgott
Status: CLOSED NOTABUG QA Contact: Kedar Bidarkar <kbidarka>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2.1.0CC: cnv-qe-bugs, ncredi, sgott
Target Milestone: ---   
Target Release: 2.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-28 12:11:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
virt-operator pod describe output none

Description Kedar Bidarkar 2019-08-22 11:12:45 UTC 
Description of problem:

virt-operator is restarting frequently 


[kbidarka@kbidarka-host osdc]$ oc get pods -n openshift-cnv | grep -i virt-operator
virt-operator-6c65f86c4f-gsdgd                  1/1       Running   47         17h
virt-operator-6c65f86c4f-jhcsj                  1/1       Running   58         18h



Version-Release number of selected component (if applicable):

with hco-bundle-registry:v2.1.0-15

How reproducible:

The virt-operator and virt-controller pods, go into CrashLoopBackOff and Error state respectively after a while, quite frequently.

virt-operator restarted almost 58 times.

Steps to Reproduce:
1.
2.
3.

Actual results:

Virt-operator pod restarts frequently

Expected results:

virt-operator pod should not restart 

Additional info:
Comment 1 Kedar Bidarkar 2019-08-22 11:14:28 UTC 
Created attachment 1606916 [details]
virt-operator pod describe output
Comment 3 sgott 2019-08-22 11:43:06 UTC 
Kedar, would you be able to add logs for the pod(s) in CrashLoopBackoff?
Comment 4 Marc Sluiter 2019-08-22 13:49:40 UTC 
I don't think this is related to virt-operator. I see a lot events like this for one of its pods, seems to be network related:

Warning  FailedMount  61m                  kubelet, host-172-16-0-22  MountVolume.SetUp failed for volume "kubevirt-operator-token-kp44c" : Get https://api-int.working.oc4:6443/api/v1/namespaces/openshift-cnv/secrets/kubevirt-operator-token-kp44c: read tcp 172.16.0.22:41276->10.0.96.108:6443: use of closed network connection

Also, e.g. cdi-operator has a similar amount of restarts...
Comment 6 Red Hat Bugzilla 2023-09-14 05:42:10 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days