Bug 2064034

Summary: On an upgraded cluster NetworkAddonsConfig seems to be reconciling in a loop
Product: Container Native Virtualization (CNV) Reporter: Debarati Basu-Nag <dbasunag>
Component: NetworkingAssignee: Ram Lavi <ralavi>
Status: CLOSED ERRATA QA Contact: Yossi Segev <ysegev>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.10.0CC: cnv-qe-bugs, phoracek, ralavi, ysegev
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: cluster-network-addons-operator v4.11.0-14 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2069055 (view as bug list) Environment:
Last Closed: 2022-09-14 19:29:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: v4.11.0
Embargoed:
Bug Depends On:    
Bug Blocks: 2069055    
Attachments:
Description Flags
hco and cnao logs none

Comment 1 Ruth Netser 2022-03-15 08:42:29 UTC
Created attachment 1865971 [details]
hco and cnao logs

Comment 2 Ram Lavi 2022-03-17 08:21:53 UTC
Checking with the U/S equivalent of http://cnv-version-explorer.apps.cnv.engineering.redhat.com/BundleDetails?ver=v4.10.0-729
I do observe a ~1 minute reconcile rate for CNAO NetworkAddonsConfig, but the only changes I see in the NetworkAddonsConfig are the status `lastHeartbeatTime` changes, that evidently cause the resourceVerion change. AFAIU this is normal CNAO behavior.

Comment 4 Ram Lavi 2022-03-23 09:26:18 UTC
Can you update the target Version? I need to know if we should backport this

Comment 5 Yossi Segev 2022-04-11 18:53:57 UTC
Verified on:
CNV 4.11.0 (upgraded from 4.10.0)
HCO version: v4.11.0-192 (registry-proxy.engineering.redhat.com/rh-osbs/iib:207830)
cluster-network-addons-operator: v4.11.0-14

Reproduction scenario:
- Install CNV 4.10.0
- Upgrade to 4.11.0
- Check the NetwokAddonsConfig resource version in both the resource itself and HCO:
[cnv-qe-jenkins@n-yoss-411-ovn-q8gnd-executor ~]$ oc get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq '.status.relatedObjects[]  | select(.kind == "NetworkAddonsConfig")'
{
  "apiVersion": "networkaddonsoperator.network.kubevirt.io/v1",
  "kind": "NetworkAddonsConfig",
  "name": "cluster",
  "resourceVersion": "9310525",
  "uid": "76c9eaf3-7635-4c51-bcca-cfab4af1130a"
}
[cnv-qe-jenkins@n-yoss-411-ovn-q8gnd-executor ~]$ oc get NetworkAddonsConfig cluster -o json | jq ".metadata"
{
  "creationTimestamp": "2022-04-11T17:11:04Z",
  "generation": 1,
  "labels": {
    "app": "kubevirt-hyperconverged",
    "app.kubernetes.io/component": "network",
    "app.kubernetes.io/managed-by": "hco-operator",
    "app.kubernetes.io/part-of": "hyperconverged-cluster",
    "app.kubernetes.io/version": "4.11.0"
  },
  "name": "cluster",
  "resourceVersion": "9310525",
  "uid": "76c9eaf3-7635-4c51-bcca-cfab4af1130a"
}
- Repeated 5 times, every few minutes, to see that the resource version (9310525) remains.
- Checked the NetworkAddonsConfig spec (oc get networkaddonsconfig cluster -o yaml) to see it doesn't contain the `lastHeartbeatTime` field.

Comment 7 Yossi Segev 2022-04-12 07:03:43 UTC
Moving back to ON_QA due to wrong upgrade method in verification.
I will re-verify the bug.

Comment 8 Yossi Segev 2022-04-15 06:18:01 UTC
Verified by repeating the scenario of comment #5 (this time with a valid upgrade process).

Comment 10 errata-xmlrpc 2022-09-14 19:29:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.11.0 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6526