Bug 2064034 - On an upgraded cluster NetworkAddonsConfig seems to be reconciling in a loop
Summary: On an upgraded cluster NetworkAddonsConfig seems to be reconciling in a loop
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Networking
Version: 4.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 4.11.0
Assignee: Ram Lavi
QA Contact: Yossi Segev
URL:
Whiteboard:
Depends On:
Blocks: 2069055
TreeView+ depends on / blocked
 
Reported: 2022-03-14 22:19 UTC by Debarati Basu-Nag
Modified: 2023-11-13 08:13 UTC (History)
4 users (show)

Fixed In Version: cluster-network-addons-operator v4.11.0-14
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2069055 (view as bug list)
Environment:
Last Closed: 2022-09-14 19:29:09 UTC
Target Upstream Version: v4.11.0
Embargoed:


Attachments (Terms of Use)
hco and cnao logs (891.63 KB, application/gzip)
2022-03-15 08:42 UTC, Ruth Netser
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker CNV-16915 0 None None None 2023-11-13 08:13:55 UTC
Red Hat Product Errata RHSA-2022:6526 0 None None None 2022-09-14 19:29:25 UTC

Comment 1 Ruth Netser 2022-03-15 08:42:29 UTC
Created attachment 1865971 [details]
hco and cnao logs

Comment 2 Ram Lavi 2022-03-17 08:21:53 UTC
Checking with the U/S equivalent of http://cnv-version-explorer.apps.cnv.engineering.redhat.com/BundleDetails?ver=v4.10.0-729
I do observe a ~1 minute reconcile rate for CNAO NetworkAddonsConfig, but the only changes I see in the NetworkAddonsConfig are the status `lastHeartbeatTime` changes, that evidently cause the resourceVerion change. AFAIU this is normal CNAO behavior.

Comment 4 Ram Lavi 2022-03-23 09:26:18 UTC
Can you update the target Version? I need to know if we should backport this

Comment 5 Yossi Segev 2022-04-11 18:53:57 UTC
Verified on:
CNV 4.11.0 (upgraded from 4.10.0)
HCO version: v4.11.0-192 (registry-proxy.engineering.redhat.com/rh-osbs/iib:207830)
cluster-network-addons-operator: v4.11.0-14

Reproduction scenario:
- Install CNV 4.10.0
- Upgrade to 4.11.0
- Check the NetwokAddonsConfig resource version in both the resource itself and HCO:
[cnv-qe-jenkins@n-yoss-411-ovn-q8gnd-executor ~]$ oc get hco kubevirt-hyperconverged -n openshift-cnv -o json | jq '.status.relatedObjects[]  | select(.kind == "NetworkAddonsConfig")'
{
  "apiVersion": "networkaddonsoperator.network.kubevirt.io/v1",
  "kind": "NetworkAddonsConfig",
  "name": "cluster",
  "resourceVersion": "9310525",
  "uid": "76c9eaf3-7635-4c51-bcca-cfab4af1130a"
}
[cnv-qe-jenkins@n-yoss-411-ovn-q8gnd-executor ~]$ oc get NetworkAddonsConfig cluster -o json | jq ".metadata"
{
  "creationTimestamp": "2022-04-11T17:11:04Z",
  "generation": 1,
  "labels": {
    "app": "kubevirt-hyperconverged",
    "app.kubernetes.io/component": "network",
    "app.kubernetes.io/managed-by": "hco-operator",
    "app.kubernetes.io/part-of": "hyperconverged-cluster",
    "app.kubernetes.io/version": "4.11.0"
  },
  "name": "cluster",
  "resourceVersion": "9310525",
  "uid": "76c9eaf3-7635-4c51-bcca-cfab4af1130a"
}
- Repeated 5 times, every few minutes, to see that the resource version (9310525) remains.
- Checked the NetworkAddonsConfig spec (oc get networkaddonsconfig cluster -o yaml) to see it doesn't contain the `lastHeartbeatTime` field.

Comment 7 Yossi Segev 2022-04-12 07:03:43 UTC
Moving back to ON_QA due to wrong upgrade method in verification.
I will re-verify the bug.

Comment 8 Yossi Segev 2022-04-15 06:18:01 UTC
Verified by repeating the scenario of comment #5 (this time with a valid upgrade process).

Comment 10 errata-xmlrpc 2022-09-14 19:29:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.11.0 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6526


Note You need to log in before you can comment on or make changes to this bug.