Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2102088

Summary:	CatalogSourcesUnhealthy error in subscription When upgrading ptp-operator from 4.10 to 4.11
Product:	OpenShift Container Platform	Reporter:	Bonnie Block <bblock>
Component:	Networking	Assignee:	Jack Ding <jacding>
Networking sub component:	ptp	QA Contact:	obochan <obochan>
Status:	CLOSED WONTFIX	Docs Contact:
Severity:	low
Priority:	medium	CC:	aputtur, bblock, jacding, yliu1
Version:	4.11
Target Milestone:	---
Target Release:	4.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-10-05 19:41:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Bonnie Block 2022-06-29 10:17:28 UTC

Description of problem:

When upgrading ptp-operator from 4.10 to 4.11, openshift-ptp subscription reports the error, "no channel heads (entries not replaced by another entry) found in channel "stable" of package "ptp-operator"

The operator does update correctly, however.

sriov and cluster-logging were upgraded in the same manner without this error.

Version-Release number of selected component (if applicable):
4.11

How reproducible:
Changed catalogsource from 4.10 to 4.11, approved installplan, checked csv and subscriptions

Steps to Reproduce:
1. oc edit catalogsources.operators.coreos.com -n openshift-marketplace redhat-operators
>#change image from redhat-operators:4.10 to redhat-operators:4.11
>spec:
>  displayName: Red Hat Operators Catalog
>  image: registry.kni-qe-10.lab.eng.tlv2.redhat.com:5000/olm/redhat-operators:v4.11

2. oc get installplans.operators.coreos.com -A
>NAMESPACE                              NAME            CSV                                          APPROVAL   APPROVED
...
>openshift-ptp                          install-tbthz   ptp-operator.4.10.0-202205121158             Manual     true
>openshift-ptp                          install-xgcws   ptp-operator.4.11.0-202206250809             Manual     false

3. oc edit installplans.operators.coreos.com -n openshift-ptp install-xgcws
>#approve installplan
>spec:
>  approval: Manual
>  approved: true
>  clusterServiceVersionNames:
>  - ptp-operator.4.11.0-202206250809

4. oc get csv -n openshift-ptp ptp-operator.4.11.0-202206250809
>NAME                               DISPLAY        VERSION               REPLACES                           PHASE
>ptp-operator.4.11.0-202206250809   PTP Operator   4.11.0-202206250809   ptp->operator.4.10.0-202205121158   Succeeded

5. oc get subscriptions.operators.coreos.com -n openshift-ptp -o yaml
...
>   conditions:
>    - lastTransitionTime: "2022-06-28T20:23:36Z"
>      message: all available catalogsources are healthy
>      reason: AllCatalogSourcesHealthy
>      status: "False"
>      type: CatalogSourcesUnhealthy
>    - message: no channel heads (entries not replaced by another entry) found in channel
>        "stable" of package "ptp-operator"
>      reason: ErrorPreventedResolution
>      status: "True"
>      type: ResolutionFailed
>    currentCSV: ptp-operator.4.11.0-202206250809

Actual results:
subscriptions.operators.coreos.com -n openshift-ptp error:
>    - message: no channel heads (entries not replaced by another entry) found in channel
>        "stable" of package "ptp-operator"


Expected results:
AllCatalogSourcesHealthy status:True

Additional info:
We were able to repeat this scenario with the same result.

must-gather.tar.gz:
https://drive.google.com/file/d/1DTtr2-zCczN-bBlp992sG1-4g_shQ3sA/view?usp=sharing

Comment 2 Jack Ding 2022-08-22 14:22:51 UTC

NOTE: The impact of this issue is low. With errors seen in this bug, the upgrade still completes successfully and ptp-operator functions properly after the upgrade.

Comment 3 Jack Ding 2022-09-30 15:19:52 UTC

When ART builds the bundle image the skipRange regexp should exactly match expected regexp in art.yaml. Currently the upper bound of skipRange
is not auto-updating because we use single quote in the skipRange string.

'>=4.3.0-0 <4.10.0'

For example, for the operators with skipRange in single quotes, the upper bound of skipRange is not updated in operator catalog pod:

[jacding@fedora Downloads]$ sqlite3 index.db 'select skiprange from operatorbundle WHERE name="nfd.4.10.0-202208241855"'
>=4.6.0 <4.10.0
[jacding@fedora Downloads]$ sqlite3 index.db 'select skiprange from operatorbundle WHERE name="ptp-operator.4.10.0-202208241855"'
>=4.3.0-0 <4.10.0

In contracts, for the operators with skipRange in double quotes, the upper bound of skipRange are updated to the current CSV in operator catalog pod:

[jacding@fedora Downloads]$ sqlite3 index.db 'select skiprange from operatorbundle WHERE name="clusterresourceoverride-operator.4.10.0-202208241855"'
>=4.3.0 <4.10.0-202208241855
[jacding@fedora Downloads]$ sqlite3 index.db 'select skiprange from operatorbundle WHERE name="local-storage-operator.4.10.0-202209080237"'
>=4.3.0 <4.10.0-202209080237

Comment 4 Jack Ding 2022-09-30 15:21:07 UTC

fix to master branch https://github.com/openshift/ptp-operator/pull/245
will propagate to 4.10 once it is reviewed.

Comment 5 Jack Ding 2022-10-05 19:41:11 UTC

closing this and replaced by https://issues.redhat.com//browse/OCPBUGS-2042 since OCP bugs are tracked in Jira OCPBUGS project now.