Bug 1889882

Summary: Unable to load operator from catalog which worked in OCP 4.5
Product: OpenShift Container Platform Reporter: Krishna Raman <kraman>
Component: OLMAssignee: Ben Luddy <bluddy>
OLM sub component: OLM QA Contact: Jian Zhang <jiazha>
Status: CLOSED WORKSFORME Docs Contact:
Severity: high    
Priority: high CC: bluddy, krizza, nhale
Version: 4.6Keywords: Reopened, UpcomingSprint
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: x86_64   
OS: Linux   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-21 21:59:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Krishna Raman 2020-10-20 19:19:52 UTC
Description of problem:
Unable to load catalog which worked in OCP 4.5. 

Error in catalog-operator logs:

type: 'Warning' reason: 'ResolutionFailed' found more than one head for channel

Version-Release number of selected component (if applicable):
4.6-rc3 and 4.6-rc4

How reproducible:

Steps to Reproduce:
1. Add the following catalog sources:
  * docker.io/ibmcom/ibm-common-service-catalog:latest
  * ibmcom/datapower-operator-catalog:latest 
  * ibmcom/ibm-apiconnect-catalog:latest 
2. Create a new namespace and create a new subscription for "IBM API Connect"
3. Look at logs for catalog-operator.

Actual results:
Operator never installs and see error in catalog-operator re more than one head for channel

Expected results:
Installs common services, datapower and apiconnect operators

Additional info:

Comment 1 Ben Luddy 2020-10-20 21:41:53 UTC
I believe this is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1866861 -- ibmcom/datapower-operator-catalog:latest and ibmcom/ibm-apiconnect-catalog:latest both exhibit the same symptoms. Please rebuild those indices using the latest registry.redhat.io/openshift4/ose-operator-registry:v4.5 as your --binary-image and reopen this issue if that doesn't resolve the problem you're observing.

*** This bug has been marked as a duplicate of bug 1866861 ***

Comment 2 Krishna Raman 2020-10-20 23:07:04 UTC
I rebuilt the registry using registry.redhat.io/openshift4/ose-operator-registry:v4.5.0-202010081312.p0 but am still running into issues.

if I install the v2.0 channel (doesnt contain a dependencies.yaml, same bundles that worked in OCP 4.5) I get the following error:

E1020 22:21:24.376108       1 queueinformer_operator.go:290] sync "apic" failed: constraints not satisfiable: ibm-apiconnect has a dependency without any candidates to satisfy it, ibm-apiconnect is mandatory
I1020 22:21:24.376205       1 event.go:278] Event(v1.ObjectReference{Kind:"Namespace", Namespace:"", Name:"apic", UID:"e3746c53-b7cb-4f68-ae21-eec7971ad2a4", APIVersion:"v1", ResourceVersion:"59233", FieldPath:""}): type: 'Warning' reason: 'ResolutionFailed' constraints not satisfiable: ibm-apiconnect has a dependency without any candidates to satisfy it, ibm-apiconnect is mandatory

if I install the v2.1-eus channel (with a dependencies.yaml, new bundle) I get the following error:

E1020 22:20:40.069166       1 queueinformer_operator.go:290] sync "apic" failed: found more than one head for channel

for your reference, ive uploaded the 2.1.0 bundle to kraman/ibm-apiconnect-operator-bundle:2.1.0
and the catalog to kraman/ibm-apiconnect-catalog:build-1.

Comment 4 Ben Luddy 2020-10-21 21:59:02 UTC
Summary of the findings from our thread in Kubernetes Slack:

- Both the apiconnect and datapower index images need to be rebuilt with newer registry binaries. The error "found more than one head for channel" appears due to a bug that exists in registry-server in these images.
- In order to use the package/version dependencies introduced in 4.6, both the catalog containing the depender and the catalog containing the dependee should be built using the 4.6 binaries. 4.6-based index images should also work on <4.6 clusters as long as they do not make use of 4.6 features.
- There is a typo in the package/version dependency of ibm-apiconnect.v2.1.0: the packageName is listed as "ibm-datapower-operator", but the package available in the datapower catalog is named "datapower-operator", without the "ibm-" prefix.

I confirmed that I am able to resolve subscriptions for apiconnect's channels v1.0, v2.0, and v2.1-eus if these items are addressed.

Please feel free to follow up with any further questions on Slack and/or file new bugs for any other issues you run into.