Bug 2127730

Summary: ODF deployment fails on Airgapped cluster due to catalog source pods
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: akgunjal <akgunjal>
Component: odf-operatorAssignee: Nitin Goyal <nigoyal>
Status: CLOSED NOTABUG QA Contact: Martin Bukatovic <mbukatov>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.10CC: assingh, dahorak, hnallurv, jrivera, madam, muagarwa, ocs-bugs, odf-bz-bot, rugouvei, sagar.kumar3, sostapov
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-10-29 03:27:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
All logs of error none

Description akgunjal@in.ibm.com 2022-09-18 17:21:13 UTC 
Created attachment 1912723 [details]
All logs of error

Created attachment 1912723 [details]
All logs of error

Description of problem
======================

We are trying to deploy ODF 4.10 on airgapped cluster on IBM Cloud. We have done pruning and mirroring of images onto IBM registry. We created the catalog source and subscription but a job on openshift-marketplace fails with image related issues which needs to be fixed.

Version of all relevant components
==================================

ODF 4.10

Does this issue impact your ability to continue to work with the product?
=========================================================================

Yes. We cannot deploy ODF 4.10 due to the issue

Is there any workaround available to the best of your knowledge?
================================================================

No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
==============================================================================

2

Can this issue reproducible?
============================

yes

Can this issue reproduce from the UI?
=====================================

no


Steps to Reproduce
==================

1.Follow link https://access.redhat.com/documentation/en-us/openshift_container_platform/4.10/html/operators/administrator-tasks#olm-restricted-networks and setup the bundle and mirrors for private registry.
2.Deploy catalog source using mirrored images
3.Create subscription for ODF 4.10 version


Actual results
==============

Subscription is not in ready state with errors

Expected results
================


Additional info
===============
Comment 2 Ashish Singh 2022-09-19 05:40:58 UTC 
Hello Akash,

Can you attach must gather output file:  
  https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/4.10/html-single/troubleshooting_openshift_data_foundation/index#downloading-log-files-and-diagnostic-information_rhodf

Also, can you confirm if at the time of pruning redhat-operator index image, you included the following list of packages for OpenShift Data Foundation:
  - ocs-operator
  - odf-operator
  - mcg-operator
  - odf-csi-addons-operator
  - (optional)local-storage-operator (only for local storage deployments)
  - (optional)odf-multicluster-orchestrator (only for Regional disaster recovery configuration) 

Regards,
Ashish Singh
Comment 3 sagar.kumar3 2022-09-19 06:27:33 UTC 
Hi Ashish, we ran the below command while pruning
```
opm index prune \
    -f registry.redhat.io/redhat/redhat-operator-index:v4.10 \
    -p local-storage-operator,ocs-operator,mcg-operator,odf-operator,odf-csi-addons-operator \
    [-i registry.redhat.io/openshift4/ose-operator-registry:v4.10] \
    -t .icr.io/sagar-odf-private/redhat-operator-index:v4.10
```
Comment 4 akgunjal@in.ibm.com 2022-09-19 10:34:02 UTC 
Summary of the issue so far:
We had a sync session with Ashish and explained the setup. Ashish confirmed the setup is done correctly but its an issue with OLM probably. We also tried the steps given here https://access.redhat.com/solutions/6459071 but it did not resolve the issue.
Have raised the support ticket as sev-1 here https://access.redhat.com/support/cases/#/case/03317210 for further investigation from OCP team.