Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2046775

Summary: ZTP gitops apps report errors and do not recover when openshift-gitops-operator.v1.4.1 is used
Product: OpenShift Container Platform Reporter: Marius Cornea <mcornea>
Component: Telco EdgeAssignee: melserng
Telco Edge sub component: ZTP QA Contact: Marius Cornea <mcornea>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: high    
Priority: unspecified CC: imiller, keyoung, melserng, vlaad
Version: 4.10   
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-26 15:15:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2058191    
Attachments:
Description Flags
openshift-gitops-repo-server-656fbcf646-fqf6t.log
none
openshift-gitops-application-controller-0.log none

Description Marius Cornea 2022-01-27 09:27:08 UTC 
Created attachment 1855951 [details]
openshift-gitops-repo-server-656fbcf646-fqf6t.log

Description of problem:

ZTP gitops apps report errors and do not recover when openshift-gitops-operator.v1.4.1 is used. Both clusters and policies apps report:

rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/policygentemplates --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)


Version-Release number of selected component (if applicable):
OCP 4.10.0-0.nightly-2022-01-25-023600
openshift-gitops-operator.v1.4.1
cnf-features-deploy bf23d6508ca085ee414c8b7d60d4ebc897fd36f5

How reproducible:
100%

Steps to Reproduce:
1. Create clusters and policies ZTP apps on a hub cluster with openshift-gitops-operator.v1.4.1 installed

Actual results:

clusters and policies apps report errors and do not recover as in BZ#2036729

rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/policygentemplates --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)

Expected results:

Apps do not report errors and proceed with cluster installation/configuration.

Additional info:

Attaching openshift-gitops-application-controller-0 and openshift-gitops-repo-server-656fbcf646-fqf6t logs.
Comment 1 Marius Cornea 2022-01-27 09:27:45 UTC 
Created attachment 1855970 [details]
openshift-gitops-application-controller-0.log
Comment 2 Marius Cornea 2022-01-27 09:44:19 UTC 
Note: with openshift-gitops-operator.v1.3.2 the same error manifests when the apps are created for the first time but they eventually recover after 5 minutes or after trigerring Hard Refresh as described in BZ#2036729.
Comment 3 Ian Miller 2022-01-27 17:14:47 UTC 
The current workaround is to pin the operator to v1.3.2. This CR can be used to install v1.3.2 of the operator:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
    #namespace: openshift-gitops
  namespace: openshift-operators
spec:
  name: openshift-gitops-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  startingCSV: openshift-gitops-operator.v1.3.2
  installPlanApproval: Manual


You may need to manually approve the installplan created for the subscription:
$ oc get installplan -n openshift-operators
NAME            CSV                                APPROVAL   APPROVED
install-7wdfv   openshift-gitops-operator.v1.3.2   Manual     false

Note the name of the install plan will vary, but you are looking for the one with CSV v1.3.2
oc edit installplan -n openshift-operators install-7wdfv

Set the "approved" field to true:
spec:
  approval: Manual
  approved: true
  clusterServiceVersionNames:
  - openshift-gitops-operator.v1.3.2
Comment 5 Marius Cornea 2022-02-24 09:36:56 UTC 
Could we please get the fix backported to release-4.10 branch?
Comment 6 Red Hat Bugzilla 2023-09-15 01:51:24 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days