Bug 2046775 - ZTP gitops apps report errors and do not recover when openshift-gitops-operator.v1.4.1 is used
Summary: ZTP gitops apps report errors and do not recover when openshift-gitops-operat...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Telco Edge
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.11.0
Assignee: melserng
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks: 2058191
TreeView+ depends on / blocked
 
Reported: 2022-01-27 09:27 UTC by Marius Cornea
Modified: 2023-09-15 01:51 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-26 15:15:55 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
openshift-gitops-repo-server-656fbcf646-fqf6t.log (37.35 KB, text/plain)
2022-01-27 09:27 UTC, Marius Cornea
no flags Details
openshift-gitops-application-controller-0.log (20.15 KB, text/plain)
2022-01-27 09:27 UTC, Marius Cornea
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift-kni cnf-features-deploy pull 973 0 None Merged Bug 2046775: ztp update Argocd patch 2022-07-21 16:41:38 UTC

Description Marius Cornea 2022-01-27 09:27:08 UTC
Created attachment 1855951 [details]
openshift-gitops-repo-server-656fbcf646-fqf6t.log

Description of problem:

ZTP gitops apps report errors and do not recover when openshift-gitops-operator.v1.4.1 is used. Both clusters and policies apps report:

rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/policygentemplates --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)


Version-Release number of selected component (if applicable):
OCP 4.10.0-0.nightly-2022-01-25-023600
openshift-gitops-operator.v1.4.1
cnf-features-deploy bf23d6508ca085ee414c8b7d60d4ebc897fd36f5

How reproducible:
100%

Steps to Reproduce:
1. Create clusters and policies ZTP apps on a hub cluster with openshift-gitops-operator.v1.4.1 installed

Actual results:

clusters and policies apps report errors and do not recover as in BZ#2036729

rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/policygentemplates --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)

Expected results:

Apps do not report errors and proceed with cluster installation/configuration.

Additional info:

Attaching openshift-gitops-application-controller-0 and openshift-gitops-repo-server-656fbcf646-fqf6t logs.

Comment 1 Marius Cornea 2022-01-27 09:27:45 UTC
Created attachment 1855970 [details]
openshift-gitops-application-controller-0.log

Comment 2 Marius Cornea 2022-01-27 09:44:19 UTC
Note: with openshift-gitops-operator.v1.3.2 the same error manifests when the apps are created for the first time but they eventually recover after 5 minutes or after trigerring Hard Refresh as described in BZ#2036729.

Comment 3 Ian Miller 2022-01-27 17:14:47 UTC
The current workaround is to pin the operator to v1.3.2. This CR can be used to install v1.3.2 of the operator:

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: openshift-gitops-operator
    #namespace: openshift-gitops
  namespace: openshift-operators
spec:
  name: openshift-gitops-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  startingCSV: openshift-gitops-operator.v1.3.2
  installPlanApproval: Manual


You may need to manually approve the installplan created for the subscription:
$ oc get installplan -n openshift-operators
NAME            CSV                                APPROVAL   APPROVED
install-7wdfv   openshift-gitops-operator.v1.3.2   Manual     false

Note the name of the install plan will vary, but you are looking for the one with CSV v1.3.2
oc edit installplan -n openshift-operators install-7wdfv

Set the "approved" field to true:
spec:
  approval: Manual
  approved: true
  clusterServiceVersionNames:
  - openshift-gitops-operator.v1.3.2

Comment 5 Marius Cornea 2022-02-24 09:36:56 UTC
Could we please get the fix backported to release-4.10 branch?

Comment 6 Red Hat Bugzilla 2023-09-15 01:51:24 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days


Note You need to log in before you can comment on or make changes to this bug.