Bug 2036729 - Argo CD apps conditions report errors when created: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed
Summary: Argo CD apps conditions report errors when created: Error: loading generator ...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Telco Edge
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: ---
Assignee: Ian Miller
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-03 17:57 UTC by Marius Cornea
Modified: 2022-04-21 18:50 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-21 18:50:35 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
openshift-gitops-application-controller-0.log (102.02 KB, text/plain)
2022-01-03 17:57 UTC, Marius Cornea
no flags Details

Description Marius Cornea 2022-01-03 17:57:35 UTC
Created attachment 1848719 [details]
openshift-gitops-application-controller-0.log

Description of problem:

Argo CD apps conditions report errors when created for the first time on a freshly deployed hub cluster..

rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/siteconfig --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)

Version-Release number of selected component (if applicable):
GitOps 1.3.2
cnf-features-deploy f400f60f6edd67408ce56a5bae160c69fb5a376b

How reproducible:
100%

Steps to Reproduce:
1. Create clusters and policies apps
2. Check their status

Actual results:

[kni cnf-features-deploy]$ oc -n openshift-gitops get apps policies -o json | jq .status.conditions
[
  {
    "lastTransitionTime": "2022-01-03T17:45:34Z",
    "message": "rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/policygentemplates --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)",
    "type": "ComparisonError"
  }
]

[kni cnf-features-deploy]$ oc -n openshift-gitops get apps clusters -o json | jq .status.conditions
[
  {
    "lastTransitionTime": "2022-01-03T17:45:34Z",
    "message": "rpc error: code = Unknown desc = `kustomize build /tmp/http___registry.kni-qe-0.lab.eng.rdu2.redhat.com_3000_kni-qe_ztp-site-configs/siteconfig --enable-alpha-plugins` failed exit status 1: Error: loading generator plugins: unable to find plugin root - tried: ('<no value>'; homed in $KUSTOMIZE_PLUGIN_HOME), ('<no value>'; homed in $XDG_CONFIG_HOME), ('/.config/kustomize/plugin'; homed in default value of $XDG_CONFIG_HOME), ('/kustomize/plugin'; homed in home directory)",
    "type": "ComparisonError"
  }
]


Expected results:

No errors reported by the apps conditions

Additional info:

After around 5 minutes the apps conditions recover and report sync OK.

Attaching openshift-gitops-application-controller-0 log.

Comment 1 Ian Miller 2022-04-04 17:37:24 UTC
In order to properly handle the generated content from either SiteConfig or PolicyGenTemplate CRs the argocd application must be patched to inject the proper generator for those CRs. Once the application is patched it can take 1-2 minutes for the argocd repo server pod to restart with the new generator in place. If the git repository is already populated with content when the application is initially created (but not yet patched) you will get generator errors until the pod has restarted.  

@mcornea In the environment where this issue occurred was the application created while pointing at a git repository which was populated with content?

Comment 2 Marius Cornea 2022-04-05 08:29:21 UTC
(In reply to Ian Miller from comment #1)
> In order to properly handle the generated content from either SiteConfig or
> PolicyGenTemplate CRs the argocd application must be patched to inject the
> proper generator for those CRs. Once the application is patched it can take
> 1-2 minutes for the argocd repo server pod to restart with the new generator
> in place. If the git repository is already populated with content when the
> application is initially created (but not yet patched) you will get
> generator errors until the pod has restarted.  
> 
> @mcornea In the environment where this issue occurred was the
> application created while pointing at a git repository which was populated
> with content?

Yes, the git repository is already populated at the time the app gets created.

Please note that I haven't noticed this issue lately so I suggest closing this issue for now, unless you think we should perhaps document this behavior.

Thanks!

Comment 3 Ian Miller 2022-04-21 18:50:35 UTC
Closing based on comment 2. If the issue recurs please reopen.


Note You need to log in before you can comment on or make changes to this bug.