Description of problem: I can not upgrade a Helm Chart that uses a library chart in the OpenShift dev console Version-Release number of selected component (if applicable): $ oc version Client Version: 4.5.7 Kubernetes Version: v1.19.0+1833054 tested on http://developers.redhat.com/developer-sandbox How reproducible: Always Steps to Reproduce: 1. Install a Helm chart using the Helm command line $ helm repo add wildfly https://jmesnil.github.io/wildfly-charts $ helm install todo-backend -f https://raw.githubusercontent.com/jmesnil/wildfly-charts/main/examples/todo-backend/todo-backend-bootable-jar.yaml wildfly/wildfly This will create a helm release (HR) named "todo-backend" 2. Go to the dev console and try to upgrade the Helm release (you don't need to change anything) This fails with the error message: --- An error occurred Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common --- The equivalent `helm upgrade` command works as expected: $ helm upgrade todo-backend -f https://raw.githubusercontent.com/jmesnil/wildfly-charts/main/examples/todo-backend/todo-backend-bootable-jar.yaml wildfly/wildfly Release "todo-backend" has been upgraded. Happy Helming! NAME: todo-backend LAST DEPLOYED: Thu Feb 25 17:34:50 2021 NAMESPACE: jmesnil1-dev STATUS: deployed REVISION: 2 TEST SUITE: None The actual Helm Chart to highlight this issue is https://github.com/jmesnil/wildfly-charts/releases/tag/wildfly-0.9.13 If you look at the tgz, the Chart.yaml define wildfly-common and it is in the charts/ directory: $ wget https://github.com/jmesnil/wildfly-charts/releases/download/wildfly-0.9.13/wildfly-0.9.13.tgz $ tar zxvf wildfly-0.9.13.tgz $ cat wildfly/Chart.yaml ... dependencies: - name: wildfly-common repository: https://jmesnil.github.io/wildfly-charts version: 0.1.1 ... $ cat wildfly/charts/wildfly-common/Chart.yaml ... apiVersion: v2 name: wildfly-common type: library version: 0.1.1 ... I'm not sure that this is not a bug in my Helm chart but the fact that the "helm upgrade" command succeeds while the upgrade in the OpenShift console fails is suspicious. And the error message does not align with the content of my chart. Please note that I had no issue to upgrade from the OpenShift console when my chart had no dependency. It started to fails once I added the dependency to the `wildfly-common` library chart. I suspect that this issue is a duplicate of 1933056 but my Helm chart is publicly available and might be better suited to reproduce the issue...
I have verified that this does not occur on OpenShift 4.7. So it seems it was fixed between 4.7 and the OpenShift version used by http://developers.redhat.com/developer-sandbox
Predrag is this a duplicate of #1933056?
> $ helm upgrade todo-backend -f What is Helm CLI version in use? If it is different from 3.2.1, could you please try with it, just to verify if the issue is not reproducible from CLI with that version?
(In reply to Christoph Jerolimov from comment #2) > Predrag is this a duplicate of #1933056? Yes, it looks like this.
I am using Helm 3.5.0: $ helm version version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"dirty", GoVersion:"go1.15.6"} I tried with Helm 3.2.1 but there were no issues from the CLI: ~/Downloads/darwin-amd64/helm version version.BuildInfo{Version:"v3.2.1", GitCommit:"fe51cd1e31e6a202cba7dead9552a6d418ded79a", GitTreeState:"clean", GoVersion:"go1.13.10"} $ ~/Downloads/darwin-amd64/helm install todo-backend -f https://raw.githubusercontent.com/jmesnil/wildfly-charts/main/examples/todo-backend/todo-backend-bootable-jar.yaml wildfly/wildfly NAME: todo-backend LAST DEPLOYED: Mon Mar 1 16:54:11 2021 NAMESPACE: jmesnil1-dev STATUS: deployed REVISION: 1 TEST SUITE: None $ ~/Downloads/darwin-amd64/helm upgrade todo-backend -f https://raw.githubusercontent.com/jmesnil/wildfly-charts/main/examples/todo-backend/todo-backend-bootable-jar.yaml wildfly/wildfly Release "todo-backend" has been upgraded. Happy Helming! NAME: todo-backend LAST DEPLOYED: Mon Mar 1 16:54:32 2021 NAMESPACE: jmesnil1-dev STATUS: deployed REVISION: 2 TEST SUITE: None
sorry, the issue is still happening in OpenShift 4.7 When I install the Helm release and then go to OpenShift dev console, I still have the same error: ``` Danger alert:An error occurred Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common ```
Hello! I have the same issue on 4.6 release. I also tested Openshift 4.7.0 (RedHat CodeReady Container) and I have the issue. Tested version: ./crc version CodeReady Containers version: 1.23.1+be17b141 OpenShift version: 4.7.0 (embedded in executable) Checked using Bitnami/mariadb chart. 1. Installed Helm Release from Web UI. 2. Waited until the Pods started 3. From Web UI tried to upgrade Helm Release (minor change - disabled Prometheus metrics) 4. Got "Failed to upgrade helm release...
*** Bug 1933056 has been marked as a duplicate of this bug. ***
@pknezevi could you please look into this?
Investigated this issue a bit more and found out that the issue arises due to not knowing the chart URL of the Helm release since it was installed using Helm CLI. So for any helm release installed using the helm CLI, there's no deterministic way of getting the chart URL in the UI and because of that the upgrade fails if the release was installed using CLI and upgraded using the UI.
For those where this fails, was the helm repo added to development catalog via custom resource definition? https://docs.openshift.com/container-platform/4.7/cli_reference/helm_cli/configuring-custom-helm-chart-repositories.html? Try to add your repo there in addition to just the client side helm client repo list.
I tried as you suggest and will report my findings below. However this is not a possible solution for us as adding a helm chart repository in this way requires cluster admin access. We want to use these charts as an entrypoint for OpenShift and some of our offerings (such as developer-sandbox) do not grant cluster admin access to the users. I added the chart repository with the YAML: ---8<-- apiVersion: helm.openshift.io/v1beta1 kind: HelmChartRepository metadata: name: wildfly-charts-repository spec: connectionConfig: url: https://docs.wildfly.org/wildfly-charts/ ---8<-- When I go to the Charts Catalog, I can see in the UI that it is display both WildFly 1.2.0 (which is an application chart) and Wildlfy Common 1.2.0 (which is an library chart). It seems incorrect to display a library chart in the list of installable charts... Then I install the wildfly chart (with the Yaml at https://raw.githubusercontent.com/wildfly/wildfly-charts/main/examples/microprofile-config/microprofile-config-app.yaml) I can verify that the application is properly up and running. Then when I try to upgrade the chart (without changing anything), I have the same error: Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common I tried to install the library-chart with the name "wildfly-common" but that does not change anything. It is still not possible to upgrade a chart release from the wildfly chart. Then I install the chart (at
I do think we have a flow problem (cli to GUI) that I will try to investigate further. I don't like either that the library chart shows as installable, and I don't like that when we install from cli we create a separate version of the helm chart not linked to the chart from the repo in the openshift catalog. I think we should make that experience smoother. Having said that here is how I got it working, which could help you as a workaround: Installed from helm cli using your suggested values file: helm install davp-wildfly wildfly/wildfly -f wild_fly_values.yaml All looks OK: dperaza@dperaza-mac ssm % helm status davp-wildfly NAME: davp-wildfly LAST DEPLOYED: Fri Apr 30 16:41:53 2021 NAMESPACE: davptest STATUS: deployed REVISION: 2 TEST SUITE: None dperaza@dperaza-mac ssm % kubectl get all | grep wildfly pod/davp-wildfly-1-build 0/1 Completed 0 50m pod/davp-wildfly-84c6958977-5ss2w 1/1 Running 0 13m pod/davp-wildfly-84c6958977-qzvxw 1/1 Running 0 45m service/davp-wildfly ClusterIP 10.217.5.150 <none> 8080/TCP 50m service/davp-wildfly-ping ClusterIP 10.217.4.26 <none> 8888/TCP 50m deployment.apps/davp-wildfly 2/2 2 2 50m replicaset.apps/davp-wildfly-84c6958977 2 2 2 45m replicaset.apps/davp-wildfly-856c5bd8d4 0 0 0 50m buildconfig.build.openshift.io/davp-wildfly Source Git.0.Final 1 build.build.openshift.io/davp-wildfly-1 Source Git@7f0eed5 Complete 50 minutes ago 5m33s I then tried to upgrade and got the same error you did. Then I noticed when I dropped down the version that in addition to the one selected by default (came with helm cli install) I see the one from the repo I added and you added to the catalog. If I use that version then it works. The issue is that if you change the version the previous values get reset as if you where installing from scratch (not nice). My solution to that was to copy the YAML from the default version then switch versions and then paste the yaml, then make whatever change. I would expect all this to be more automatic. I will investigate if this a known issue but this seems like a bug to me. We might need to split this into two bugs.
Some more info about this issue after some investigation: On the issue of wildfly-common showing in the catalog: We basically have not choice as this is listed in the index.yaml file of the repo: https://docs.wildfly.org/wildfly-charts/index.yaml. If you don't want wildfly-common to show it cannot be in index yaml. We also see the same behavior on helm cli: dperaza@dperaza-mac ssm % helm search repo wildfly NAME CHART VERSION APP VERSION DESCRIPTION bitnami/wildfly 10.0.2 23.0.2 Chart for Wildfly wildfly/wildfly 1.3.0 23.0 Build and Deploy WildFly applications on OpenShift wildfly/wildfly-common 1.3.0 1.0.0 A library chart for WildFly-based applications On the issue of install from CLI not able to upgrade. The flow I describe above as a "workaround" is actually the expected behavior. Helm releases installed from cli are seen from the console as releases without repo, since the repo was defined at the client side. However, if the repo is also added to OpenShift as a Custom Resource then you will be able to update the release using the repo defined in OpenShift. I would suggest this to issue to be close as "working as designed". I don't see anything we could do at the moment.
Thanks for investigating this issue. > On the issue of wildfly-common showing in the catalog: We basically have not choice as this is listed in the index.yaml file of the repo: https://docs.wildfly.org/wildfly-charts/index.yaml. If you don't want wildfly-common to show it cannot be in index yaml. I disagree. The library chart must be in the index.yaml so that it can be referenced by other charts. However it is properly listed as "type: library". From an UX perspective, the OpenShift dev consol should only list Chart of "type: application" that can actually be installed. > I would suggest this to issue to be close as "working as designed". I don't see anything we could do at the moment. If you can detect that a Helm release can not be upgraded from the dev console (such in my case), should the UX disable (or gray out) the "Upgrade" button so that we do not lead the users to perform an action that we know will fail?
The Helm Chart for WildFly has been added to the Red Hat Helm Chart repository (https://github.com/redhat-developer/redhat-helm-charts). the Helm Chart version is "1.3.0 / App Version 23.0 (Provided by Red Hat Helm Charts)" I use this chart to install a release (let's copy the YAML from https://raw.githubusercontent.com/wildfly/wildfly-charts/main/examples/microprofile-config/microprofile-config-app.yaml) When I try to upgrade it, it still fails with: Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common The helm cli is not involved at all here. I don't understand why it would not be able to upgrade as the helm tgz properly contains the wildly-common directory (https://github.com/redhat-developer/redhat-helm-charts/blob/master/charts/wildfly-1.3.0.tgz)
Sorry, I did not mention some information. I tested on OpenShift developer sandbox (https://developers.redhat.com/developer-sandbox) Something weird is happening with the chart version. When I about writing the YAML content, the chart version is "1.3.0 / App Version 23.0 (Provided by Red Hat Helm Charts)" Once the release is installed and I try to upgrade it, the chart version becomes "1.3.0 / App Version 23.0" and I don't have a dropdown menu to select the 'provided by Red Hat Helm Charts' variant so I can't use your workaround to upgrade.
@dperaza Can you please have another look? > However, if the repo is also added to OpenShift as a Custom Resource then you will be able to update the release using the repo defined in OpenShift. The wildfly chart is now by default in the OpenShift Dev console but we are still facing that issue. I did not found a workaround to be able to upgrade an Helm Release installed exclusively from the OpenShift Dev console.
On the Library chart thread... I don't think we should decide whether to show or not as a bug discussion/fix. The fact that from both helm cli and from dev console we can see the library chart tells me we need to bring it up to the community or to the console design team. The console team will alway try to provide same functionality that helm cli interface provides. BTW the library chart needs to be served under a site to be able to reference un the dependencies section and bring it down with helm dependency build. But they don't need to be in the index file. Like for example now you have it under redhat helm charts and the index does not show common library chart. I did noticed a difference in behavior from cli and ui. From cli I do get and error when I try to install: dperaza@dperaza-mac ssm % helm install wcommon wildfly/wildfly-common Error: library charts are not installable However from UI. I'm able to click install and even a release shows. That should be something we can fix as a bug. Let me spend some more time trying to reproduced the scenario you mentioned from redhat helm charts repo and will post here.
Correction on library chart being on the index.yaml. The library chart needs to be under an index.yaml to be able to do helm dependency build but once you bring the charts in the package. You can send that to another index.yaml like we did for redhat helm charts repo. Maybe that is part of the second problem you see. Will let you know what I find out.
Here are my findings on the actual upgrade. I can reproduce your issue. Even when installing from console (no cli involved) when I try to upgrade without changing the version. Lets say replica change. I get the error: An error occurred Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common However I can still workaround that issue by selecting from drop down this: "1.3.0 App Version 23.0 (Provided by Red Hat Helm Charts)" and using the value file from the installed version. When I select that it tells me this warning: Are you sure you want to change the chart version from 1.3.0 to 1.3.0--Red Hat Helm Charts? All data entered for version 1.3.0 will be reset. Almost like if they were two separate versions. I would expect to recognize that is the same version as it was installed right from console. I will bring this up to the dev console team and let you know what they think. I think is not great user experience. Please let me know if you can workaround as I did in the mean time.
I'm not able to use your workaround. I'm testing on OpenShift 4.8.0-0.ci-2021-05-25-001005 I have installed the Helm Chart from the "Red Hat Helm Charts" in the dev console. When I click on "Actions > Upgrade", it opens the "Upgrade Helm Release" page. The Chart version is "1.3.0 / App Version 23.0". However the dropdown is disabled and I can't select "1.3.0 App Version 23.0 (Provided by Red Hat Helm Charts)". How are you able to select it?
This fixes the library issue: https://github.com/openshift/console/pull/9035. Review is in progress.
OK I disabled the wildfly repo to only let red hat helm repo be the one serving that chart. I a see the same problem. I do not see a dropdown to be able to select the version. However when I enable the wildfly repo then I can see in the dropdown both the one from wildfly and the one from redhat. Can you confirm the same behavior on your end @jmesnil
Created attachment 1786845 [details] Dropdown when both repos are present This is what I see in my OpenShift Console
I confirm that I see the same results if I had a 2nd HelmChartRepository (based on WIldFly's own index.yaml at https://docs.wildfly.org/wildfly-charts/). This is not a suitable workaround as it requires the user to install this 2nd HelmChartRepository instead of relying only on the official Red Hat Helm Charts repository. I did not look at the OpenShift dev console so this is only my hypothesis but it seems that after installation, the dev console lost track that the Helm Chart was installed from the repository. It is installed from a "1.3.0 App Version 23.0 (Provided by Red Hat Helm Charts)" but the chart then identifies only its version with "1.3.0 App Version 23.0". It's very possible that the dropdown is disabled because there is only 1 listed version. I suspect that this workaround might eventually be possible if I release a new version (e.g. 1.3.1) as the drop down will then provide multiple choices letting me "reset" to the version from the Red Hat Helm Charts.
Thanks for verifying this @jmesnil. I have separated the Library chart issue to Bug 1964997 that way we can focus here on the Upgrade of charts with dependencies issue on this one.
Fix for this bug is being implemented here: https://github.com/openshift/console/pull/9161
I tried to verify the bugfix with OCP v4.8.0-0.nightly-2021-06-13-101614 with in-cluster devconsole as well as with the bridge of the latest release-4.8 branch of https://github.com/openshift/console repo (which includes the #9161 PR). With the https://docs.wildfly.org/wildfly-charts I still see the issue in: --- Danger alert:An error occurred Failed to upgrade helm release: found in Chart.yaml, but missing in charts/ directory: wildfly-common ---
@abai could you please check the issue Pavel is facing
Just to clarify something. The initial scenario described in this chart: Install from cli and try to upgrade from console is not being fixed here. There is not enough information in the data sent from the client to be able to read original chart url. The fix provided here is only taking care of the charts that were installed from console and upgraded from the console. If we need to create another bug with this specific scenario we can do that. @jmesnil can you verify with customer reporting if this will work for them: If you install from cli you will need to upgrade from cli and all will work If you install from console with this fix you will be able to upgrade from console. If you install from cli and for some reason still want to upgrade from console you will need to select the chart version from the dropdown even if it is the same version install. This will provide the chart information in addition to the release.
Just to be clear, I did not report this issue on behalf of a customer but as an engineer working on WildFly and EAP products. So from our own perspective, David's requirements are ok for us. * If we install the Helm Release using the CLI, we can use the CLI to manage the Helm Release * If we install the Helm Release using the OpenShift Dev Console, we can use the Dev Console to manage the Helm Release * a mix of using the CLI and Dev Console might have edge cases (such as "resetting" the chart version) and is not recommended
Thanks for clarification @jmesnil I thought you where helping a third party with this. Also thanks for confirming the scenarios. @pmacik fyi
Thx @dperaza and @jmesnil for clarification. I've tested with OCP v4.8.0-0.nightly-2021-06-14-145150 and Helm CLI v3.5.0 GA (3.5.0+6.el8). I confirmed that #9161 PR is in: $ oc get secrets sh.helm.release.v1.wildfly-from-ui.v2 -o json | jq -r .data.release | base64 --decode | base64 --decode | gunzip -c | jq -r .chart.metadata.annotations.chart_url https://github.com/wildfly/wildfly-charts/releases/download/wildfly-1.3.0/wildfly-1.3.0.tgz And I was able to successfully install and upgrade chart via both UI and CLI (not mixing the two) using these YAML configs: * app-v1.yaml = https://raw.githubusercontent.com/wildfly/wildfly-charts/main/examples/microprofile-config/microprofile-config-app.yaml) * app-v2.yaml = app-v1.yaml with updated {.deploy.env[name = "CONFIG_PROP"].value} field CLI: helm install wildfly-from-cli wildfly/wildfly -f https://raw.githubusercontent.com/wildfly/wildfly-charts/main/examples/microprofile-config/microprofile-config-app.yaml and helm upgrade wildfly-from-cli wildfly/wildfly -f app-v2.yaml UI: DevConsole->Add->Helm Chart->Wildfly->Install Helm Chart (name: wildfly-from-ui)->YAML view [content of app-v1.yaml]->Install and DevConsole->Helm->(wildfly-from-ui)->Actions->Upgrade->YAML view [content of app-v2.yaml]->Upgrade
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438
Hello Allen, I have a customer facing this issue in OCP 4.7 and is asking if there could be a backport for this bug?
Hi Silvia, Yes, the fix was planned to get backported to 4.7: https://github.com/openshift/console/pull/9299 and https://bugzilla.redhat.com/show_bug.cgi?id=1973707. I've added a comment and tagged the approver to get some attention. Hope this answers your question :)
Hi Allen, Thanks for quick reply. Do we have to follow the PR or the Errata for 4.7 would be available here?
I don't think they are linked directly under this BZ, so conversations related to 4.7 will most likely happen in the cloned BZ. But I will also monitor this ticket for future discussions.