Bug 1996638
| Summary: | Helm operator manager container restart when CR is creating&deleting | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Fan Jia <jfan> |
| Component: | Operator SDK | Assignee: | Bryce Palmer <bpalmer> |
| Status: | CLOSED ERRATA | QA Contact: | Fan Jia <jfan> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 4.9 | CC: | aos-bugs, bpalmer, jesusr, jlanford |
| Target Milestone: | --- | ||
| Target Release: | 4.11.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | No Doc Update | |
| Doc Text: |
N/A
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-08-10 10:36:53 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Comment 3
Jesus M. Rodriguez
2022-06-06 18:51:40 UTC
The expected and actual results from the original comment are flipped. We expect it to work and the actual results is a failure. When taking a look at this and testing with the latest ocp-release-operator-sdk, following the steps results in a controller-manager log output that contains the logged error.
Output of `kubectl logs nginx-controller-manager-6fd9f84dbc-x2g8c`
```
{"level":"error","ts":1654551120.0068138,"logger":"controller.nginx-controller","msg":"Reconciler error","name":"nginx-sample","namespace":"default","error":"Operation cannot be fulfilled on nginxes.helmdemo.example.com \"nginx-sample\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\toperator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}
```
Pod does not restart, seen by output of `kubectl get pods`:
```
NAME READY STATUS RESTARTS AGE
3a5a1bcea941fcf2724f8c6000df5fab6f6ba6972c7b9d70018f8b9888mms9j 0/1 Completed 0 4m15s
docker-io-bpalmer-nginx-bundle-v4-9 1/1 Running 0 4m24s
nginx-controller-manager-6fd9f84dbc-x2g8c 2/2 Running 0 4m3s
nginx-sample-685f77dcbc-84mf6 1/1 Running 0 3m37s
nginx-sample-685f77dcbc-tbsld 1/1 Running 0 3m37s
```
I tested this using the latest commit on the master branch of https://github.com/openshift/ocp-release-operator-sdk and OpenShift 4.11.0-fc.0 using the following steps: 1. `operator-sdk version`: ``` operator-sdk version: "v1.22.0-ocp", commit: "8e224f3704e7c9fdd84aac3a8f76364c68249372", kubernetes version: "v1.24.1", go version: "go1.18.3", GOOS: "linux", GOARCH: "amd64" ``` 2. `mkdir ocp-latest-nginx && cd ocp-latest-nginx` 3. `operator-sdk init --plugins=helm --domain=example.com --group=helmdemo --version=v1 --kind=Nginx` 4. edit *config/samples/helmdemo_v1_nginx.yaml* ``` apiVersion: helmdemo.example.com/v1 kind: Nginx metadata: name: nginx-sample ... spec: ... replicaCount: 2 service: port: 8080 ``` 5. edit `Dockerfile` ``` FROM quay.io/operator-framework/helm-operator:v1.22 ``` (Used the upstream image for verification that it works as I am not sure the downstream image has been updated to use OSDK v1.22.0 at this time, but the v4.11 should be updated soon as the downstream repo has been synced by: https://github.com/openshift/ocp-release-operator-sdk/pull/264 ) 6. generate the bundle image: - `make docker-build docker-push IMG=docker.io/bpalmer/ocp-latest-nginx-operator:v4.11` - edit Makefile by setting the IMG to be docker.io/bpalmer/ocp-latest-nginx-operator:v4.11 ``` IMG ?= docker.io/bpalmer/ocp-latest-nginx-operator:v4.11 ``` - `make bundle` - `make bundle-build bundle-push BUNDLE_IMG=docker.io/bpalmer/ocp-latest-nginx-bundle:v4.11` 7. `operator-sdk run bundle docker.io/bpalmer/ocp-latest-nginx-bundle:v4.11` 8. Create CR with `kubectl apply -f config/samples/helmdemo_v1_nginx.yaml` After letting the cluster run for a few minutes after creating the CR we can see that there are no restarts on the controller-manager: ``` NAME READY STATUS RESTARTS AGE c384cf1887a53bd1be9ca55a1476ddf47beace8603af733b862348edccgzmd9 0/1 Completed 0 4m34s docker-io-bpalmer-ocp-latest-nginx-bundle-v4-11 1/1 Running 0 4m39s nginx-sample-5f9c6d457-hhn8k 1/1 Running 0 3m7s nginx-sample-5f9c6d457-k2tt7 1/1 Running 0 3m7s ocp-latest-nginx-controller-manager-7cbd84bc8c-shbfm 2/2 Running 0 4m18s ``` The results from my testing lead me to believe that this problem should be resolved with the latest ocp-release-operator-sdk version (v1.22.0). test env: operator-sdk version: "v1.22.0-ocp", commit: "7cce8230bda4895aacaee6103fe9c1dadc87b77d", kubernetes version: "v1.24.1", go version: "go1.18.3", GOOS: "linux", GOARCH: "amd64" test pass. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5069 |