Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1996638

Summary: Helm operator manager container restart when CR is creating&deleting
Product: OpenShift Container Platform Reporter: Fan Jia <jfan>
Component: Operator SDKAssignee: Bryce Palmer <bpalmer>
Status: CLOSED ERRATA QA Contact: Fan Jia <jfan>
Severity: high Docs Contact:
Priority: high    
Version: 4.9CC: aos-bugs, bpalmer, jesusr, jlanford
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
N/A
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-10 10:36:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 3 Jesus M. Rodriguez 2022-06-06 18:51:40 UTC
@bpalmer

Comment 4 Jesus M. Rodriguez 2022-06-06 20:38:04 UTC
The expected and actual results from the original comment are flipped. We expect it to work and the actual results is a failure.

Comment 5 Bryce Palmer 2022-06-06 21:37:22 UTC
When taking a look at this and testing with the latest ocp-release-operator-sdk, following the steps results in a controller-manager log output that contains the logged error.

Output of `kubectl logs nginx-controller-manager-6fd9f84dbc-x2g8c`

```
{"level":"error","ts":1654551120.0068138,"logger":"controller.nginx-controller","msg":"Reconciler error","name":"nginx-sample","namespace":"default","error":"Operation cannot be fulfilled on nginxes.helmdemo.example.com \"nginx-sample\": the object has been modified; please apply your changes to the latest version and try again","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\toperator-sdk/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227"}

```


Pod does not restart, seen by output of `kubectl get pods`:

```
NAME                                                              READY   STATUS      RESTARTS   AGE
3a5a1bcea941fcf2724f8c6000df5fab6f6ba6972c7b9d70018f8b9888mms9j   0/1     Completed   0          4m15s
docker-io-bpalmer-nginx-bundle-v4-9                               1/1     Running     0          4m24s
nginx-controller-manager-6fd9f84dbc-x2g8c                         2/2     Running     0          4m3s
nginx-sample-685f77dcbc-84mf6                                     1/1     Running     0          3m37s
nginx-sample-685f77dcbc-tbsld                                     1/1     Running     0          3m37s

```

Comment 7 Bryce Palmer 2022-06-22 18:48:02 UTC
I tested this using the latest commit on the master branch of https://github.com/openshift/ocp-release-operator-sdk and OpenShift 4.11.0-fc.0 using the following steps:

1. `operator-sdk version`:
```
operator-sdk version: "v1.22.0-ocp", commit: "8e224f3704e7c9fdd84aac3a8f76364c68249372", kubernetes version: "v1.24.1", go version: "go1.18.3", GOOS: "linux", GOARCH: "amd64"

```

2. `mkdir ocp-latest-nginx && cd ocp-latest-nginx`

3. `operator-sdk init --plugins=helm --domain=example.com --group=helmdemo --version=v1 --kind=Nginx`

4. edit *config/samples/helmdemo_v1_nginx.yaml*
```
apiVersion: helmdemo.example.com/v1
kind: Nginx
metadata:
  name: nginx-sample
...
spec:
...
  replicaCount: 2
  service:
    port: 8080

```

5. edit `Dockerfile`
```
FROM quay.io/operator-framework/helm-operator:v1.22
```
(Used the upstream image for verification that it works as I am not sure the downstream image has been updated to use OSDK v1.22.0 at this time, but the v4.11 should be updated soon as the downstream repo has been synced by: https://github.com/openshift/ocp-release-operator-sdk/pull/264 )

6. generate the bundle image:
- `make docker-build docker-push IMG=docker.io/bpalmer/ocp-latest-nginx-operator:v4.11`

- edit Makefile by setting the IMG to be docker.io/bpalmer/ocp-latest-nginx-operator:v4.11
```
IMG ?= docker.io/bpalmer/ocp-latest-nginx-operator:v4.11
```

- `make bundle`

- `make bundle-build bundle-push BUNDLE_IMG=docker.io/bpalmer/ocp-latest-nginx-bundle:v4.11`

7. `operator-sdk run bundle docker.io/bpalmer/ocp-latest-nginx-bundle:v4.11`

8. Create CR with `kubectl apply -f config/samples/helmdemo_v1_nginx.yaml`

After letting the cluster run for a few minutes after creating the CR we can see that there are no restarts on the controller-manager:
```
NAME                                                              READY   STATUS      RESTARTS   AGE
c384cf1887a53bd1be9ca55a1476ddf47beace8603af733b862348edccgzmd9   0/1     Completed   0          4m34s
docker-io-bpalmer-ocp-latest-nginx-bundle-v4-11                   1/1     Running     0          4m39s
nginx-sample-5f9c6d457-hhn8k                                      1/1     Running     0          3m7s
nginx-sample-5f9c6d457-k2tt7                                      1/1     Running     0          3m7s
ocp-latest-nginx-controller-manager-7cbd84bc8c-shbfm              2/2     Running     0          4m18s

```

The results from my testing lead me to believe that this problem should be resolved with the latest ocp-release-operator-sdk version (v1.22.0).

Comment 8 Fan Jia 2022-06-23 07:19:47 UTC
test env: 
operator-sdk version: "v1.22.0-ocp", commit: "7cce8230bda4895aacaee6103fe9c1dadc87b77d", kubernetes version: "v1.24.1", go version: "go1.18.3", GOOS: "linux", GOARCH: "amd64"

test pass.

Comment 10 errata-xmlrpc 2022-08-10 10:36:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069