Bug 2227607

Summary: After upgrading to 4.14, MCG endpoint minCount and maxCount in ocs-storagecluster are ignored
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Vijay Avuthu <vavuthu>
Component: Multi-Cloud Object GatewayAssignee: Naveen Paul <napaul>
Status: CLOSED ERRATA QA Contact: Vijay Avuthu <vavuthu>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.14CC: dzaken, napaul, nigoyal, odf-bz-bot
Target Milestone: ---Keywords: Automation, Regression
Target Release: ODF 4.14.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.14.0-96 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-08 18:52:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vijay Avuthu 2023-07-31 05:06:20 UTC
Description of problem (please be detailed as possible and provide log
snippests):

After upgrading cluster from v4.13.1 to 4.14.0-90, MCG endpoint minCount and maxCount in ocs-storagecluster are ignored


Version of all relevant components (if applicable):
ODF cluster upgraded from v4.13.1 to 4.14.0-90


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
No

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
2/2

Can this issue reproduce from the UI?
Not tried

If this is a regression, please provide more details to justify this:
Yes

Steps to Reproduce:
1. install ODF 4.13.1 using ocs-ci
2. upgrade cluster to 4.14.0-90
3. run acceptance suite ( tests/manage/mcg/test_bucket_creation.py::TestBucketCreation::test_bucket_creation[3-S3-DEFAULT-BACKINGSTORE] )


Actual results:

2023-07-29 13:44:30  08:14:30 - MainThread - tests.conftest - INFO  - Changing minimum Noobaa endpoints to 2
2023-07-29 13:44:30  08:14:30 - MainThread - ocs_ci.ocs.ocp - INFO  - Command: patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"minCount":2}}}}' --type merge
2023-07-29 13:44:30  08:14:30 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc -n openshift-storage patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"minCount":2}}}}' --type merge
2023-07-29 13:44:30  08:14:30 - MainThread - tests.conftest - INFO  - Changing maximum Noobaa endpoints to 2
2023-07-29 13:44:30  08:14:30 - MainThread - ocs_ci.ocs.ocp - INFO  - Command: patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"maxCount":2}}}}' --type merge
2023-07-29 13:44:30  08:14:30 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc -n openshift-storage patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"maxCount":2}}}}' --type merge


$ oc get Pod  -n openshift-storage --selector=noobaa-s3=noobaa
NAME                              READY   STATUS    RESTARTS   AGE
noobaa-endpoint-d77965955-b55ww   1/1     Running   0          45h



Expected results:

expecting 2 endpoints


Additional info:

$ oc get storagecluster ocs-storagecluster -o yaml | grep multi -A3
  multiCloudGateway:
    endpoints:
      maxCount: 2
      minCount: 2


$ oc get Pod  -n openshift-storage --selector=noobaa-s3=noobaa
NAME                              READY   STATUS    RESTARTS   AGE
noobaa-endpoint-d77965955-b55ww   1/1     Running   0          45h

job link: https://url.corp.redhat.com/2f96e21
must gather: https://url.corp.redhat.com/d81201c

Comment 8 Vijay Avuthu 2023-10-19 11:51:59 UTC
verified with OCS operator: 4.14.0-154

url: https://url.corp.redhat.com/24fa2fc

must gather: https://url.corp.redhat.com/564520a

acceptance test suite passed ( BUILD ID: 4.14.0-154 RUN ID: 1697685034 )

33 tests ran in 8735.30 seconds.
33 passed, 5 skipped, 0 failed, 0 errors, 0 expected failures, 0 unexpected passes


>

2023-10-19 08:42:29  03:12:28 - MainThread - tests.conftest - INFO  - Changing minimum Noobaa endpoints to 2
2023-10-19 08:42:29  03:12:28 - MainThread - ocs_ci.ocs.ocp - INFO  - Command: patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"minCount":2}}}}' --type merge
2023-10-19 08:42:29  03:12:28 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc -n openshift-storage patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"minCount":2}}}}' --type merge
2023-10-19 08:42:29  03:12:29 - MainThread - tests.conftest - INFO  - Changing maximum Noobaa endpoints to 2
2023-10-19 08:42:29  03:12:29 - MainThread - ocs_ci.ocs.ocp - INFO  - Command: patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"maxCount":2}}}}' --type merge
2023-10-19 08:42:29  03:12:29 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc -n openshift-storage patch storagecluster ocs-storagecluster -n openshift-storage -p '{"spec":{"multiCloudGateway":{"endpoints":{"maxCount":2}}}}' --type merge
2023-10-19 08:42:29  03:12:29 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n openshift-storage get Pod  -n openshift-storage --selector=noobaa-s3=noobaa -o yaml
2023-10-19 08:42:29  03:12:29 - MainThread - tests.conftest - INFO  - Waiting for the NooBaa endpoints to stabilize. Current ready count: 1
2023-10-19 08:42:29  03:12:29 - MainThread - ocs_ci.utility.utils - INFO  - Going to sleep for 30 seconds before next iteration
2023-10-19 08:43:01  03:12:59 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n openshift-storage get Pod  -n openshift-storage --selector=noobaa-s3=noobaa -o yaml
2023-10-19 08:43:01  03:13:00 - MainThread - tests.conftest - INFO  - Waiting for the NooBaa endpoints to stabilize. Current ready count: 1
2023-10-19 08:43:01  03:13:00 - MainThread - ocs_ci.utility.utils - INFO  - Going to sleep for 30 seconds before next iteration
2023-10-19 08:43:33  03:13:30 - MainThread - ocs_ci.utility.utils - INFO  - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n openshift-storage get Pod  -n openshift-storage --selector=noobaa-s3=noobaa -o yaml
2023-10-19 08:43:33  03:13:30 - MainThread - tests.conftest - INFO  - NooBaa endpoints stabilized. Ready endpoints: 2

Comment 10 errata-xmlrpc 2023-11-08 18:52:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6832