Bug 1991826

Summary: Reports "Unable to apply resources: unable to sync storage configuration" when set swift storage
Product: OpenShift Container Platform Reporter: XiuJuan Wang <xiuwang>
Component: Image RegistryAssignee: Oleg Bulatov <obulatov>
Status: CLOSED DEFERRED QA Contact: XiuJuan Wang <xiuwang>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.9CC: aos-bugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-01-28 16:08:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description XiuJuan Wang 2021-08-10 06:18:23 UTC
Description of problem:
When set swift application credentials, There are error "Failed to authenticate provider client: Post \"https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens\": EOF" in the registry operator log

Version-Release number of selected component (if applicable):

4.8.0-0.ci.test-2021-08-09-015622-ci-ln-km2d11t-latest

How reproducible:
always

Steps to Reproduce:
1.Create the application credential on openstack
 $openstack application credential list 
+----------------------------------+------+----------------------------------+-------------+------------+
| ID                               | Name | Project ID                       | Description | Expires At |
+----------------------------------+------+----------------------------------+-------------+------------+
| 7964d5af93ce477c9c35b1286998c516 | wxj  | 542c6ebd48bf40fa857fc245c7572e30 | None        | None       |
+----------------------------------+------+----------------------------------+-------------+------------+
2.Check the default value of image-registry-private-configuration after cluster installation

$oc get secret image-registry-private-configuration   -o yaml 
apiVersion: v1
data:
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALID: IiI=
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALNAME: IiI=
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALSECRET: IiI=
  REGISTRY_STORAGE_SWIFT_PASSWORD: ****
  REGISTRY_STORAGE_SWIFT_USERNAME: ****

3. Set application creds
$oc get  secret image-registry-private-configuration  -o yaml 
apiVersion: v1
data:
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALID: Nzk2NGQ1YWY5M2NlNDc3YzljMzViMTI4Njk5OGM1MTYK
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALNAME: d3hqCg==
  REGISTRY_STORAGE_SWIFT_APPLICATIONCREDENTIALSECRET: bE5JQTA4cUh1NGxm**********
  REGISTRY_STORAGE_SWIFT_PASSWORD: ****
  REGISTRY_STORAGE_SWIFT_USERNAME: ****

Actual results:
The registry are re-deployed successfully after secret setting.
But there are some errors in image registry operator logs

I0810 05:40:22.489228       1 controller.go:330] object changed: *v1.Config, Name=cluster (status=true): changed:status.conditions.0.lastTransitionTime={"2021-08-10T05:40:21Z" -> "2021-08-10T05:40:22Z"}, removed:status.conditions.0.message="Failed to authenticate provider client: Post \"https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens\": EOF", changed:status.conditions.0.reason={"Could not connect to registry storage" -> "Swift container Exists"}, changed:status.conditions.0.status={"Unknown" -> "True"}, changed:status.conditions.2.lastTransitionTime={"2021-08-10T05:40:21Z" -> "2021-08-10T05:40:22Z"}, changed:status.conditions.2.message={"Unable to apply resources: unable to sync storage configuration: Failed to authenticate provider client: Post \"https://rhos-d.infra.prod.upshift.rdu2.redhat.com:13000/v3/auth/tokens\": EOF" -> "The registry is ready"}, changed:status.conditions.2.reason={"Error" -> "Ready"}, changed:status.conditions.2.status={"True" -> "False"}


Expected results:
Should use application creds well

Additional info:

Comment 1 XiuJuan Wang 2021-08-10 06:24:08 UTC
The must gather log http://virt-openshift-05.lab.eng.nay.redhat.com/xiuwang/swfitinfra.tar.gz

Comment 2 Oleg Bulatov 2021-09-03 14:21:34 UTC
So this is a temporary problem, the operator has eventually become ready, and it happens with all Swift clusters?

Comment 3 XiuJuan Wang 2021-09-06 07:33:34 UTC
Yeah, It happens with all swift cluster, it's not a temporary problem. When image registry got ready, it happens sometime.

Comment 4 Oleg Bulatov 2022-01-28 16:08:54 UTC
Taking into account the size of the engineering backlog and that we don't have customers who complained about it, it's unlikely that this bug will be fixed in near future. If this bug disrupts your workflow, please open a new one with details how it affects you.