Bug 1828008

Summary:	Image registry cannot be fully removed with 409 error about the deleting bucket is not empty
Product:	OpenShift Container Platform	Reporter:	Wenjing Zheng <wzheng>
Component:	Image Registry	Assignee:	Oleg Bulatov <obulatov>
Status:	CLOSED DUPLICATE	QA Contact:	Wenjing Zheng <wzheng>
Severity:	low	Docs Contact:
Priority:	medium
Version:	4.4	CC:	aos-bugs
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-04-27 09:50:17 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Wenjing Zheng 2020-04-26 09:45:36 UTC

This bug was initially created as a copy of Bug #1827075

I am copying this bug because: 



Description of problem:
Below message appreas when try to remove non-empty image registry:
message: ’googleapi: Error 409: The bucket you tried to delete was not empty.,

Version-Release number of selected component (if applicable):
4.4.0-0.nightly-2020-04-25-191512

How reproducible:
Not always

Steps to Reproduce:
1.Import some images to image registry with GSC storage backend
2.Try to remove image registry
3.Try to bring image registry back and check pods under openshift-image-registry
$ oc get pods
NAME                                               READY   STATUS        RESTARTS   AGE
cluster-image-registry-operator-78c9b64968-dr6f5   2/2     Running       0          20h
image-registry-549fbdc64f-jlzmt                    0/1     Pending       0          81s
image-registry-69bd779cdd-9pbwg                    0/1     Terminating   0          82s
image-registry-7bb54df499-lpb8g                    0/1     Pending       0          81s
node-ca-79vw9                                      1/1     Running       1          20h
node-ca-chnkb                                      1/1     Running       1          20h
node-ca-w6csh                                      1/1     Running       1          20h
node-ca-x5kbv                                      1/1     Running       1          20h
node-ca-z2hzd                                      1/1     Running       1          20h


Actual results:
Image registry pod cannot come back.

Expected results:
It should be back.

Additional info:
Workaround: manually delete bucket from gcp console(operator will prompt "unable to sync: unable to finalize resource: unable to remove storage: googleapi: Error 404: Not Found, notFound, googleapi: Error 404: Not Found"), the bucket can be deleted in GCP console, then recreate a bucket with same name of the deleted one.