Bug 1791281

Summary:	[IPI][OSP] Installer cannot finish because image-registry operator never gets available
Product:	OpenShift Container Platform	Reporter:	David Sanz <dsanzmor>
Component:	Image Registry	Assignee:	Oleg Bulatov <obulatov>
Status:	CLOSED ERRATA	QA Contact:	David Sanz <dsanzmor>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.4	CC:	aos-bugs, mfedosin
Target Milestone:	---	Keywords:	BetaBlocker, TestBlocker
Target Release:	4.4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: swift authVersion were serialized as a string, but the registry expects an integer Consequence: the registry couldn't parse the configuration Fix: serialize it as an integer Result: the registry understands authVersion	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-05-04 11:24:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description David Sanz 2020-01-15 12:33:58 UTC

Description of problem:

Image registry is failing to deploy because of this error in the image-registry pods:

* 'AuthVersion' expected type 'int', got unconvertible type 'string'

oc logs pod/image-registry-6d8555886f-jddr4
time="2020-01-15T12:19:19.811734335Z" level=info msg="start registry" distribution_version=v2.6.0+unknown go.version=go1.13.4 openshift_version=v4.4.0-202001102230+c710383-dirty
time="2020-01-15T12:19:19.812428035Z" level=info msg="caching project quota objects with TTL 1m0s" go.version=go1.13.4
panic: 1 error(s) decoding:
* 'AuthVersion' expected type 'int', got unconvertible type 'string'
goroutine 1 [running]:
github.com/docker/distribution/registry/handlers.NewApp(0x1d91d00, 0xc0000440a0, 0xc000380e00, 0xc0005b7bc0)
	/go/src/github.com/openshift/image-registry/vendor/github.com/docker/distribution/registry/handlers/app.go:127 +0x315a
github.com/openshift/image-registry/pkg/dockerregistry/server/supermiddleware.NewApp(0x1d91d00, 0xc0000440a0, 0xc000380e00, 0x1d9ac80, 0xc0003a5d40, 0x1da3400)
	/go/src/github.com/openshift/image-registry/pkg/dockerregistry/server/supermiddleware/app.go:96 +0x85
github.com/openshift/image-registry/pkg/dockerregistry/server.NewApp(0x1d91d00, 0xc0000440a0, 0x1d5e040, 0xc000010300, 0xc000380e00, 0xc0004e4000, 0x0, 0x0, 0x18, 0xc0005b16f8)
	/go/src/github.com/openshift/image-registry/pkg/dockerregistry/server/app.go:138 +0x2c3
github.com/openshift/image-registry/pkg/cmd/dockerregistry.NewServer(0x1d91d00, 0xc0000440a0, 0xc000380e00, 0xc0004e4000, 0x0, 0x0, 0x1dcab00)
	/go/src/github.com/openshift/image-registry/pkg/cmd/dockerregistry/dockerregistry.go:210 +0x1be
github.com/openshift/image-registry/pkg/cmd/dockerregistry.Execute(0x1d4e320, 0xc000010028)
	/go/src/github.com/openshift/image-registry/pkg/cmd/dockerregistry/dockerregistry.go:164 +0xa78
main.main()
	/go/src/github.com/openshift/image-registry/cmd/dockerregistry/main.go:93 +0x4b7


Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.Install IPI cluster on OSP
2.Check openshift-image-registry namespace
3.oc logs image-registry-*

Actual results:
Installer waits for image registry cluster operator to finish. failing because it is not deployed

Expected results:
Installer finished correctly

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 David Sanz 2020-01-15 13:16:25 UTC

Reassigned to image-registry component

Comment 3 David Sanz 2020-01-17 18:21:01 UTC

Verified on 4.4.0-0.nightly-2020-01-17-164828

[morenod@morenod-laptop ~]$ oc get co | grep image-registry
image-registry                             4.4.0-0.nightly-2020-01-17-164828   True        False         False      55s

[morenod@morenod-laptop ~]$ oc project openshift-image-registry
Already on project "openshift-image-registry" on server "https://api.mrnd-13-44-no.qe.devcluster.openshift.com:6443".
[morenod@morenod-laptop ~]$ oc get all
NAME                                                   READY   STATUS    RESTARTS   AGE
pod/cluster-image-registry-operator-67bc448487-fdh85   2/2     Running   0          5m25s
pod/image-registry-5f8db4c6d7-skn6f                    1/1     Running   0          3m15s
pod/node-ca-2cgx5                                      1/1     Running   0          2m35s
pod/node-ca-p8pvp                                      1/1     Running   0          5m7s
pod/node-ca-sjjnt                                      1/1     Running   0          5m7s
pod/node-ca-x25cm                                      1/1     Running   0          5m7s

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)     AGE
service/image-registry            ClusterIP   172.30.136.183   <none>        5000/TCP    5m9s
service/image-registry-operator   ClusterIP   None             <none>        60000/TCP   14m

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/node-ca   4         4         4       4            4           kubernetes.io/os=linux   5m9s

NAME                                              READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster-image-registry-operator   1/1     1            1           14m
deployment.apps/image-registry                    1/1     1            1           5m8s

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-image-registry-operator-67bc448487   1         1         1       14m
replicaset.apps/image-registry-5f8db4c6d7                    1         1         1       3m16s
replicaset.apps/image-registry-5f994dc889                    0         0         0       5m8s
replicaset.apps/image-registry-77d68fcbc                     0         0         0       3m18s

Comment 5 errata-xmlrpc 2020-05-04 11:24:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581