Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1717764

Summary:	Service catalog not working after Upgrade
Product:	OpenShift Container Platform	Reporter:	Jaspreet Kaur <jkaur>
Component:	Service Catalog	Assignee:	Jesus M. Rodriguez <jesusr>
Status:	CLOSED ERRATA	QA Contact:	Bruno Andrade <bandrade>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	3.11.0	CC:	emahoney, jiazha, shurley, wmeng
Target Milestone:	---
Target Release:	3.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Service Catalog and Broker certificates were not redeployed after upgrade Consequence: Service Catalog not properly installed, access denied because of invalid certificates. Fix: Update openshift-ansible to deploy the certificates. Result: For installation, Service Catalog, Ansible Service Broker, and Template Service Broker all default to enabled, so they should default to enabled when redeploying certificates as well (otherwise when using the same inventory the catalog/brokers will go from a working state to a broken state). The Service Catalog certs should be redeployed whether or not the brokers are enabled.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-07-23 19:56:23 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Comment 18 Shawn Hurley 2019-06-19 17:44:30 UTC

@emahoney: I think that there was an issue in the openshift-installer around redeploying certificates that caused the same symptoms:

Here is a PR to fix it: https://github.com/openshift/openshift-ansible/pull/11704

Moving this to POST as I believe this to be the fix. Please let me know if that does not make sense

Comment 22 Bruno Andrade 2019-07-11 02:40:09 UTC

LGTM, marking as verified.

Steps used to validate:

1) Provisioned a cluster on 3.11.98 version

rpm --nodigest -qa | grep -i openshift
atomic-openshift-hyperkube-3.11.98-1.git.0.0cbaff3.el7.x86_64
atomic-openshift-clients-3.11.98-1.git.0.0cbaff3.el7.x86_64
atomic-openshift-3.11.98-1.git.0.0cbaff3.el7.x86_64
atomic-openshift-excluder-3.11.98-1.git.0.0cbaff3.el7.noarch
atomic-openshift-node-3.11.98-1.git.0.0cbaff3.el7.x86_64
atomic-openshift-docker-excluder-3.11.98-1.git.0.0cbaff3.el7.n

oc v3.11.98
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://qe-bandrademerrn-1:8443
openshift v3.11.98
kubernetes v1.11.0+d4cacc0

oc get clusterservicebroker
NAME                      URL                                                                                         STATUS    AGE
ansible-service-broker    https://asb.openshift-ansible-service-broker.svc:1338/osb                                   Ready     39m
template-service-broker   https://apiserver.openshift-template-service-broker.svc:443/brokers/template.openshift.io   Ready     37m

oc exec apiserver-4h5t4 -- service-catalog --version
v3.11.98;Upstream:v0.1.35
oc exec controller-manager-k89mx -- service-catalog --version
v3.11.98;Upstream:v0.1.35

2) Uninstalled service catalog

/usr/share/ansible/openshift-ansible/playbooks/openshift-service-catalog/config.yml

openshift_enable_service_catalog=false
ansible_service_broker_install=false
template_service_broker_install=false
openshift_service_catalog_remove=true

PLAY RECAP *********************************************************************
ci-vm-10-0-149-205.hosted.upshift.rdu2.redhat.com : ok=54   changed=11   unreachable=0    failed=0   
localhost                  : ok=12   changed=0    unreachable=0    failed=0   


3) Upgraded to 3.11.128 successfully

PLAY RECAP **************************************************************************************************************************************************************
ci-vm-10-0-149-205.hosted.upshift.rdu2.redhat.com : ok=618  changed=138  unreachable=0    failed=0   

4) Installed Service Catalog again without any issue

PLAY RECAP ************************************************************************
ci-vm-10-0-149-205.hosted.upshift.rdu2.redhat.com : ok=124  changed=32   unreachable=0    failed=0   
localhost                  : ok=12   changed=0    unreachable=0    failed=0   


INSTALLER STATUS ******************************************************************
Initialization           : Complete (0:01:01)
Service Catalog Install  : Complete (0:01:46)

oc exec controller-manager-qbmj8 -- service-catalog --version
v3.11.128;Upstream:v0.1.35

oc exec apiserver-psvzc -- service-catalog --version
v3.11.128;Upstream:v0.1.35

Comment 23 Bruno Andrade 2019-07-11 02:44:58 UTC

The upgrade also worked without the need to uninstall the service catalog components

Comment 25 errata-xmlrpc 2019-07-23 19:56:23 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1753