1746159 – upgrade from 4.1.9 to 4.1.11 failed by 'replicaset-controller Error creating: pods "packageserver-6474c74cdd-" is forbidden: error looking up service account openshift-operator-lifecycle-manager/packageserver: serviceaccount "packageserver" not found'

Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.

Bug 1746159 - upgrade from 4.1.9 to 4.1.11 failed by 'replicaset-controller Error creating: pods "packageserver-6474c74cdd-" is forbidden: error looking up service account openshift-operator-lifecycle-manager/packageserver: serviceaccount "packageserver" not found'

Summary: upgrade from 4.1.9 to 4.1.11 failed by 'replicaset-controller Error creating:...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	OLM
Sub Component:
Version:	4.2.0
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	high
Target Milestone:	---
Target Release:	4.2.0
Assignee:	Nick Hale
QA Contact:	Bruno Andrade
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1743061 1755646
TreeView+	depends on / blocked

Reported:	2019-08-27 19:28 UTC by Evan Cordell
Modified:	2019-10-16 06:38 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1743061
Clones:	1755646 (view as bug list)
Environment:
Last Closed:	2019-10-16 06:37:54 UTC
Target Upstream Version:

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	operator-framework operator-lifecycle-manager pull 1025	0	'None'	'closed'	'Bug 1746159: Cleanup leftover cross-namespace OwnerReferences'	2019-11-18 15:33:48 UTC
Red Hat Product Errata	RHBA-2019:2922	0	None	None	None	2019-10-16 06:38:05 UTC

Comment 2 Bruno Andrade 2019-09-10 02:20:43 UTC

LGTM, marking as VERIFIED.

Steps used to validate:

1) Upgrade from 4.1.9 to 4.1.14

oc get clusterversion
NAME      VERSION   AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.9     True        False         5m57s   Cluster version is 4.1.9

oc get clusteroperators
NAME                                 VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                       4.1.9     True        False         False      6m30s
cloud-credential                     4.1.9     True        False         False      16m
cluster-autoscaler                   4.1.9     True        False         False      16m
console                              4.1.9     True        False         False      8m18s
dns                                  4.1.9     True        False         False      16m
image-registry                       4.1.9     True        False         False      10m
ingress                              4.1.9     True        False         False      10m
kube-apiserver                       4.1.9     True        False         False      13m
kube-controller-manager              4.1.9     True        False         False      13m
kube-scheduler                       4.1.9     True        False         False      14m
machine-api                          4.1.9     True        False         False      16m
machine-config                       4.1.9     True        False         False      14m
marketplace                          4.1.9     True        False         False      11m
monitoring                           4.1.9     True        False         False      8m30s
network                              4.1.9     True        False         False      16m
node-tuning                          4.1.9     True        False         False      11m
openshift-apiserver                  4.1.9     True        False         False      12m
openshift-controller-manager         4.1.9     True        False         False      15m
openshift-samples                    4.1.9     True        False         False      10m
operator-lifecycle-manager           4.1.9     True        False         False      15m
operator-lifecycle-manager-catalog   4.1.9     True        False         False      15m
service-ca                           4.1.9     True        False         False      16m
service-catalog-apiserver            4.1.9     True        False         False      11m
service-catalog-controller-manager   4.1.9     True        False         False      12m
storage                              4.1.9     True        False         False      11m

oc get clusterversion -o json|jq ".items[0].status.availableUpdates"
[
  {
    "force": false,
    "image": "quay.io/openshift-release-dev/ocp-release@sha256:bfca31dbb518b35f312cc67516fa18aa40df9925dc84fdbcd15f8bbca425d7ff",
    "version": "4.1.11"
  },
  {
    "force": false,
    "image": "quay.io/openshift-release-dev/ocp-release@sha256:212296a41e04176c308bfe169e7c6e05d77b76f403361664c3ce55cd30682a94",
    "version": "4.1.13"
  },
  {
    "force": false,
    "image": "quay.io/openshift-release-dev/ocp-release@sha256:fd41c9bda9e0ff306954f1fd7af6428edff8c3989b75f9fe984968db66846231",
    "version": "4.1.14"
  }
]


oc adm upgrade --to=4.1.14
Updating to 4.1.14

No issues were found related to ownerrefs
oc get clusteroperator
NAME                                 VERSION   AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                       4.1.14    True        False         False      3h21m
cloud-credential                     4.1.14    True        False         False      3h31m
cluster-autoscaler                   4.1.14    True        False         False      3h31m
console                              4.1.14    True        False         False      3h23m
dns                                  4.1.14    True        False         False      3h31m
image-registry                       4.1.14    True        False         False      154m
ingress                              4.1.14    True        False         False      3h25m
kube-apiserver                       4.1.14    True        False         False      3h28m
kube-controller-manager              4.1.14    True        False         False      3h28m
kube-scheduler                       4.1.14    True        False         False      3h29m
machine-api                          4.1.14    True        False         False      3h31m
machine-config                       4.1.14    True        False         False      122m
marketplace                          4.1.14    True        False         False      139m
monitoring                           4.1.14    True        False         False      127m
network                              4.1.14    True        False         False      3h31m
node-tuning                          4.1.14    True        False         False      129m
openshift-apiserver                  4.1.14    True        False         False      129m
openshift-controller-manager         4.1.14    True        False         False      3h30m
openshift-samples                    4.1.14    True        False         False      173m
operator-lifecycle-manager           4.1.14    True        False         False      3h30m
operator-lifecycle-manager-catalog   4.1.14    True        False         False      3h30m
service-ca                           4.1.14    True        False         False      3h31m
service-catalog-apiserver            4.1.14    True        False         False      3h26m
service-catalog-controller-manager   4.1.14    True        False         False      3h26m
storage 

2) Upgrade to 4.2.0-0.nightly-2019-09-09-110620, no issues found and the upgrade finished successfully.

oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.2.0-0.nightly-2019-09-09-110620 --force=true
Updating to release image registry.svc.ci.openshift.org/ocp/release:4.2.0-0.nightly-2019-09-09-110620


oc get clusteroperator operator-lifecycle-manager -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: "2019-09-09T19:52:34Z"
  generation: 1
  name: operator-lifecycle-manager
  resourceVersion: "120938"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/operator-lifecycle-manager
  uid: 5e32e153-d33b-11e9-bae5-02d708fadfba
spec: {}
status:
  conditions:
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "False"
    type: Degraded
  - lastTransitionTime: "2019-09-10T01:40:30Z"
    message: Deployed 0.11.0
    status: "True"
    type: Progressing
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2019-09-09T20:18:56Z"
    status: "True"
    type: Upgradeable
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-operator-lifecycle-manager
    resource: namespaces
  versions:
  - name: operator
    version: 4.2.0-0.nightly-2019-09-09-110620
  - name: operator-lifec


oc get clusteroperator operator-lifecycle-manager -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: "2019-09-09T19:52:34Z"
  generation: 1
  name: operator-lifecycle-manager
  resourceVersion: "120938"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/operator-lifecycle-manager
  uid: 5e32e153-d33b-11e9-bae5-02d708fadfba
spec: {}
status:
  conditions:
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "False"
    type: Degraded
  - lastTransitionTime: "2019-09-10T01:40:30Z"
    message: Deployed 0.11.0
    status: "True"
    type: Progressing
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2019-09-09T20:18:56Z"
    status: "True"
    type: Upgradeable
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-operator-lifecycle-manager
    resource: namespaces
  versions:
  - name: operator
    version: 4.2.0-0.nightly-2019-09-09-110620
  - name: operator-lifecycle-manager
    version: 0.11.0

oc get clusteroperator operator-lifecycle-manager-catalog -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: "2019-09-09T19:52:34Z"
  generation: 1
  name: operator-lifecycle-manager-catalog
  resourceVersion: "121892"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/operator-lifecycle-manager-catalog
  uid: 5e3afc92-d33b-11e9-bae5-02d708fadfba
spec: {}
status:
  conditions:
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "False"
    type: Degraded
  - lastTransitionTime: "2019-09-10T01:40:59Z"
    message: Deployed 0.11.0
    status: "True"
    type: Progressing
  - lastTransitionTime: "2019-09-09T19:52:34Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2019-09-09T20:19:27Z"
    status: "True"
    type: Upgradeable
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-operator-lifecycle-manager
    resource: namespaces
  versions:
  - name: operator
    version: 4.2.0-0.nightly-2019-09-09-110620
  - name: operator-lifecycle-manager
    version: 0.11.0

oc get clusteroperator operator-lifecycle-manager-packageserver -o yaml
apiVersion: config.openshift.io/v1
kind: ClusterOperator
metadata:
  creationTimestamp: "2019-09-10T01:40:30Z"
  generation: 1
  name: operator-lifecycle-manager-packageserver
  resourceVersion: "122698"
  selfLink: /apis/config.openshift.io/v1/clusteroperators/operator-lifecycle-manager-packageserver
  uid: f90e0197-d36b-11e9-a0dc-024004c39ce6
spec: {}
status:
  conditions:
  - lastTransitionTime: "2019-09-10T01:40:30Z"
    status: "False"
    type: Degraded
  - lastTransitionTime: "2019-09-10T01:41:52Z"
    status: "True"
    type: Available
  - lastTransitionTime: "2019-09-10T01:41:52Z"
    message: Deployed version 0.11.0
    status: "False"
    type: Progressing
  - lastTransitionTime: "2019-09-10T01:41:08Z"
    message: Safe to upgrade
    status: "True"
    type: Upgradeable
  extension: null
  relatedObjects:
  - group: ""
    name: openshift-operator-lifecycle-manager
    resource: namespaces
  - group: operators.coreos.com
    name: packageserver
    namespace: openshift-operator-lifecycle-manager
    resource: ClusterServiceVersion
  versions:
  - name: operator
    version: 4.2.0-0.nightly-2019-09-09-110620
  - name: packageserver
    version: 0.11.0

Comment 3 errata-xmlrpc 2019-10-16 06:37:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922

Note You need to log in before you can comment on or make changes to this bug.