Bug 1876571 - Openshift installation fails - authentication and network operators are in degraded state
Summary: Openshift installation fails - authentication and network operators are in de...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 4.6.0
Assignee: Maysa Macedo
QA Contact: GenadiC
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-07 14:11 UTC by Itzik Brown
Modified: 2020-10-27 16:38 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 16:38:28 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 342 0 None open Bug 1876571: Ensure updated lb sgs is used on the CRD 2020-09-09 12:41:41 UTC
OpenStack gerrit 750470 0 None NEW Ensure updated lb sgs is used on the CRD 2020-09-09 11:08:52 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:38:41 UTC

Description Itzik Brown 2020-09-07 14:11:58 UTC 
Description of problem:
As described

Version-Release number of selected component (if applicable):
OCP 4.6.0-0.nightly-2020-09-04-093211
OSP 13 2020-06-11.1

It seems that there are missing security group rules 
e.g. missing 443/TCP for openshift-authentication service

$ openstack security group rule list |grep ea5a68c1-d499-4358-98a3-bc890be16cc4
| 23920e92-b301-48cd-beac-b77a37209fc6 | None        | None           |             | None                                 | ea5a68c1-d499-4358-98a3-bc890be16cc4 |
| 8667a067-c408-4238-96e9-4cd144d100d0 | tcp         | None           | 1025:1025   | None                                 | ea5a68c1-d499-4358-98a3-bc890be16cc4 |
| 89ceda08-21aa-458c-944d-2f83713e1385 | None        | None           |             | None  




How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
$ oc get pods -n openshift-kuryr
NAME                                   READY   STATUS    RESTARTS   AGE
kuryr-cni-2wrjc                        1/1     Running   0          10h
kuryr-cni-cmwx5                        1/1     Running   3          11h
kuryr-cni-fxsgb                        1/1     Running   1          11h
kuryr-cni-gggfk                        1/1     Running   2          11h
kuryr-cni-twmsg                        1/1     Running   2          11h
kuryr-cni-v4x9p                        1/1     Running   106        7h48m
kuryr-controller-7b457fcd9-gs5tn       1/1     Running   10         125m
kuryr-dns-admission-controller-m56w5   1/1     Running   0          11h
kuryr-dns-admission-controller-w9shl   1/1     Running   0          11h
kuryr-dns-admission-controller-wb9b4   1/1     Running   0          11h
Comment 3 weiwei jiang 2020-09-10 07:27:42 UTC 
Checked with 4.6.0-0.nightly-2020-09-09-224210, and it's fixed now.

$ oc get co 
NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
cloud-credential                           4.6.0-0.nightly-2020-09-09-224210   True        False         False      85m
cluster-autoscaler                         4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
config-operator                            4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
console                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m
csi-snapshot-controller                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
dns                                        4.6.0-0.nightly-2020-09-09-224210   True        False         False      74m
etcd                                       4.6.0-0.nightly-2020-09-09-224210   True        False         False      79m
image-registry                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      44m
ingress                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      65m
insights                                   4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
kube-apiserver                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      78m
kube-controller-manager                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
kube-scheduler                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
kube-storage-version-migrator              4.6.0-0.nightly-2020-09-09-224210   True        False         False      43m
machine-api                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      68m
machine-approver                           4.6.0-0.nightly-2020-09-09-224210   True        False         False      74m
machine-config                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      71m
marketplace                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m
monitoring                                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
network                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      80m
node-tuning                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
openshift-apiserver                        4.6.0-0.nightly-2020-09-09-224210   True        False         False      42m
openshift-controller-manager               4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
openshift-samples                          4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
operator-lifecycle-manager                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
operator-lifecycle-manager-catalog         4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
operator-lifecycle-manager-packageserver   4.6.0-0.nightly-2020-09-09-224210   True        False         False      24m
service-ca                                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      79m
storage                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-09-09-224210   True        False         47m     Cluster version is 4.6.0-0.nightly-2020-09-09-224210

$ oc get nodes -o wide 
NAME                                 STATUS   ROLES    AGE   VERSION                INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINER-RUNTIME
wj46ioskr910a-klxng-master-0         Ready    master   85m   v1.19.0-rc.2+40d85fc   192.168.2.182   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-master-1         Ready    master   85m   v1.19.0-rc.2+40d85fc   192.168.0.142   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-master-2         Ready    master   86m   v1.19.0-rc.2+40d85fc   192.168.1.19    <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-khcrx   Ready    worker   67m   v1.19.0-rc.2+40d85fc   192.168.1.22    <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-ndjtw   Ready    worker   68m   v1.19.0-rc.2+40d85fc   192.168.1.129   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-q9c66   Ready    worker   66m   v1.19.0-rc.2+40d85fc   192.168.1.171   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1


$ oc describe co authentication 
Name:         authentication
Namespace:    
Labels:       <none>
Annotations:  exclude.release.openshift.io/internal-openshift-hosted: true
API Version:  config.openshift.io/v1
Kind:         ClusterOperator
Metadata:
  Creation Timestamp:  2020-09-10T05:58:24Z
  Generation:          1
  Managed Fields:
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:exclude.release.openshift.io/internal-openshift-hosted:
      f:spec:
      f:status:
        .:
        f:extension:
    Manager:      cluster-version-operator
    Operation:    Update
    Time:         2020-09-10T05:58:24Z
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
        f:relatedObjects:
        f:versions:
    Manager:         authentication-operator
    Operation:       Update
    Time:            2020-09-10T06:48:59Z
  Resource Version:  77033
  Self Link:         /apis/config.openshift.io/v1/clusteroperators/authentication
  UID:               9aecec81-66e6-455f-a1b4-50d2deff6d3e
Spec:
Status:
  Conditions:
    Last Transition Time:  2020-09-10T06:37:36Z
    Reason:                AsExpected
    Status:                False
    Type:                  Degraded
    Last Transition Time:  2020-09-10T06:48:32Z
    Reason:                AsExpected
    Status:                False
    Type:                  Progressing
    Last Transition Time:  2020-09-10T06:48:46Z
    Message:               OAuthServerDeploymentAvailable: availableReplicas==2
    Reason:                AsExpected
    Status:                True
    Type:                  Available
    Last Transition Time:  2020-09-10T06:05:51Z
    Reason:                AsExpected
    Status:                True
    Type:                  Upgradeable
  Extension:               <nil>
  Related Objects:
    Group:      operator.openshift.io
    Name:       cluster
    Resource:   authentications
    Group:      config.openshift.io
    Name:       cluster
    Resource:   authentications
    Group:      config.openshift.io
    Name:       cluster
    Resource:   infrastructures
    Group:      config.openshift.io
    Name:       cluster
    Resource:   oauths
    Group:      route.openshift.io
    Name:       oauth-openshift
    Namespace:  openshift-authentication
    Resource:   routes
    Group:      
    Name:       oauth-openshift
    Namespace:  openshift-authentication
    Resource:   services
    Group:      
    Name:       openshift-config
    Resource:   namespaces
    Group:      
    Name:       openshift-config-managed
    Resource:   namespaces
    Group:      
    Name:       openshift-authentication
    Resource:   namespaces
    Group:      
    Name:       openshift-authentication-operator
    Resource:   namespaces
    Group:      
    Name:       openshift-ingress
    Resource:   namespaces
    Group:      
    Name:       openshift-oauth-apiserver
    Resource:   namespaces
  Versions:
    Name:     oauth-apiserver
    Version:  4.6.0-0.nightly-2020-09-09-224210
    Name:     operator
    Version:  4.6.0-0.nightly-2020-09-09-224210
    Name:     oauth-openshift
    Version:  4.6.0-0.nightly-2020-09-09-224210_openshift
Events:       <none>
Comment 5 errata-xmlrpc 2020-10-27 16:38:28 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196
Note You need to log in before you can comment on or make changes to this bug.