Bug 1876571 - Openshift installation fails - authentication and network operators are in degraded state
Summary: Openshift installation fails - authentication and network operators are in de...
Keywords:
Status: VERIFIED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 4.6.0
Assignee: Maysa Macedo
QA Contact: GenadiC
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-07 14:11 UTC by Itzik Brown
Modified: 2020-09-10 07:27 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 342 None open Bug 1876571: Ensure updated lb sgs is used on the CRD 2020-09-09 12:41:41 UTC
OpenStack gerrit 750470 None NEW Ensure updated lb sgs is used on the CRD 2020-09-09 11:08:52 UTC

Description Itzik Brown 2020-09-07 14:11:58 UTC
Description of problem:
As described

Version-Release number of selected component (if applicable):
OCP 4.6.0-0.nightly-2020-09-04-093211
OSP 13 2020-06-11.1

It seems that there are missing security group rules 
e.g. missing 443/TCP for openshift-authentication service

$ openstack security group rule list |grep ea5a68c1-d499-4358-98a3-bc890be16cc4
| 23920e92-b301-48cd-beac-b77a37209fc6 | None        | None           |             | None                                 | ea5a68c1-d499-4358-98a3-bc890be16cc4 |
| 8667a067-c408-4238-96e9-4cd144d100d0 | tcp         | None           | 1025:1025   | None                                 | ea5a68c1-d499-4358-98a3-bc890be16cc4 |
| 89ceda08-21aa-458c-944d-2f83713e1385 | None        | None           |             | None  




How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
$ oc get pods -n openshift-kuryr
NAME                                   READY   STATUS    RESTARTS   AGE
kuryr-cni-2wrjc                        1/1     Running   0          10h
kuryr-cni-cmwx5                        1/1     Running   3          11h
kuryr-cni-fxsgb                        1/1     Running   1          11h
kuryr-cni-gggfk                        1/1     Running   2          11h
kuryr-cni-twmsg                        1/1     Running   2          11h
kuryr-cni-v4x9p                        1/1     Running   106        7h48m
kuryr-controller-7b457fcd9-gs5tn       1/1     Running   10         125m
kuryr-dns-admission-controller-m56w5   1/1     Running   0          11h
kuryr-dns-admission-controller-w9shl   1/1     Running   0          11h
kuryr-dns-admission-controller-wb9b4   1/1     Running   0          11h

Comment 3 weiwei jiang 2020-09-10 07:27:42 UTC
Checked with 4.6.0-0.nightly-2020-09-09-224210, and it's fixed now.

$ oc get co 
NAME                                       VERSION                             AVAILABLE   PROGRESSING   DEGRADED   SINCE
authentication                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
cloud-credential                           4.6.0-0.nightly-2020-09-09-224210   True        False         False      85m
cluster-autoscaler                         4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
config-operator                            4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
console                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m
csi-snapshot-controller                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
dns                                        4.6.0-0.nightly-2020-09-09-224210   True        False         False      74m
etcd                                       4.6.0-0.nightly-2020-09-09-224210   True        False         False      79m
image-registry                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      44m
ingress                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      65m
insights                                   4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
kube-apiserver                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      78m
kube-controller-manager                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
kube-scheduler                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
kube-storage-version-migrator              4.6.0-0.nightly-2020-09-09-224210   True        False         False      43m
machine-api                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      68m
machine-approver                           4.6.0-0.nightly-2020-09-09-224210   True        False         False      74m
machine-config                             4.6.0-0.nightly-2020-09-09-224210   True        False         False      71m
marketplace                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m
monitoring                                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      37m
network                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      80m
node-tuning                                4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
openshift-apiserver                        4.6.0-0.nightly-2020-09-09-224210   True        False         False      42m
openshift-controller-manager               4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
openshift-samples                          4.6.0-0.nightly-2020-09-09-224210   True        False         False      72m
operator-lifecycle-manager                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
operator-lifecycle-manager-catalog         4.6.0-0.nightly-2020-09-09-224210   True        False         False      76m
operator-lifecycle-manager-packageserver   4.6.0-0.nightly-2020-09-09-224210   True        False         False      24m
service-ca                                 4.6.0-0.nightly-2020-09-09-224210   True        False         False      79m
storage                                    4.6.0-0.nightly-2020-09-09-224210   True        False         False      38m

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-09-09-224210   True        False         47m     Cluster version is 4.6.0-0.nightly-2020-09-09-224210

$ oc get nodes -o wide 
NAME                                 STATUS   ROLES    AGE   VERSION                INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINER-RUNTIME
wj46ioskr910a-klxng-master-0         Ready    master   85m   v1.19.0-rc.2+40d85fc   192.168.2.182   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-master-1         Ready    master   85m   v1.19.0-rc.2+40d85fc   192.168.0.142   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-master-2         Ready    master   86m   v1.19.0-rc.2+40d85fc   192.168.1.19    <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-khcrx   Ready    worker   67m   v1.19.0-rc.2+40d85fc   192.168.1.22    <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-ndjtw   Ready    worker   68m   v1.19.0-rc.2+40d85fc   192.168.1.129   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1
wj46ioskr910a-klxng-worker-0-q9c66   Ready    worker   66m   v1.19.0-rc.2+40d85fc   192.168.1.171   <none>        Red Hat Enterprise Linux CoreOS 46.82.202009091306-0 (Ootpa)   4.18.0-193.19.1.el8_2.x86_64   cri-o://1.19.0-11.rhaos4.6.gitf83564f.el8-rc.1


$ oc describe co authentication 
Name:         authentication
Namespace:    
Labels:       <none>
Annotations:  exclude.release.openshift.io/internal-openshift-hosted: true
API Version:  config.openshift.io/v1
Kind:         ClusterOperator
Metadata:
  Creation Timestamp:  2020-09-10T05:58:24Z
  Generation:          1
  Managed Fields:
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:exclude.release.openshift.io/internal-openshift-hosted:
      f:spec:
      f:status:
        .:
        f:extension:
    Manager:      cluster-version-operator
    Operation:    Update
    Time:         2020-09-10T05:58:24Z
    API Version:  config.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
        f:relatedObjects:
        f:versions:
    Manager:         authentication-operator
    Operation:       Update
    Time:            2020-09-10T06:48:59Z
  Resource Version:  77033
  Self Link:         /apis/config.openshift.io/v1/clusteroperators/authentication
  UID:               9aecec81-66e6-455f-a1b4-50d2deff6d3e
Spec:
Status:
  Conditions:
    Last Transition Time:  2020-09-10T06:37:36Z
    Reason:                AsExpected
    Status:                False
    Type:                  Degraded
    Last Transition Time:  2020-09-10T06:48:32Z
    Reason:                AsExpected
    Status:                False
    Type:                  Progressing
    Last Transition Time:  2020-09-10T06:48:46Z
    Message:               OAuthServerDeploymentAvailable: availableReplicas==2
    Reason:                AsExpected
    Status:                True
    Type:                  Available
    Last Transition Time:  2020-09-10T06:05:51Z
    Reason:                AsExpected
    Status:                True
    Type:                  Upgradeable
  Extension:               <nil>
  Related Objects:
    Group:      operator.openshift.io
    Name:       cluster
    Resource:   authentications
    Group:      config.openshift.io
    Name:       cluster
    Resource:   authentications
    Group:      config.openshift.io
    Name:       cluster
    Resource:   infrastructures
    Group:      config.openshift.io
    Name:       cluster
    Resource:   oauths
    Group:      route.openshift.io
    Name:       oauth-openshift
    Namespace:  openshift-authentication
    Resource:   routes
    Group:      
    Name:       oauth-openshift
    Namespace:  openshift-authentication
    Resource:   services
    Group:      
    Name:       openshift-config
    Resource:   namespaces
    Group:      
    Name:       openshift-config-managed
    Resource:   namespaces
    Group:      
    Name:       openshift-authentication
    Resource:   namespaces
    Group:      
    Name:       openshift-authentication-operator
    Resource:   namespaces
    Group:      
    Name:       openshift-ingress
    Resource:   namespaces
    Group:      
    Name:       openshift-oauth-apiserver
    Resource:   namespaces
  Versions:
    Name:     oauth-apiserver
    Version:  4.6.0-0.nightly-2020-09-09-224210
    Name:     operator
    Version:  4.6.0-0.nightly-2020-09-09-224210
    Name:     oauth-openshift
    Version:  4.6.0-0.nightly-2020-09-09-224210_openshift
Events:       <none>


Note You need to log in before you can comment on or make changes to this bug.