Bug 1764704

This looks like it's the same as Bug 1712525.

Hi Jakub, I used following steps to verify the bug

1. Generate a CA and certificate (for testing):

    BASE_DOMAIN="$(oc get dns.config/cluster -o 'jsonpath={.spec.baseDomain}')"
    INGRESS_DOMAIN="$(oc get ingress.config/cluster -o 'jsonpath={.spec.domain}')"
    openssl genrsa -out example-ca.key 2048
    openssl req -x509 -new -key example-ca.key -out example-ca.crt -days 1 -subj "/C=US/ST=NC/L=Chocowinity/O=OS3/OU=Eng/CN=$BASE_DOMAIN"
    openssl genrsa -out example.key 2048
    openssl req -new -key example.key -out example.csr -subj "/C=US/ST=NC/L=Chocowinity/O=OS3/OU=Eng/CN=*.$INGRESS_DOMAIN"
    openssl x509 -req -in example.csr -CA example-ca.crt -CAkey example-ca.key -CAcreateserial -out example.crt -days 1

2. Configure the certificate as the ingresscontroller's default certificate:
[yapei@dhcp-141-192 test-files]$ oc -n openshift-ingress create secret tls custom-default-cert --cert=example.crt --key=example.key
secret/custom-default-cert created
[yapei@dhcp-141-192 test-files]$ oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{"spec":{"defaultCertificate":{"name":"custom-default-cert"}}}'
ingresscontroller.operator.openshift.io/default patched

3. ingress pods are re-created
[yapei@dhcp-141-192 test-files]$ oc get pods -n openshift-ingress
NAME                              READY   STATUS    RESTARTS   AGE
router-default-798959bfd4-kzddm   1/1     Running   0          26s
router-default-798959bfd4-l6snk   1/1     Running   0          38s

4. Check console accessibility, opening console still have issues. 

But it looks like this is blocked by a new issue which is opened at bug 1772759

Also after we configure ingress operator to use custom certificate, no cm/router-ca in openshift-config-managed namespace, this is tracked in another bug 1772775

I tried again follow the steps in comment 4

This time bug 1772759 was not reproduced, but accessing console still met the same issue(got error page and looping, see attached)


$ oc logs -f console-operator-68c4b88777-kgfql -n openshift-console-operator
I1119 06:00:29.143631       1 event.go:255] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-console-operator", Name:"console-operator", UID:"b005894c-c166-4282-9d28-a12793b596e6", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'TargetConfigDeleted' Deleted target configmap openshift-console/router-ca because source config does not exist

$ oc logs -f console-657df767dc-mqvpl -n openshift-console
2019/11/19 06:05:12 server: authentication failed: unauthenticated
2019/11/19 06:05:12 auth: failed to get latest auth source data: request to OAuth issuer endpoint https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com/oauth/token failed: Head https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com: x509: certificate signed by unknown authority
2019/11/19 06:05:12 server: authentication failed: unauthenticated
2019/11/19 06:05:12 server: authentication failed: unauthenticated
2019/11/19 06:05:12 auth: failed to get latest auth source data: request to OAuth issuer endpoint https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com/oauth/token failed: Head https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com: x509: certificate signed by unknown authority
2019/11/19 06:05:12 auth: failed to get latest auth source data: request to OAuth issuer endpoint https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com/oauth/token failed: Head https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com: x509: certificate signed by unknown authority
2019/11/19 06:05:12 auth: unable to verify auth code with issuer: Post https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com/oauth/token: x509: certificate signed by unknown authority
2019/11/19 06:05:13 auth: failed to get latest auth source data: request to OAuth issuer endpoint https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com/oauth/token failed: Head https://oauth-openshift.apps.qe-yapei43.qe.devcluster.openshift.com: x509: certificate signed by unknown authority
2019/11/19 06:05:13 server: authentication failed: unauthenticated
2019/11/19 06:05:13 server: authentication failed: unauthenticated
2019/11/19 06:05:13 server: authentication failed: unauthenticated
2019/11/19 06:05:13 server: authentication failed: unauthenticated



Confirmed fix PR already landed in 4.3.0-0.nightly-2019-11-18-175710
$ oc get pods -n openshift-console-operator -o yaml | grep -i image
      - name: IMAGE
      image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d83281dcc25dddd4ac7f0c58e488cdacfec5d47568ed278143f8fe86e5ecc98
      imagePullPolicy: IfNotPresent
    imagePullSecrets:
      image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d83281dcc25dddd4ac7f0c58e488cdacfec5d47568ed278143f8fe86e5ecc98
      imageID: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d83281dcc25dddd4ac7f0c58e488cdacfec5d47568ed278143f8fe86e5ecc98
$ oc image info quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6d83281dcc25dddd4ac7f0c58e488cdacfec5d47568ed278143f8fe86e5ecc98 | grep -i commit
Environment: SOURCE_GIT_COMMIT=365e1e43ea2c523c040324e4f8e674bf819732e4
             io.openshift.build.commit.id=365e1e43ea2c523c040324e4f8e674bf819732e4
             io.openshift.build.commit.url=https://github.com/openshift/console-operator/commit/365e1e43ea2c523c040324e4f8e674bf819732e4
[yapei@dhcp-141-192 console-operator]$ git log 365e1e43ea2c523c040324e4f8e674bf819732e4 | grep '#328'
    Merge pull request #328 from jhadvig/bz1764704


Let me know if the steps are wrong

Based on https://bugzilla.redhat.com/show_bug.cgi?id=1772775#c1, it looks like the proposed fix won't work. I'm going to duplicate this to bug 1712525 since they're really the same issue.

*** This bug has been marked as a duplicate of bug 1712525 ***

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days