Description of problem: I was unable to configure insecureRegistries as per https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html#images-configuration-cas_image-configuration Version-Release number of selected component (if applicable): 4.4.0-rc.9 How reproducible: Always Steps to Reproduce: Insecure registry hosted at: https://docker-registry-default.apps.ocp.example.com This shall be used to deploy applications from within OpenShift Container Platform 4.3.12, both via CLI as well as the web console. ---------- ~~~ $ curl -I https://docker-registry-default.apps.ocp.example.com curl: (60) SSL certificate problem: self signed certificate in certificate chain # cat config.json { "auths": { "docker-registry-default.apps.ocp.example.com": { "auth": "asdasdasdasdasdasdasda" } } } # oc new-project case02630065 # oc create secret generic ckoep --from-file=.dockerconfigjson=config.json --type=kubernetes.io/dockerconfigjson secret/ckoep created # oc get image.config.openshift.io/cluster -o yaml apiVersion: config.openshift.io/v1 kind: Image metadata: annotations: release.openshift.io/create-only: "true" creationTimestamp: "2020-04-14T13:33:56Z" generation: 7 name: cluster resourceVersion: "4296983" selfLink: /apis/config.openshift.io/v1/images/cluster uid: 0bc5fd63-0e6d-4ba9-ace8-118dfc917f19 spec: {} status: internalRegistryHostname: image-registry.openshift-image-registry.svc:5000 # oc new-app --docker-image=docker-registry-default.apps.ocp.example.com/foo/ruby-ex:latest --name=insecuresource W0422 14:48:21.596901 8486 dockerimagelookup.go:236] container image registry lookup failed: docker-registry-default.apps.ocp.example.com/foo/ruby-ex:latest: Get https://docker-registry-default.apps.ocp.example.com/v2/: x509: certificate signed by unknown authority ~~~ The web interface reports the same, graphically: ~~~ Internal error occurred: docker-registry-default.apps.ocp.example.com/foo/ruby-ex:latest: Get https://docker-registry-default.apps.ocp.example.com/v2/: x509: certificate signed by unknown authority ~~~ So far, so good. After editing image.config.openshift.io/cluster according to https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html#images-configuration-cas_image-configuration we end up with: ~~~ # oc get image.config.openshift.io/cluster -o yaml apiVersion: config.openshift.io/v1 kind: Image metadata: annotations: release.openshift.io/create-only: "true" creationTimestamp: "2020-04-14T13:33:56Z" generation: 8 name: cluster resourceVersion: "4310066" selfLink: /apis/config.openshift.io/v1/images/cluster uid: 0bc5fd63-0e6d-4ba9-ace8-118dfc917f19 spec: registrySources: insecureRegistries: - docker-registry-default.apps.ocp.example.com status: internalRegistryHostname: image-registry.openshift-image-registry.svc:5000 ~~~ After waiting for every node to pick up the changes, /etc/containers/registries.conf looks as follows on all nodes: ~~~ # oc debug node/ocp4-cktqs-worker-kzp7t -- chroot /host cat /etc/containers/registries.conf Starting pod/ocp4-cktqs-worker-kzp7t-debug ... To use host binaries, run `chroot /host` unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] [[registry]] prefix = "" location = "docker-registry-default.apps.ocp.example.com" insecure = true Removing debug pod ... ~~~ However, the issue persists. ~~~ # oc new-app --docker-image=docker-registry-default.apps.ocp.example.com/foo/ruby-ex:latest --name=insecuresource W0422 15:12:57.563461 23746 dockerimagelookup.go:236] container image registry lookup failed: docker-registry-default.apps.ocp.example.com/foo/ruby-ex:latest: Get https://docker-registry-default.apps.ocp.example.com/v2/: x509: certificate signed by unknown authority ~~~ Actual results: x509: certificate signed by unknown authority Expected results: Successful usage of an insecure registry Additional info: Is it possible that I misinterpret what this configuration is supposed to do?
new-app itself is trying to reach the registry to inspect the image you want to deploy. So your client machine (the one where you are running new-app) needs to be able to trust the registry also. That is where the error is coming from, not the cluster/node. you can pass --insecure-registry to new-app to get around this.
Alternatively, you can add a CA to the cluster with these instructions: https://docs.openshift.com/container-platform/4.3/builds/setting-up-trusted-ca.html#configmap-adding-ca_setting-up-trusted-ca
Adding a CA to the cluster won't help, this is purely a client side operation within oc new-app, it doesn't use cluster CAs. (adding the CA will help when it comes time for the cluster to actually pull the image, though, after new-app sets up all the resources).
(In reply to Ryan Phillips from comment #2) > Alternatively, you can add a CA to the cluster with these instructions: > > https://docs.openshift.com/container-platform/4.3/builds/setting-up-trusted- > ca.html#configmap-adding-ca_setting-up-trusted-ca I can confirm that this works, thank you. (In reply to Ben Parees from comment #1) > new-app itself is trying to reach the registry to inspect the image you want > to deploy. So your client machine (the one where you are running new-app) > needs to be able to trust the registry also. That is where the error is > coming from, not the cluster/node. > > you can pass --insecure-registry to new-app to get around this. Thank you Ben for your update. I can see how the setting does not affect the "oc new-app" command based on the way it actually works (it does not use or care about /etc/containers/registries.conf on the nodes). However, the same behaviour applies to the developer console - in which we do not have a way to "accept insecure" registries. https://i.imgur.com/lyh41Uy.png This is a problem for customers looking to migrate workloads from Kubernetes ("non-OpenShift") or from OpenShift Container Platform 3. Hence I'll move this bug to the Dev Console team as discussed with Ben.
Verified: OpenShift Version: 4.6.0-0.nightly-2020-06-16-214732 Kubernetes Version :v1.18.3+e1ba7b6 cluster URL: https://console-openshift-console.apps.ci-ln-w51vw82-d5d6b.origin-ci-int-aws.dev.rhcloud.com/ Browser: Chrome
I have validated the bugzilla on: Version:4.6.0-0.nightly-2020-07-05-234845 Browser: Google Chrome Version 81.0.4044.129 Marking this as verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4196
*** Bug 1902344 has been marked as a duplicate of this bug. ***