We are running OpenShift Pipeline 1.1.1 on Openshift 4.5.8 on Azure Cloud (subscription: preview). Tried to create the following Buildah task in which it is not able to pull the image from Openshift 4.5 internal registry. I have now tried with "image-registry.openshift-image-registry.svc" and this "default-route-openshift-image-registry.app.testing.oc.company.com". Also tried to add the pull secret and its token to service account "pipeline" (which was created by default by the Openshift Pipeline operator) --- $ oc get sa pipeline -o yaml apiVersion: v1 imagePullSecrets: - name: pipeline-dockercfg-v2w68 kind: ServiceAccount metadata: creationTimestamp: "2020-09-01T21:50:00Z" name: pipeline namespace: app resourceVersion: "31842524" selfLink: /api/v1/namespaces/app/serviceaccounts/pipeline uid: 1e283908-0523-460b-bf2b-bc257faa6bc7 secrets: - name: deployer-token-8phfl - name: deployer-dockercfg-zmt64 - name: pipeline-token-jf5dh - name: pipeline-dockercfg-v2w68 - name: gitreposecret --- Docker file: "app1.Dockerfile" FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest --- How the image looks on Openshift: --- $ oc get is NAME IMAGE REPOSITORY TAGS UPDATED app1 default-route-openshift-image-registry.app.testing.oc.company.com/app/app1 latest 14 hours ago --- Error on Buildah task: --- + df -h /var/lib/containers Filesystem Size Used Avail Use% Mounted on /dev/sdd 30G 45M 30G 1% /var/lib/containers + buildah --storage-driver vfs bud -f /workspace/sourcerepos/pipeline/dockerfiles/app1.Dockerfile -t image-registry.openshift-image-registry.svc:5000/app/app1-timer:latest /workspace/sourcerepos/artefacts STEP 1: FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest error creating build container: Error initializing source docker://image-registry.openshift-image-registry.svc:5000/app/app1:latest: error pinging docker registry image-registry.openshift-image-registry.svc:5000: Get "https://image-registry.openshift-image-registry.svc:5000/v2/": x509: certificate signed by unknown authority --- Buildah task: --- apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: bake-image spec: params: - name: imageTag type: string description: The image tag - name: imageName type: string description: The image name - name: dockerfile type: string description: The dockerfile - name: contextPath type: string description: The context to run the commands - name: pvc type: string description: The PVC to use for layers workspaces: - name: sourcerepos steps: - name: build image: quay.io/buildah/stable:latest securityContext: privileged: true script: | df -h /var/lib/containers buildah --storage-driver vfs bud \ -f /workspace/sourcerepos/pipeline/dockerfiles/$(params.dockerfile) \ -t image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag) \ $(params.contextPath) buildah --storage-driver vfs push \ --tls-verify=false \ image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag) df -h /var/lib/containers volumeMounts: - mountPath: /var/lib/containers name: varlibcontainers resources: requests: memory: "4Gi" cpu: "1" ephemeral-storage: "1Gi" limits: ephemeral-storage: "1Gi" volumes: - name: varlibcontainers persistentVolumeClaim: claimName: $(params.pvc)
I see similar errors running Shipwright builds (which leverages Tekton) and reference the internal registry as my base image. There are two reasons for this: 1. In 4.x the internal registry serves all traffic via HTTPS 2. The internal registry's service uses a cluster-signed certificate that is not signed by a globally trusted certificate authority. To immediately work around this issue, you can add the `--tls-verify=false` flag to your buildah commands: ``` $ buildah --storage-driver vfs bud --tls-verify false \ ... ``` However, this does not address the root issue of containers not trusting the internal registry. For that to happen: 1. The CA for the internal registry needs to be mounted into the Tekton task steps 2. The CA needs to be added to /etc/pki/ca-trust/source/anchors, and subsequently the command `update-ca-trust extract` needs to be run as root
Thank you, that worked. In order to fix it properly, could you please elaborate step 1? --- Suggested Workaround: buildah --storage-driver vfs bud \ --tls-verify=false \ ---
I think this should be fixed by 1.4 (mounting openshift internal certs in, …) cc @shmingla
@vdemeest yep!
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days