Bug 1879680 - Pipelines not working with Openshift 4.5 internal image registry in buildah task
Summary: Pipelines not working with Openshift 4.5 internal image registry in buildah task
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenShift Pipelines
Classification: Red Hat
Component: pipelines
Version: 1.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: 1.0
Assignee: Vincent Demeester
QA Contact: Ruchir Garg
Robert Krátký
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-16 19:04 UTC by Abu Davis
Modified: 2024-06-13 23:05 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-05-06 07:24:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github tektoncd pipeline issues 3243 0 None open Pipelines not working with Openshift 4.5 internal registry 2020-09-28 03:25:34 UTC

Description Abu Davis 2020-09-16 19:04:33 UTC
We are running OpenShift Pipeline 1.1.1 on Openshift 4.5.8 on Azure Cloud (subscription: preview).

Tried to create the following Buildah task in which it is not able to pull the image from Openshift 4.5 internal registry. I have now tried with "image-registry.openshift-image-registry.svc" and this "default-route-openshift-image-registry.app.testing.oc.company.com". Also tried to add the pull secret and its token to service account "pipeline" (which was created by default by the Openshift Pipeline operator)

---
$ oc get sa pipeline -o yaml
apiVersion: v1
imagePullSecrets:
- name: pipeline-dockercfg-v2w68
kind: ServiceAccount
metadata:
  creationTimestamp: "2020-09-01T21:50:00Z"
  name: pipeline
  namespace: app
  resourceVersion: "31842524"
  selfLink: /api/v1/namespaces/app/serviceaccounts/pipeline
  uid: 1e283908-0523-460b-bf2b-bc257faa6bc7
secrets:
- name: deployer-token-8phfl
- name: deployer-dockercfg-zmt64
- name: pipeline-token-jf5dh
- name: pipeline-dockercfg-v2w68
- name: gitreposecret
---

Docker file: "app1.Dockerfile"
FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest
---

How the image looks on Openshift:
---
$ oc get is 
NAME                         IMAGE REPOSITORY                                                                             TAGS     UPDATED
app1                     default-route-openshift-image-registry.app.testing.oc.company.com/app/app1                     latest   14 hours ago
---

Error on Buildah task:
---
+ df -h /var/lib/containers
Filesystem Size Used Avail Use% Mounted on
/dev/sdd 30G 45M 30G 1% /var/lib/containers
+ buildah --storage-driver vfs bud -f /workspace/sourcerepos/pipeline/dockerfiles/app1.Dockerfile -t image-registry.openshift-image-registry.svc:5000/app/app1-timer:latest /workspace/sourcerepos/artefacts
STEP 1: FROM image-registry.openshift-image-registry.svc:5000/app/app1:latest
error creating build container: Error initializing source docker://image-registry.openshift-image-registry.svc:5000/app/app1:latest: error pinging docker registry image-registry.openshift-image-registry.svc:5000: Get "https://image-registry.openshift-image-registry.svc:5000/v2/": x509: certificate signed by unknown authority
---

Buildah task:
---

apiVersion: tekton.dev/v1beta1
  kind: Task
  metadata:
    name: bake-image
  spec:
    params:
      - name: imageTag
        type: string
        description: The image tag
      - name: imageName
        type: string
        description: The image name
      - name: dockerfile
        type: string
        description: The dockerfile
      - name: contextPath
        type: string
        description: The context to run the commands
      - name: pvc
        type: string
        description: The PVC to use for layers
    workspaces:
      - name: sourcerepos
    steps:
      - name: build
        image: quay.io/buildah/stable:latest
        securityContext:
          privileged: true
        script: |
          df -h /var/lib/containers
          buildah --storage-driver vfs bud \
            -f /workspace/sourcerepos/pipeline/dockerfiles/$(params.dockerfile) \
            -t image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag) \
            $(params.contextPath)
          buildah --storage-driver vfs push \
            --tls-verify=false \
            image-registry.openshift-image-registry.svc:5000/app/$(params.imageName):$(params.imageTag)
          df -h /var/lib/containers
        volumeMounts:
           - mountPath: /var/lib/containers
             name: varlibcontainers
        resources:
          requests:
            memory: "4Gi"
            cpu: "1"
            ephemeral-storage: "1Gi"
          limits:
            ephemeral-storage: "1Gi"
    volumes:
      - name: varlibcontainers
        persistentVolumeClaim:
          claimName: $(params.pvc)

Comment 1 Adam Kaplan 2020-09-17 19:41:11 UTC
I see similar errors running Shipwright builds (which leverages Tekton) and reference the internal registry as my base image. There are two reasons for this:

1. In 4.x the internal registry serves all traffic via HTTPS
2. The internal registry's service uses a cluster-signed certificate that is not signed by a globally trusted certificate authority.

To immediately work around this issue, you can add the `--tls-verify=false` flag to your buildah commands:

```
$ buildah --storage-driver vfs bud --tls-verify false \
... 
```

However, this does not address the root issue of containers not trusting the internal registry. For that to happen:

1. The CA for the internal registry needs to be mounted into the Tekton task steps
2. The CA needs to be added to /etc/pki/ca-trust/source/anchors, and subsequently the command `update-ca-trust extract` needs to be run as root

Comment 2 Abu Davis 2020-09-17 23:06:21 UTC
Thank you, that worked. In order to fix it properly, could you please elaborate step 1?
---
Suggested Workaround:
buildah --storage-driver vfs bud \
 --tls-verify=false \
---

Comment 5 Vincent Demeester 2021-08-04 05:34:32 UTC
I think this should be fixed by 1.4 (mounting openshift internal certs in, …)
cc @shmingla

Comment 6 Shubham 2022-05-06 06:51:40 UTC
@vdemeest yep!

Comment 7 Red Hat Bugzilla 2023-09-15 01:30:37 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days


Note You need to log in before you can comment on or make changes to this bug.