Created attachment 1638119 [details] Failed build log Description of problem: This might be related to fix needed for https://bugzilla.redhat.com/show_bug.cgi?id=1771335. In addition this is likely a corner case of running incremental S2I build. We have "incrementalRebuild" test, that defines it's own BuildConfig with "forcePull:true" property and from as direct DockerImage reference. The test is passing on 4.2.x. On 4.3.0 nighly it started to fail on image pull from internal registry when base image from *authenticated registry* is used. I was also able to reproduce this behavior also on ImageStream source when "forcePull:true" is added, see the steps below. From my tests so far it's *only* reproducible with "forcePull:true" property. ``` apiVersion: build.openshift.io/v1 kind: BuildConfig metadata: creationTimestamp: 2019-11-20T11:34:52Z name: sti namespace: dsimansk resourceVersion: "67319" selfLink: /apis/build.openshift.io/v1/namespaces/dsimansk/buildconfigs/sti uid: 71e8a5ff-15da-4091-a01a-741a680441af spec: failedBuildsHistoryLimit: 5 nodeSelector: {} output: to: kind: ImageStreamTag name: sti:latest postCommit: {} resources: {} runPolicy: Serial source: binary: {} type: Binary strategy: sourceStrategy: forcePull: true from: kind: DockerImage name: registry.redhat.io/openjdk/openjdk-11-rhel8:latest incremental: true type: Source successfulBuildsHistoryLimit: 5 triggers: - generic: allowEnv: true secret: secret101 type: Generic status: lastVersion: 2 ``` Version-Release number of selected component (if applicable): 4.3.0-0.nightly-2019-11-19-053808 How reproducible: Always Steps to Reproduce: 1. Create pull secret for registry.redhat.io 2. Link them to builder and deployer SAs: https://docs.openshift.com/container-platform/4.2/openshift_images/managing-images/using-image-pull-secrets.html#images-allow-pods-to-reference-images-from-secure-registries_using-image-pull-secrets 3. oc new-app registry.redhat.io/openjdk/openjdk-11-rhel8:latest~https://github.com/openshift-vertx-examples/vertx-http-example.git - image from authenticated registry must be used 4. Add "forcePull: true" to BC vertx-http-example - property must be set to true 5. oc start-build --incremental -e BUILD_LOGLEVEL=8 vertx-http-example Actual results: Build fails. Expected results: Build is successfully completed. Additional info:
Think I see the bug here. Using `from` with an incremental s2i build is not necessary - incremental s2i builds pull from the output image. However it appears that with `forcePull: true`, we are not setting the authentication for the true "base" image (the internal registry).
Moving target to 4.4.0, as the proposed fix warrants more discussion.
10-4 re: 4.4 fwiw I was able to execute the test case running a version of the openshift builder image with the changes from https://github.com/openshift/builder/pull/116 1) used BC of apiVersion: build.openshift.io/v1 kind: BuildConfig metadata: name: sti spec: output: to: kind: ImageStreamTag name: sti:latest postCommit: {} resources: {} runPolicy: Serial source: binary: {} type: Binary strategy: sourceStrategy: forcePull: true from: kind: DockerImage name: registry.redhat.io/openjdk/openjdk-11-rhel8:latest incremental: true type: Source 2) created a secret for my registry.redhat.io credentials 3) linked the secret to the default and builder SAs 4) ran the build from my local clone of the openshift jenkins plugin maven/java based repo (as the binary build input)
Verified in version: 4.4.0-0.nightly-2020-02-02-225006 Steps: 1. Create a bc with follow content: sourceStrategy: forcePull: true from: kind: DockerImage name: registry.redhat.io/openjdk/openjdk-11-rhel8:latest incremental: true 2. Create pull secret for registry.redhat.io 3. Link builder to secret 4. Set pull build-secret 5. Start a build vertx-http-example-3 Source Git@71ef489 Complete 8 minutes ago 4m10s 6. Check build log [root@wangwen ~]# oc logs -f build/vertx-http-example-3 Cloning "https://github.com/openshift-vertx-examples/vertx-http-example.git" ... Commit: 71ef48965c062a04a5bdaac9b1641ccc9e6034a0 (Merge pull request #91 from openshift-vertx-examples/dependabot/maven/io.fabri..) Author: Rodney Russ <rdruss> Date: Fri Sep 20 11:00:47 2019 -0600 Caching blobs under "/var/cache/blobs". Getting image source signatures Copying blob sha256:0e8ea260d0262eac3725175d3d499ead6fd77cb1fa8272b3e665e8f64044fb89 Copying blob sha256:340ff6d7f58c908c438ce89bb845caee3649de828a81569317ed0fe169a97de2 Copying blob sha256:9839973fd99c42bc282e1814bcbbca41770f977a30e44fcb29fb09e13a67ce8a Copying config sha256:1ccd33f066a61debc231fd56f3cfa0b5a096f02d3273f0e60ed711728012607a Writing manifest to image destination Storing signatures Getting image source signatures Copying blob sha256:0e8ea260d0262eac3725175d3d499ead6fd77cb1fa8272b3e665e8f64044fb89 Copying blob sha256:340ff6d7f58c908c438ce89bb845caee3649de828a81569317ed0fe169a97de2 Copying blob sha256:9839973fd99c42bc282e1814bcbbca41770f977a30e44fcb29fb09e13a67ce8a Copying blob sha256:d649e8c0ed19c70ab4b642ff46bccdba048dac3cb9ac9f3a868d4cd68a72082d Copying config sha256:4321e1a2bfb3ee7bf52ac6826cb8cec4151192c14810f80baff47076813fcf40 Writing manifest to image destination Storing signatures Generating dockerfile with builder image registry.redhat.io/openjdk/openjdk-11-rhel8:latest STEP 1: FROM image-registry.openshift-image-registry.svc:5000/wewang/vertx-http-example:latest AS cached
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581