Bug 1684427

To avoid the confusion I faced, this is the dockerfile in question:
https://github.com/xiuwang/multi-stage-builds/blob/externalimage/Dockerfile

I assume that buildah is injecting the "FROM nginx:latest" message as a result of this:
https://github.com/xiuwang/multi-stage-builds/blob/externalimage/Dockerfile#L13

(and it probably has to, in order to be able to copy from that image...  it is slightly confusing that it prints it, but it's probably correct).

Moving this to the container runtimes team since they own buildah.

(I would agree that buildah should not "FROM" the image multiple times, if the same image is referenced multiple times, that seems like a legitimate bug).

(In reply to Ben Parees from comment #1)
> To avoid the confusion I faced, this is the dockerfile in question:
> https://github.com/xiuwang/multi-stage-builds/blob/externalimage/Dockerfile
> 
> I assume that buildah is injecting the "FROM nginx:latest" message as a
> result of this:
> https://github.com/xiuwang/multi-stage-builds/blob/externalimage/
> Dockerfile#L13
> 
> (and it probably has to, in order to be able to copy from that image...  it
> is slightly confusing that it prints it, but it's probably correct).

I think you're correct about what's happening - internally we're prepending FROMs so that we'll have a container root filesystem, based on the specified image, that we can use instead of the build context's root.

Any chance this is fixed in buildah 1.7?

This one might be fixed, let me see if I can test/verify.

No luck on the prior fix, it still is a problem.  I'm going to guess it's somewhere in the imagebuilder stageing prep code.

Forgot to add, Docker does not resort the order like we're apparently doing.

IDK why, but something wonky is going on due to this line in the Dockerfile it appears:

COPY --from=nginx:latest /etc/nginx/nginx.conf /tmp/bar

I should have caught this earlier, apologies on not doing so.  This functionality is due to this PR: https://github.com/containers/buildah/pull/1181.  Prior to this PR, any COPY commands with a '--from' directive were failing unless the image specified in the --from directive had been pulled previously.  The changes created in this PR first goes through the Dockerfile looking for any "COPY --from" statements and if found, prepends a FROM directive to the Dockerfile in memory and then the Dockerfile is run.  That way we ensure that the image is present by the time the 'COPY --from' is encountered.

This does cause the output of the 'podman build' or 'buildah bud' commands to differ from 'docker build' output.  I believe that the resulting container is equivalent though.

We could possibly move the pre-pended FROM statement to live just above the 'COPY --from' statement in the internal Dockerfile, but I think we still would have a difference in the output between Docker and Buildah/Podman.  I'm not sure we can get around that entirely, at least not easily.  

Nalin might have other thoughts, but won't be checking in until early next week.

I've got some more changes for our multistage logic coming soon, and I think this can be sorted (by pulling the named image if it isn't found in local storage at the time we encounter the COPY instruction) after that effort.  The current set of changes that it would build on aren't expected for this next release, though, so I think we need to defer this a bit.

Okay, this should be fixed in the buildah library since https://github.com/containers/buildah/pull/1489 was merged.  We'll need to update openshift/builder to include it or a newer version in order for the changes to take effect for OpenShift builds.

Can't reproduce this issue with 4.1.0-0.nightly-2019-05-14-202907 payload. Could we mark this to on_qa, then I can verify it?

Here is build logs:

STEP 1: FROM centos:7 AS test
STEP 2: USER 1001
STEP 3: FROM centos:7
STEP 4: COPY --from=test /usr/bin/curl /test/
STEP 5: COPY --from=busybox:latest /bin/echo /test/
STEP 6: COPY --from=busybox:latest /bin/ping /test/
STEP 7: ENV "OPENSHIFT_BUILD_NAME"="multi-stage" "OPENSHIFT_BUILD_NAMESPACE"="e2e-test-build-multistage-7w2ck"

Here is my build:

    source:
      dockerfile: |2

        FROM scratch as test
        USER 1001
        FROM centos:7
        COPY --from=test /usr/bin/curl /test/
        COPY --from=busybox:latest /bin/echo /test/
        COPY --from=busybox:latest /bin/ping /test/
      images:
      - as:
        - scratch
        from:
          kind: DockerImage
          name: centos:7
        paths: null
      type: Dockerfile

As comment #13, move this bug to verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922