Description: A docker type build on OpenShift 4.0 fails because it is using a Dockerfile from the wrong directory, or mixing cached image layers from a second build in the same project, when using spec.source.contextDir, and the two builds in the same project are for the same Git repository. The same code base builds fine on 3.11, and also in an automated build on quay.io.

Affects Version: 4.1

oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth

Server https://api.cluster-9586.9586.openshiftworkshop.com:6443
kubernetes v1.12.4+0f8e04e

Steps To Reproduce:

In a 4.0 cluster run:

oc new-app https://raw.githubusercontent.com/openshift-labs/workshop-spawner/develop/templates/jumpbox-server-development.json
In addition to some deployments, this creates two build configurations, both using the same Git repository, but different contextDir within the same Git repository. The separate directories used for the build both have a Dockerfile.

Current Result
The build configuration for the custom keycloak image fails with:

Generating dockerfile with builder image jboss/keycloak-openshift:4.0.0.Final
STEP 1: FROM jboss/keycloak-openshift:4.0.0.Final
STEP 2: LABEL "io.openshift.build.source-location"="/tmp/build/inputs" "io.openshift.build.source-context-dir"="keycloak" "io.openshift.build.image"="jboss/keycloak-openshift:4.0.0.Final"
STEP 3: ENV OPENSHIFT_BUILD_NAME="jumpbox-keycloak-2" OPENSHIFT_BUILD_NAMESPACE="workshop"
STEP 4: USER root
STEP 5: COPY upload/scripts /tmp/scripts
STEP 6: COPY upload/src /tmp/src
STEP 7: RUN chown -R 1001:0 /tmp/scripts /tmp/src
time="2019-04-05T01:35:57Z" level=warning msg="pkg/chroot: error unmounting \"/tmp/buildah180432153/mnt/rootfs\": error checking if \"/tmp/buildah180432153/mnt/rootfs/sys/fs/cgroup/blkio\" is mounted: no such file or directory"
time="2019-04-05T01:35:57Z" level=warning msg="pkg/bind: error unmounting \"/tmp/buildah180432153/mnt/rootfs\": error checking if \"/tmp/buildah180432153/mnt/rootfs/sys/fs/cgroup/blkio\" is mounted: no such file or directory"
STEP 8: USER 1001
STEP 9: RUN /tmp/scripts/assemble
+ '[' -f /tmp/src/jupyterhub-realm.json ']'
+ mv /tmp/src/jupyterhub-realm.json .
mv: cannot move ‘/tmp/src/jupyterhub-realm.json’ to ‘./jupyterhub-realm.json’: Permission denied
subprocess exited with status 1
subprocess exited with status 1
error: build error: error building at step {Env:[OPENSHIFT_BUILD_NAMESPACE=workshop OPENSHIFT_BUILD_NAME=jumpbox-keycloak-2 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin JAVA_HOME=/usr/lib/jvm/java KEYCLOAK_VERSION=4.0.0.Final LAUNCH_JBOSS_IN_BACKGROUND=1 PROXY_ADDRESS_FORWARDING=false JDBC_POSTGRES_VERSION=42.2.2 JDBC_MYSQL_VERSION=5.1.46 JDBC_MARIADB_VERSION=2.2.3 JBOSS_HOME=/opt/jboss/keycloak LANG=en_US.UTF-8   OPENSHIFT_BUILD_NAME=jumpbox-keycloak-2 OPENSHIFT_BUILD_NAMESPACE=workshop] Command:run Args:[/tmp/scripts/assemble] Flags:[] Attrs:map[] Message:RUN /tmp/scripts/assemble Original:RUN /tmp/scripts/assemble}: exit status 1
The keycloak/Dockerfile it was supposed to use contained:

FROM jboss/keycloak-openshift:4.0.0.Final

USER root

COPY . /tmp/src

RUN mv /tmp/src/.s2i/bin /tmp/scripts

RUN rm -rf /tmp/src/.git* && \
    chown -R 1000 /tmp/src && \
    chgrp -R 0 /tmp/src && \
    chmod -R g+w /tmp/src

USER 1000

RUN /tmp/scripts/assemble

CMD [ "/tmp/scripts/run" ]
This Dockerfile uses user ID 1000. The log of the build show it using user ID 1001.

The user ID 1001 appears in a separate jupyterhub/Dockerfile, which is what the other build was using.

It appears to be either using the wrong Dockerfile, or more likely, is using a cached image layer from the wrong build context, with the log messages show what was the original command used when it built the layer it did use.

Not sure if this is specifically tied to fact that two builds use same Git repository, but with different contextDir, or whether it more general problem.

Expected Result
Build should complete okay.

Additional Information
The key part of the build configuration for the custom keycloak image is:

spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: jumpbox-keycloak:latest
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    contextDir: keycloak
    git:
      ref: develop
      uri: https://github.com/openshift-labs/workshop-spawner.git
    type: Git
  strategy:
    sourceStrategy:
      from:
        kind: DockerImage
        name: jboss/keycloak-openshift:4.0.0.Final
    type: Source
See the template used in the oc new-app command above for more details of the builds.