Bug 1649063

+++ This bug was initially created as a clone of Bug #1648775 +++

Description of problem: 


You should be able to reproduce the issue by creating a namespace called base and put a docker Image in the namespace called web:latest now you create another namespace let’s call it test and allow the service accounts in test namespace to pull images from the base namespace.
In the test namespace you start a binary build that has a dockerfile with FROM docker-registry.default.svc:5000/base/web


The partial build success results from the fact that the requested image is locally available in the docker pool.

A workaround is to add the pull secret of the builder service account to the dockerStrategy like this:
You have to make sure that the image is not already pulled on the node where your build is running.

A workaround is to add the pull secret of the builder service account to the dockerStrategy like this:
strategy:
    dockerStrategy:
      pullSecret:
        name: builder-dockercfg-1775b

another workaround is to reference the ImageStreamTag (this will override the first FROM in the Dockerfile) like this:
strategy:
    dockerStrategy:
      from:
        kind: ImageStreamTag
        name: 'mip-web:latest'
        namespace: base

The workarounds are not feasible as the user need to change all the pipelines to incorporate the workaround.

Anyway the normal k8s behavior is that pullSecret is mounted in the pod (https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller)
----
3. If the pod does not contain any ImagePullSecrets, then ImagePullSecrets of the ServiceAccount should be added to the pod.
----
1] Nov 06 11:05:46 master.example.com atomic-openshift-master-api[50426]: I1106 11:05:46.400209   50426 rbac.go:116] RBAC DENY: user "system:anonymous" groups ["system:unauthenticated"] cannot "get" resource "imagestreams/layers.image.openshift.io" named "mip-web" in namespace "base"




Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
oc new-project base
oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git -n base

cat <<EOF | oc create -n base -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: image-pull-access-fptest
  namespace: base
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: 'system:image-puller'
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: 'system:serviceaccounts:fptest'
EOF

oc new-project fptest
oc create is myimage -n fptest

cat <<EOF | oc create -n fptest -f -
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  name: binary-build
  namespace: fptest
spec:
  failedBuildsHistoryLimit: 5
  output:
    to:
      kind: ImageStreamTag
      name: 'myimage:latest'
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    binary: {}
    type: Binary
  strategy:
    dockerStrategy: {}
    type: Docker
  successfulBuildsHistoryLimit: 5
  triggers: []
EOF

echo "FROM docker-registry.default.svc:5000/base/ruby-25-centos7" > Dockerfile

oc start-build binary-build --from-file=Dockerfile -n fptest

Actual results: Binary builds doesnt automatically include the pull secret from the service account during the build.


Expected results: We expect the build pod has mounted the pullSecret of its SA, even when it is not explicitly configured (https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller point 3):
"If the pod does not contain any ImagePullSecrets, then ImagePullSecrets of the ServiceAccount are added to the pod."


Additional info:

--- Additional comment from Ben Parees on 2018-11-12 09:21:11 EST ---

Does this occur w/ a non-binary build?

--- Additional comment from Ben Parees on 2018-11-12 15:04:01 EST ---

The k8s documented behavior has no bearing on this because builds must do their own determination of what secret to use because builds can reference images in different ways than pods do, and the secrets that k8s mounts into the build pod are not the secrets the build will use.

I do not think this is specific to binary builds.  Any build where you have not explicitly told us in the buildconfig, what image you will be referencing, will have this issue.

The build must make the decision, prior to starting the build pod, what secrets to mount into the build pod so that docker can pull the images.  Since we don't know what your dockerfile looks like until after the build pod starts (at which point the dockerfile is either streamed in (binary build) or git cloned(normal build)), we can't mount it in advance.

Yes, it is technically possible that we can fallback to using the builder service account's docker secret in this scenario, but that will only solve the problem for builds that reference the internal registry.  You would still have this problem if you referenced some other private registry.

As you've noted, the solutions are either:

1) explicitly indicate the image you want to reference via the buildconfig's From field
2) explicitly indicate the secret you want to use to pull the image via the buildconfig's pullsecret field.

I will work an update that causes us to fallback to the service account's docker secret(if available) when we have no other information about what image is being pulled (as is the case here), but you will still have problems if you reference some other private registry in this way, so it would really be better to solve this in what I consider to be the correct way:  reference a secret you know can pull the image.

--- Additional comment from Ben Parees on 2018-11-12 15:04:40 EST ---

https://github.com/openshift/origin/pull/21470