Hide Forgot
+++ This bug was initially created as a clone of Bug #1648775 +++ Description of problem: You should be able to reproduce the issue by creating a namespace called base and put a docker Image in the namespace called web:latest now you create another namespace let’s call it test and allow the service accounts in test namespace to pull images from the base namespace. In the test namespace you start a binary build that has a dockerfile with FROM docker-registry.default.svc:5000/base/web The partial build success results from the fact that the requested image is locally available in the docker pool. A workaround is to add the pull secret of the builder service account to the dockerStrategy like this: You have to make sure that the image is not already pulled on the node where your build is running. A workaround is to add the pull secret of the builder service account to the dockerStrategy like this: strategy: dockerStrategy: pullSecret: name: builder-dockercfg-1775b another workaround is to reference the ImageStreamTag (this will override the first FROM in the Dockerfile) like this: strategy: dockerStrategy: from: kind: ImageStreamTag name: 'mip-web:latest' namespace: base The workarounds are not feasible as the user need to change all the pipelines to incorporate the workaround. Anyway the normal k8s behavior is that pullSecret is mounted in the pod (https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller) ---- 3. If the pod does not contain any ImagePullSecrets, then ImagePullSecrets of the ServiceAccount should be added to the pod. ---- 1] Nov 06 11:05:46 master.example.com atomic-openshift-master-api[50426]: I1106 11:05:46.400209 50426 rbac.go:116] RBAC DENY: user "system:anonymous" groups ["system:unauthenticated"] cannot "get" resource "imagestreams/layers.image.openshift.io" named "mip-web" in namespace "base" Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: oc new-project base oc new-app centos/ruby-25-centos7~https://github.com/sclorg/ruby-ex.git -n base cat <<EOF | oc create -n base -f - apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: image-pull-access-fptest namespace: base roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: 'system:image-puller' subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: 'system:serviceaccounts:fptest' EOF oc new-project fptest oc create is myimage -n fptest cat <<EOF | oc create -n fptest -f - apiVersion: build.openshift.io/v1 kind: BuildConfig metadata: name: binary-build namespace: fptest spec: failedBuildsHistoryLimit: 5 output: to: kind: ImageStreamTag name: 'myimage:latest' postCommit: {} resources: {} runPolicy: Serial source: binary: {} type: Binary strategy: dockerStrategy: {} type: Docker successfulBuildsHistoryLimit: 5 triggers: [] EOF echo "FROM docker-registry.default.svc:5000/base/ruby-25-centos7" > Dockerfile oc start-build binary-build --from-file=Dockerfile -n fptest Actual results: Binary builds doesnt automatically include the pull secret from the service account during the build. Expected results: We expect the build pod has mounted the pullSecret of its SA, even when it is not explicitly configured (https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/#service-account-admission-controller point 3): "If the pod does not contain any ImagePullSecrets, then ImagePullSecrets of the ServiceAccount are added to the pod." Additional info: --- Additional comment from Ben Parees on 2018-11-12 09:21:11 EST --- Does this occur w/ a non-binary build? --- Additional comment from Ben Parees on 2018-11-12 15:04:01 EST --- The k8s documented behavior has no bearing on this because builds must do their own determination of what secret to use because builds can reference images in different ways than pods do, and the secrets that k8s mounts into the build pod are not the secrets the build will use. I do not think this is specific to binary builds. Any build where you have not explicitly told us in the buildconfig, what image you will be referencing, will have this issue. The build must make the decision, prior to starting the build pod, what secrets to mount into the build pod so that docker can pull the images. Since we don't know what your dockerfile looks like until after the build pod starts (at which point the dockerfile is either streamed in (binary build) or git cloned(normal build)), we can't mount it in advance. Yes, it is technically possible that we can fallback to using the builder service account's docker secret in this scenario, but that will only solve the problem for builds that reference the internal registry. You would still have this problem if you referenced some other private registry. As you've noted, the solutions are either: 1) explicitly indicate the image you want to reference via the buildconfig's From field 2) explicitly indicate the secret you want to use to pull the image via the buildconfig's pullsecret field. I will work an update that causes us to fallback to the service account's docker secret(if available) when we have no other information about what image is being pulled (as is the case here), but you will still have problems if you reference some other private registry in this way, so it would really be better to solve this in what I consider to be the correct way: reference a secret you know can pull the image. --- Additional comment from Ben Parees on 2018-11-12 15:04:40 EST --- https://github.com/openshift/origin/pull/21470
https://github.com/openshift/origin/pull/21472
this has been in modified for 2 weeks. whatever script is supposed to be moving things to on_qa is not working.
Verified in openshift: v4.0.0-0.80.0
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758