Bug 1476038

Summary: Can't deploy images from private registries
Product: OpenShift Container Platform Reporter: Mike Barrett <mbarrett>
Component: Image RegistryAssignee: Oleg Bulatov <obulatov>
Status: CLOSED NOTABUG QA Contact: ge liu <geliu>
Severity: low Docs Contact:
Priority: low    
Version: 3.6.0CC: aos-bugs, aweiteka, bparees, dyan, jokerman, maszulik, mbarrett, mfojtik, mmccomas, obulatov, pweil
Target Milestone: ---   
Target Release: 3.6.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-10-03 20:54:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mike Barrett 2017-07-27 22:16:50 UTC
Description of problem:

Trying to follow this process:

https://docs.openshift.com/container-platform/3.5/dev_guide/managing_images.html#private-registries

For using images that are from the Docker Store.

Version-Release number of selected component (if applicable):

OCP 3.6


How reproducible:

Always


Steps to Reproduce:

From my laptop I can pull the Oracle database from the store.docker.com

$ docker pull store/oracle/database-enterprise:12.1.0.2
12.1.0.2: Pulling from store/oracle/database-enterprise
e1ef1eb9fcad: Pull complete 
7b36741b57dd: Pull complete 
183fb4717901: Pull complete 
19e8aae86959: Pull complete 
5264789218d2: Pull complete 
Digest: sha256:4a6ce8480d5e0aa0dd798564a5340fbfc45033a26468745482457d92008195a1
Status: Downloaded newer image for store/oracle/database-enterprise:12.1.0.2
$ 

So I tried to get that to work from OpenShift Pro tier and I can't seem to figure out the correct commands.

$ oc secrets new-dockercfg docker-store --docker-server=docker.io --docker-username=<username> --docker-password=<password> --docker-email="<email>"

$ oc secret link default secret/docker-store --for=pull

$ oc tag --reference-policy=local --source=docker docker.io/store/oracle/database-enterprise:12.1.0.2 database-enterprise:12.1.0.2

Now I can see the imagestream great from the webconsole.  I can drill into it and see the layers and dockerfile.  So the tag worked.  I can select it from the webconsole and deploy it...but I get these errors over and over again when I deploy it:

Failed to pull image "172.30.118.67:5000/database/database-enterprise@sha256:4a6ce8480d5e0aa0dd798564a5340fbfc45033a26468745482457d92008195a1": image pull failed for 172.30.118.67:5000/database/database-enterprise@sha256:4a6ce8480d5e0aa0dd798564a5340fbfc45033a26468745482457d92008195a1, this may be because there are no credentials on this request. details: (unauthorized: authentication required)
Back-off pulling image "172.30.118.67:5000/database/database-enterprise@sha256:4a6ce8480d5e0aa0dd798564a5340fbfc45033a26468745482457d92008195a1"

So from a Linux back I logged into docker and captured the .docker/config.json file.  I then add username and email to that file (because all it has in it from docker 1.12 and higher is the auth line).

{
        "auths": {
                "https://index.docker.io/v1/": {
                "username": "<username>",
                "auth": "<blahblah>",
                "email": "<email>"
                }
        }
}

I then tried the same thing above but with the .dockerconfigjson secret:

$ oc secret new store2 .dockerconfigjson=/.docker/test.json
$ oc secret link default secret/store2 --for=pull


Failed with same auth error.


Expected results:

Expect to be able to pull and deploy images from private repos.

Additional info:

Comment 5 Paul Weil 2017-07-28 18:10:15 UTC
*** Bug 1476330 has been marked as a duplicate of this bug. ***

Comment 17 Maciej Szulik 2017-08-10 11:32:45 UTC
Here's the series of steps needed to do what Mike wants:

1. Import image: 
oc import-image hello --from=<private_pull_spec> --confirm

This will fail because repo is private.

2. Create import secret:
oc secrets new-dockercfg hello --docker-server=<server_address> --docker-username=<username> --docker-password=<password> --docker-email=<email>

3. Re-import image:
oc import-image hello 

4. Link secret to the default SA running our app:
oc secrets link default hello --for=pull

5. Run the image:
oc new-app hello

Does that help Mike?

Comment 23 Ben Parees 2017-10-03 01:43:18 UTC
To summarize this bug as i understand it:

When referencing an external, private docker registry from an imagestreamtag, and *not* using the local reference policy, users must:

1) provide a secret to perform the import

2) provide that same secret, or another one, to the deployment service account so it can pull the image at deployment time (same would be true for a job, builder, etc).

The concern is that the user must perform two steps (in reality step 1 might be performed by an admin and step 2 would have to be performed by a user).  *Because* the two steps might be done by two different users, it is necessary to supply the secret separately.  There is no guarantee the user in (2) should be able to see the secret used in (1).

Therefore:

1) it's not a bug, at best it's an RFE
2) i don't think we can or should fix it any more so than we'd try to share secrets between builds and deployments.  (A build could push an image to a registry that a deployment can't pull from, because the build has different secret configured than the deployment does).

I am lowering the severity to remove this from the blocker list but my disposition is to close it entirely unless my understanding of the problem is incorrect.

Comment 24 Oleg Bulatov 2017-10-03 12:46:06 UTC
I don't understand, which two steps?

1. oc secrets new-dockercfg dockerhub --docker-server=index.docker.io --docker-
username=dmage --docker-password=REDACTED --docker-email=unused
2. oc new-app store/couchbase/couchbase:3.1.5

It works.

Comment 25 Ben Parees 2017-10-03 20:54:42 UTC
Closing as not a bug.  As Maciej noted earlier in this thread, the way to do this is to follow:

https://docs.openshift.com/container-platform/3.5/dev_guide/managing_images.html#private-registries

to import the imagestream

and 
https://docs.openshift.com/container-platform/3.5/dev_guide/managing_images.html#allowing-pods-to-reference-images-from-other-secured-registries

to deploy.

The web console experience may or may not need improvement, but there's nothing fundamentally broken about deploying images from private registries.

Effectively the steps are:

1) create a docker secret that can pull the image (this will allow image imports to succeed)

2) link that secret to your default service account (this will allow pods to pull the image)

3) create the imagestream and allow it to be imported (it will use the secret from (1))
4) create the deployment config referencing the imagestream (the pod that gets created will pull the image using the secret from (1) as long as it is run with the service account from (2).

Comment 26 Red Hat Bugzilla 2023-09-14 04:01:47 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days