Bug 1434897 - docker tries to push to registry.redhat.com
Summary: docker tries to push to registry.redhat.com
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: docker
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Antonio Murdaca
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-22 14:45 UTC by Dusty Mabe
Modified: 2018-08-27 21:26 UTC (History)
15 users (show)

Fixed In Version: docker-1.13.1-6.git5be1549.fc26
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-09 05:58:06 UTC
Type: Bug


Attachments (Terms of Use)
dockersearch.txt (24.54 KB, text/plain)
2017-03-28 20:33 UTC, Dusty Mabe
no flags Details

Description Dusty Mabe 2017-03-22 14:45:51 UTC
Description of problem:

when I docker push on Fedora it is trying to push to registry.redhat.com. This is not the behavior in fedora 25. I assume it is a mistake and the configuration needs to be taken out.

```
[root@vanilla-f26atomic ~]# docker push dustymabe/fedora-min 
The push refers to a repository [registry.access.redhat.com/dustymabe/fedora-min]
6d6fcb84c31f: Preparing 
error parsing HTTP 405 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 3.2 Final//EN\">\n<title>405 Method Not Allowed</title>\n<h1>Method Not Allowed</h1>\n<p>The method POST is not allowed for the requested URL.</p>\n"
```

Version-Release number of selected component (if applicable):
docker-1.13.1-3.git5be1549.fc26.x86_64

Comment 1 Antonio Murdaca 2017-03-22 14:54:11 UTC
This has changed in https://bugzilla.redhat.com/show_bug.cgi?id=1384202

Even if the related bug says "for searching purposes" afaict, there's no way to tell docker to just use the RH registry to search for images. Hence we added that.
I'm not sure what to do here honestly. My preference would be to drop this from Fedora.

Both Dan(s), what do you think?

Comment 2 Daniel Walsh 2017-03-22 16:01:22 UTC
Well this would be an issue with RHEL also.  If we setup the tools to 

--block-registry=all --add-registry registry.access.redhat.com --add-registry=docker.io

Does this fix the issue, where I can continue to pull from redhat or docker and default to pushing to docker.io?

Comment 3 Dusty Mabe 2017-03-22 16:27:30 UTC
even if we are able to achieve that behavior do we really want to default to pulling from the red hat registry when the openness of that registry could change?

Comment 4 Daniel Walsh 2017-03-22 17:02:59 UTC
You can get credentials for RHEL on a Fedora host, so you could get a developer license and use that content.  Daniel Riek requested that change for that reason.

Bottom line is we have the same issue on RHEL systems.  Dusty could you make the change to the /etc/sysconfig/docker and see if it solves your issue.

Comment 5 Antonio Murdaca 2017-03-22 19:29:07 UTC
(In reply to Daniel Walsh from comment #2)
> Well this would be an issue with RHEL also.  If we setup the tools to 
> 
> --block-registry=all --add-registry registry.access.redhat.com
> --add-registry=docker.io
> 
> Does this fix the issue, where I can continue to pull from redhat or docker
> and default to pushing to docker.io?

Dan, I wouldn't block all registries. That means the user could just push/pull/search on RH registry and docker.io - which seems weird in Fedora at least.

I'll check how this impacts RHEL though (hadn't a chance till now).

To summarize:

- the original BZ wanted the ability to __docker search__ the RH registry for images (nothing to do with push/pull)
- the original BZ resolution added the rhel registry as the default one, making pushing images just go to the RH registry (which is plain wrong to me, both in RHEL and Fedora)
- so, let's check if pushing an unqualified image in RHEL with the default add-registry causes the push to go to the RH registry (which will fail as it's read-only)
- to workaround the point above, one could push with a qualified image (e.g. docker push docker.io/myawesomeimage
- research if it could be possible to enhance the registry patch we carry to search on additional registries w/o impacting push/pull operations

Comment 6 Dusty Mabe 2017-03-22 19:38:58 UTC
(In reply to Daniel Walsh from comment #2)
> Well this would be an issue with RHEL also.  If we setup the tools to 
> 
> --block-registry=all --add-registry registry.access.redhat.com
> --add-registry=docker.io
> 
> Does this fix the issue, where I can continue to pull from redhat or docker
> and default to pushing to docker.io?

No, even with those options I still default to pointing to registry.access.redhat.com.

Comment 7 Colin Walters 2017-03-23 13:06:52 UTC
I'll still say I think that all we should do is add a flag to disable the "foo" → "docker.io/foo" auto-mapping.  With that flag on, *everyone* is on equal terms.

Yes, indeed this means people will need to type:
docker pull registry.access.redhat.com/rhel7, but the solution to that is to shorten the URL IMO (redhat.io/rhel7) ?

Searching registry.access.redhat.com but *not* registry.fedoraproject.org seems weird to me for example.

Comment 8 Colin Walters 2017-03-23 13:12:45 UTC
Also, with this flag it's then `docker search -r/--registry docker.io term1 term2` ?

Comment 10 Antonio Murdaca 2017-03-23 13:20:27 UTC
The only downside to a flag which removes the auto-mapping is that we'll break *lots* of people/customers who have "FROM fedora" in their Dockerfiles.

Comment 11 Colin Walters 2017-03-23 13:33:35 UTC
Not just Dockerfiles of course, but Kubernetes pod definitions etc etc.

But we only have to pull the bandaid off once.  And don't we want people saying `FROM registry.fedoraproject.org/fedora` Or alternatively, `docker.io/fedora`.

Note I'm not arguing for having this flag on by default.  What I am arguing is we should:

 - Implement it, including the `docker search --registry <registry>`
 - Encourage people to use it, and gradually fix up our own Dockerfiles/"Kubefiles"
 - Don't add hacks that muddy the waters like injecting "our own" registries - that gets into messy (and political) topics like search priorities.  Even among our 3 distributions!  With the "no auto-docker.io" flag, everyone is equal.

Comment 13 Daniel Walsh 2017-03-23 13:56:53 UTC
Kubernetes always pulls with the fully qualified name,  This only applies to the docker CLI.

Comment 14 Dusty Mabe 2017-03-23 14:09:28 UTC
Are there not plenty of kube yaml files that people use that don't use the fully qualified name? My experience has been that specifying just 'fedora' would work when defining a pod in kube land. Maybe my memory serves me wrong.

Comment 15 Daniel Walsh 2017-03-23 14:11:53 UTC
Well that would be a problem as we move to alternative CRI like CRI-O and RKT which certainly do not have a hard coded docker.io in them.

Comment 16 Dusty Mabe 2017-03-28 16:04:43 UTC
If we can't find a solution to this problem soon, can we remove the push plugin from Fedora until a proper solution is found?

Comment 17 Antonio Murdaca 2017-03-28 17:48:58 UTC
(In reply to Dusty Mabe from comment #16)
> If we can't find a solution to this problem soon, can we remove the push
> plugin from Fedora until a proper solution is found?

we need to remove the "--add-registry" stuff from /etc/sysconfig/docker - this issue seems unrelated to the docker push plugin.

Comment 18 Dusty Mabe 2017-03-28 17:50:01 UTC
(In reply to Antonio Murdaca from comment #17)
> we need to remove the "--add-registry" stuff from /etc/sysconfig/docker -
> this issue seems unrelated to the docker push plugin.

oops, sorry. Thought it was related.

Comment 19 Dusty Mabe 2017-03-28 20:06:28 UTC
(In reply to Daniel Walsh from comment #2)
> Well this would be an issue with RHEL also.  If we setup the tools to 
> 
> --block-registry=all --add-registry registry.access.redhat.com
> --add-registry=docker.io
> 
> Does this fix the issue, where I can continue to pull from redhat or docker
> and default to pushing to docker.io?

This works if I explicitly add docker.io as the first --add-registry by having the following in my /etc/sysconfig/docker:

ADD_REGISTRY='--add-registry docker.io --add-registry registry.access.redhat.com'

I did not apply --block-registry=all. I was also able to pull from the different registries, but it will try them in order.

Comment 20 Antonio Murdaca 2017-03-28 20:21:12 UTC
(In reply to Dusty Mabe from comment #19)
> (In reply to Daniel Walsh from comment #2)
> > Well this would be an issue with RHEL also.  If we setup the tools to 
> > 
> > --block-registry=all --add-registry registry.access.redhat.com
> > --add-registry=docker.io
> > 
> > Does this fix the issue, where I can continue to pull from redhat or docker
> > and default to pushing to docker.io?
> 
> This works if I explicitly add docker.io as the first --add-registry by
> having the following in my /etc/sysconfig/docker:
> 
> ADD_REGISTRY='--add-registry docker.io --add-registry
> registry.access.redhat.com'
>

The question is - is this configuration searching in our RH registry when someone does "docker search rhel"? or it goes just to docker.io (which was why the RH registry was added first)

Comment 21 Dusty Mabe 2017-03-28 20:33:51 UTC
Created attachment 1267131 [details]
dockersearch.txt

It searches both registries, but the (potentially disturbing) docker.io results are first.

Comment 22 Daniel Walsh 2017-03-28 20:48:12 UTC
This is ok for now, and I think we should add a new field, --default-pushregistry or something like that so that you can specify multiple read registries in order and one default push reg.

Comment 23 Fedora Update System 2017-03-31 14:55:50 UTC
docker-1.13.1-6.git5be1549.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-80cdb3361d

Comment 24 Fedora Update System 2017-04-02 00:51:17 UTC
docker-1.13.1-6.git5be1549.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-80cdb3361d

Comment 25 Fedora Update System 2017-04-09 05:58:06 UTC
docker-1.13.1-6.git5be1549.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 26 Dennis Schafroth 2017-07-29 11:08:11 UTC
I am stil seeing this on F26 with package docker-1.13.1-19.git27e468e.fc26.x86_64

I have some images on docker.io but after upgrading to F26 docker push is using registry.fedoraproject.org

I guess the work-around (getting same behaviour as in F25) for this, is to add docker.io as first entry in /etc/containers/registries.conf ?

Comment 27 LiZhenbo 2017-08-19 17:50:08 UTC
(In reply to Dennis Schafroth from comment #26)
> I am stil seeing this on F26 with package
> docker-1.13.1-19.git27e468e.fc26.x86_64
> 
> I have some images on docker.io but after upgrading to F26 docker push is
> using registry.fedoraproject.org
> 
> I guess the work-around (getting same behaviour as in F25) for this, is to
> add docker.io as first entry in /etc/containers/registries.conf ?

Removing all the entries for redhat/fedora in /etc/containers/registries.conf is a workaround in F26

Comment 28 Daniel Walsh 2017-08-20 11:07:41 UTC
You can also change the order in registries.conf.

Comment 29 Colin Walters 2018-08-27 21:26:47 UTC
For people who agree with me, all you need to do is change /etc/containers/registries.conf to read:

[registries.search]
registries = []

Then fully qualified image names will always be required and you'll never be confused about where images come from.


Note You need to log in before you can comment on or make changes to this bug.