Bug 1265526 - Regression: docker-1.8 rejects older versions of the API
Regression: docker-1.8 rejects older versions of the API
Status: CLOSED NOTABUG
Product: Fedora
Classification: Fedora
Component: docker (Show other bugs)
22
x86_64 Linux
unspecified Severity medium
: ---
: ---
Assigned To: Lokesh Mandvekar
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-23 03:08 EDT by tadas
Modified: 2015-09-28 14:29 EDT (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-28 14:29:13 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description tadas 2015-09-23 03:08:30 EDT
Description of problem:
I am not able to download any docker image after clean installing Fedora server 22 and running dnf update. Downloading docker image using command line on server works without any issues.

Version-Release number of selected component (if applicable):
Fedora release 22 (Twenty Two) - 4.1.6-201.fc22.x86_64
[root@docker01 ~]# docker version
Client:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Thu Sep 10 19:10:10 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Thu Sep 10 19:10:10 UTC 2015
 OS/Arch:      linux/amd64



How reproducible:
Easily

Steps to Reproduce:
1. Download Fedora server 22
2. Run dnf update
3. Try to download docker image from cockpit interface and it fails with error

Actual results:
Docker image fails to download

Expected results:


Additional info:
Fedora server is running on Oracle VirtualBox v5
Comment 1 Stef Walter 2015-09-23 03:22:33 EDT
Confirming. I see this in the Cockpit console:


pull failed: Object { status: 400, reason: "Bad Request", message: "client is too old, minimum supporte…", problem: null, valueOf: HttpError/this.valueOf(), toString: HttpError/this.toString() }


This is a docker regression in a released version of Fedora 22.

Dan, Lokesh, Mike ... why are we pushing docker-1.8 into a released stable released version of operating system like Fedora 22.

Regression affects versions of Cockpit prior to 0.70.

The version of Cockpit in Fedora 22 is 0.67. We stopped pushing new versions of Cockpit into Fedora 22 when Fedora 23 branched.
Comment 2 Stephen Gallagher 2015-09-23 07:42:25 EDT
From https://fedoraproject.org/wiki/Updates_Policy#All_other_updates:

"Package maintainers MUST: Avoid Major version updates, ABI breakage or API changes if at all possible."
Comment 5 Daniel Walsh 2015-09-23 08:32:46 EDT
We have been updating docker consistently since the start.  With its rapid development we here very quickly when we are out of date.   I am not sure what is the problem here.  It looks like there is a  problem with the server that is serving the content.  Are you seeing this with docker.io or a private registry.
Comment 6 Stef Walter 2015-09-23 10:31:27 EDT
> We have been updating docker consistently since the start. With its rapid
> development we here very quickly when we are out of date.   I am not sure what
> is the problem here. 

Yes, but the issue is where the updated docker packages are being pushed. If you push docker API changes into Fedora 23, or Rawhide ... that's great. But this is about a regression in Fedora 22 ... released 4 months ago.

>  It looks like there is a  problem with the server that is serving the
> content.  Are you seeing this with docker.io or a private registry.

No, this is because the docker daemon stopped answering requests to its /v1.10/  API. The version of Cockpit in Fedora 22 (ie: 0.67) still uses that API for pulling images. Cockpit 0.70 and later (including Cockpit in Fedora 23 and Rawhide) use later versions of the docker API and do not exhibit this issue.
Comment 7 Daniel Walsh 2015-09-23 10:42:53 EDT
Ah ok. Now I understand.  I guess we can have the version of docker reverted for Docker 1.8 and we will stop updating docker versions in Released Fedora.  Not sure why Fedora would break its backwards API. But we will end up with Fedora Released up to three or four versions behind the state of the art.

We will also end up with newer versions of docker in RHEL7 then the Released version of Fedora.
Comment 8 Stephen Gallagher 2015-09-23 10:50:33 EDT
Dan, to be clear: there would have been no problem with the upgrade if not for the fact that it discontinued a necessary API. This is the point at which released Fedora would need to stop rebasing to the latest upstream. There's *always* a Branched or Rawhide version ready to handle the new version.

(We should also be recommending that Docker upstream not discontinue APIs without a reasonable migration period, but that's a different issue.)
Comment 9 Matthew Miller 2015-09-24 16:16:52 EDT
How long does upstream docker consider their old releases supported?
Comment 10 Daniel Walsh 2015-09-24 16:20:46 EDT
No idea.
Comment 11 Matthew Miller 2015-09-24 16:22:30 EDT
Answering own question, from https://www.docker.com/compatibility-maintenance

Quarterly releases, with support for n and n-1. In other words, six month lifecycle.

Because it's not nice to have unsupported docker releases in the stable branch either, unless we're going to backport fixes.
Comment 12 Matthew Miller 2015-09-24 16:30:54 EDT
Stef, how comfortable are you with keeping Cockpit patched for changes (or updating Cockpit wholesale in stable Fedora releases)?

Assuming I'm understanding the situation right, I think the right thing to do here is to apply for an exception to the updates policy as outlined at https://fedoraproject.org/wiki/Updates_Policy#Exceptions, for the "Atomic stack" — kube, docker, cockpit, and etc.

This case isn't on the normal lists of reasons for exceptions, but hey, that's why it's called an "exception", right?
Comment 13 Stef Walter 2015-09-24 17:13:02 EDT
We've manually backported a patch cockpit-0.67-2 to work around this issue: 

https://bodhi.fedoraproject.org/updates/FEDORA-2015-5b605ec0cf

This cockpit-0.67-2 build needs to be included in the next Fedora 22 Atomic build as well.

> Stef, how comfortable are you with keeping Cockpit patched for changes (or updating Cockpit wholesale in stable Fedora releases)?

We're really heading towards continuous development with Cockpit anyway. And we're currently in the the situation where RHEL 7 Extras has a more recent Cockpit released than Fedora 22.

The issue arises when we need to depend on things that are not in Fedora 22, or new versions of other packages ... and I guess that's when we'd stop pushing releases of Cockpit into the released version of Fedora even if we do have an exception.

> Assuming I'm understanding the situation right, I think the right thing to do here is to apply for an exception to the updates policy as outlined at https://fedoraproject.org/wiki/Updates_Policy#Exceptions, for the "Atomic stack" — kube, docker, cockpit, and etc.

I think we'd be fine with that.

But this doesn't remove the pressure on these packages to get it right every time and not introduce bug regressions that affect users. It's a risky business. The *whole* reason Fedora users aren't using Fedora 23 ... is because it's not stable and not ready yet. We can't turn Fedora 22 into the same sorta situation.
Comment 14 Daniel Walsh 2015-09-28 14:29:13 EDT
This is the way docker designed it.  We can not fix this.

Note You need to log in before you can comment on or make changes to this bug.