Bug 1380474

Summary:	Image pull fails with "Cannot overwrite digest sha256:hash"
Product:	[Fedora] Fedora	Reporter:	Seth Jennings <sjenning>
Component:	docker	Assignee:	Antonio Murdaca <amurdaca>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	24	CC:	adimania, admiller, agoldste, amurdaca, ccoleman, dwalsh, ichavero, jcajka, jchaloup, lsm5, marianne, me, miminar, nalin, riek, vbatts
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	docker-1.10.3-53.gite03ddb8.fc24	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1387435 (view as bug list)		Environment:
Last Closed:	2016-10-11 19:54:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1387435

Description Seth Jennings 2016-09-29 17:14:44 UTC

Description of problem:

Image pull w/ digest fails when already exists with error "Cannot overwrite digest sha256:<hash>"

Version-Release number of selected component (if applicable):

# docker version
Client:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-1.10.3-52.git8b7fa4a.fc24.x86_64
 Go version:      go1.6.3
 Git commit:      8b7fa4a/1.10.3
 Built:           
 OS/Arch:         linux/amd64

Server:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-1.10.3-52.git8b7fa4a.fc24.x86_64
 Go version:      go1.6.3
 Git commit:      8b7fa4a/1.10.3
 Built:           
 OS/Arch:         linux/amd64

How reproducible:

Always

Steps to Reproduce:

1. oc cluster up
2. oc new-app openshift/ruby-20-centos7~https://github.com/openshift/ruby-ex.git
3. built container pushes successfully but pull fails

Actual results:

# docker pull 172.30.127.214:5000/myproject/ruby-ex@sha256:a57ba614a14e61474849800060434a3dac2d363cd0725bc65a95336b0e90d983
Trying to pull repository 172.30.127.214:5000/myproject/ruby-ex ...
sha256:a57ba614a14e61474849800060434a3dac2d363cd0725bc65a95336b0e90d983: Pulling from 172.30.127.214:5000/myproject/ruby-ex
 
8d30e94188e7: Already exists
a3ed95caeb02: Already exists
2b72055030a2: Already exists
e20c3dd7e3a6: Already exists
006f4b559f92: Already exists
9604ab814ad2: Already exists
eeb1317c455f: Already exists
e43d85361c1f: Already exists
ccffafc28655: Already exists
b49da86489c0: Already exists
Digest: sha256:a57ba614a14e61474849800060434a3dac2d363cd0725bc65a95336b0e90d983
Cannot overwrite digest sha256:a57ba614a14e61474849800060434a3dac2d363cd0725bc65a95336b0e90d983

Expected results:

docker recognizes the image with the digest already exists and does nothing.

Additional info:

Old docker builds based on a612434 work fine.

Seems that one of these commits in the culprit.  I'm thinking 34771e8.

$ git log --no-merges --oneline a612434.. | grep -i digest
6c0bb14 Do not show empty tags for digest references in output
a216b04 Treat digest only images as dangling
34771e8 Always store the image digest when pulling and pushing an image.

Full openshift debug session:
https://paste.fedoraproject.org/438222/51651371/

Comment 3 Gerard Braad 2016-09-30 08:13:58 UTC

thanks for reporting this on Bugzilla. I thought I made a mistake during deployment (as image pushed), but noticed it succeeds on CentOS 7 with the same configuration. It is consistently reproducible:

Output of tests:
  https://gist.github.com/gbraad/e82edffb671a5dd154a939491514f7f8/

  Fedora 24 fails, also on Atomic
  CentOS 7 succeeds

Initial report at:
  http://lists.openshift.redhat.com/openshift-archives/users/2016-September/msg00198.html

Comment 7 Antonio Murdaca 2016-10-10 11:31:28 UTC

Fixed by https://github.com/projectatomic/docker/commit/e03ddb80cc2e3244b70052ec38d29cf526444c59

I'm rebuilding docker (+ update) for F24 shortly.

Comment 8 Antonio Murdaca 2016-10-10 13:20:42 UTC

update here https://bodhi.fedoraproject.org/updates/FEDORA-2016-dfc4d2e82a

Comment 9 Fedora Update System 2016-10-11 03:22:23 UTC

docker-1.10.3-53.gite03ddb8.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-dfc4d2e82a

Comment 10 Fedora Update System 2016-10-11 19:54:40 UTC

docker-1.10.3-53.gite03ddb8.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.