Bug 1411161 - [3.3] Fail to push built image to registry due to "manifest blob unknown: blob unknown to registry" without define output image via oc new-build
Summary: [3.3] Fail to push built image to registry due to "manifest blob unknown: blo...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 3.3.1
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.3.1
Assignee: Alexey Gladkov
QA Contact: Wenjing Zheng
URL:
Whiteboard:
Depends On: 1408661 1411162
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-08 22:08 UTC by Scott Dodson
Modified: 2017-04-26 05:36 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Manifest verification didn't pay attention to non local layers. Consequence: Verification has failed if pullthrough enabled. Fix: Use pullthrough to verify remote layers. Result: Manifest verification is successful.
Clone Of: 1408661
Environment:
Last Closed: 2017-04-26 05:36:05 UTC
Target Upstream Version:


Attachments (Terms of Use)
This is docker registry log (137.06 KB, text/plain)
2017-02-14 07:42 UTC, Wenjing Zheng
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:1129 0 normal SHIPPED_LIVE OpenShift Container Platform 3.5, 3.4, 3.3, and 3.2 bug fix update 2017-04-26 09:35:35 UTC

Comment 1 Wenjing Zheng 2017-02-03 07:15:29 UTC
Can reproduce with 3.3.1 as below:
Registry server User Name: serviceaccount
Registry server Email: serviceaccount@example.org
Registry server Password: <<non-empty>>
error: build error: Failed to push image: errors:
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
manifest blob unknown: blob unknown to registry
[root@openshift-139 ~]# openshift version
openshift v3.3.1.11
kubernetes v1.3.0+52492b4
etcd 2.3.0+git

Comment 2 Alexey Gladkov 2017-02-08 12:57:17 UTC
what docker version are you using ?

Comment 3 Wenjing Zheng 2017-02-09 02:11:50 UTC
Here is my docker version:
# docker version
Client:
 Version:         1.12.5
 API version:     1.24
 Package version: docker-common-1.12.5-14.el7.x86_64
 Go version:      go1.7.4
 Git commit:      047e51b/1.12.5
 Built:           Wed Jan 11 17:53:20 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.5
 API version:     1.24
 Package version: docker-common-1.12.5-14.el7.x86_64
 Go version:      go1.7.4
 Git commit:      047e51b/1.12.5
 Built:           Wed Jan 11 17:53:20 2017
 OS/Arch:         linux/amd64

Comment 4 Alexey Gladkov 2017-02-09 10:47:57 UTC
So, this is not https://rhn.redhat.com/errata/RHSA-2017-0116.html

Can you attach docker-registry and docker daemon logs ?

Comment 5 Wenjing Zheng 2017-02-14 07:42:13 UTC
Created attachment 1250138 [details]
This is docker registry log

Comment 6 Wenjing Zheng 2017-02-14 08:03:26 UTC
No special log from docker , so just attach the docker registry log in attachment.

Comment 7 Alexey Gladkov 2017-02-14 15:56:54 UTC
Thanks! Now I see the problem.

Comment 14 Michal Fojtik 2017-03-29 10:22:36 UTC
(the PR merged)

Comment 15 Wenjing Zheng 2017-03-30 07:03:25 UTC
Verified with below version:
# openshift version
openshift v3.3.1.17
kubernetes v1.3.0+52492b4
etcd 2.3.0+git
# oc logs builds/centos-1
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Removing intermediate container daf304085600
Successfully built 61e34f5e8ba8

Pushing image 172.30.74.144:5000/wzheng1/centos:latest ...
Pushed 0/2 layers, 2% complete
Pushed 1/2 layers, 63% complete
Pushed 2/2 layers, 100% complete
Push successful
# oc get builds
NAME       TYPE      FROM         STATUS     STARTED         DURATION
centos-1   Docker    Dockerfile   Complete   6 minutes ago   1m42s

Comment 17 errata-xmlrpc 2017-04-26 05:36:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1129


Note You need to log in before you can comment on or make changes to this bug.