1430266 – Docker pull results in 'manifest unknown'

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1430266 - Docker pull results in 'manifest unknown'

Summary: Docker pull results in 'manifest unknown'

Keywords:
Status:	CLOSED WONTFIX
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	docker
Sub Component:
Version:	7.3
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Antonio Murdaca
QA Contact:	atomic-bugs@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1420851
TreeView+	depends on / blocked

Reported:	2017-03-08 09:05 UTC by Jaroslav Spanko
Modified:	2020-12-14 08:18 UTC (History)
CC List:	5 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2017-06-30 15:09:08 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Description Jaroslav Spanko 2017-03-08 09:05:53 UTC

Description of problem:
The image was built with docker-1.10.3 client, pull works:

$ sudo docker pull test.registry/devel/image
Trying to pull repository test.registry/devel/image ... 
mark-test: Pulling from test.registry/devel/image
af20bebc5fc1: Pull complete 
Digest: sha256:cd9bab8b80803eb4205f347aa63a15141ad61e432b2768579e9b1bdd80873985
Status: Downloaded newer image for test.registry/devel/image-test

pull with docker-1.12.5 client fails on manifest unknown:
# docker pull test.registry/devel/image-test
Trying to pull repository test.registry/devel/image-test
manifest unknown: manifest unknown

Also docker 1.12 and docker 1.13 non-redhat provided do not have this issue

Info about the image
image
{
    "name": "devel/image-test",
    "schemaVersion": 1,
    "signatures": [
        {
            "header": {
                "alg": "ES256",
                "jwk": {
                    "crv": "P-256",
                    "kid": "x",
                    "kty": "EC",
                    "x": "x",
                    "y": "y"
                }
            },
            "protected": "p",
            "signature": "s"
        }
    ],
    "tag": "mark-test"
}

After upgrade of registry to docker-distribution-2.5.1-1.el7.x86_64 pull works without problem on docker-1.12.5.

Version-Release number of selected component (if applicable):
Host: RHEL 7.3
Docker: docker-1.12.5-59
Registry:  docker-distribution-2.4.1-1

How reproducible:

Steps to Reproduce:
1. Build image with older version of docker
2. Try to pull with newer version of docker 
3. Fail with error manifest unknown: manifest unknown

Actual results:
Message manifest unknown: manifest unknown, after upgrade of registry to newer version pull works

Expected results:


Additional info:

Comment 2 Antonio Murdaca 2017-03-20 09:33:28 UTC

(In reply to Jaroslav Spanko from comment #0)
> Description of problem:
> The image was built with docker-1.10.3 client, pull works:
> 
> $ sudo docker pull test.registry/devel/image
> Trying to pull repository test.registry/devel/image ... 
> mark-test: Pulling from test.registry/devel/image
> af20bebc5fc1: Pull complete 
> Digest:
> sha256:cd9bab8b80803eb4205f347aa63a15141ad61e432b2768579e9b1bdd80873985
> Status: Downloaded newer image for test.registry/devel/image-test
> 
> pull with docker-1.12.5 client fails on manifest unknown:
> # docker pull test.registry/devel/image-test
> Trying to pull repository test.registry/devel/image-test
> manifest unknown: manifest unknown
> 
> Also docker 1.12 and docker 1.13 non-redhat provided do not have this issue
> 
> Info about the image
> image
> {
>     "name": "devel/image-test",
>     "schemaVersion": 1,
>     "signatures": [
>         {
>             "header": {
>                 "alg": "ES256",
>                 "jwk": {
>                     "crv": "P-256",
>                     "kid": "x",
>                     "kty": "EC",
>                     "x": "x",
>                     "y": "y"
>                 }
>             },
>             "protected": "p",
>             "signature": "s"
>         }
>     ],
>     "tag": "mark-test"
> }
> 
> After upgrade of registry to docker-distribution-2.5.1-1.el7.x86_64 pull
> works without problem on docker-1.12.5.
> 
> Version-Release number of selected component (if applicable):
> Host: RHEL 7.3
> Docker: docker-1.12.5-59

which docker version is this? is this coming from extras-rhel-7.3? I cannot locate a "-59" build.

Most likely, the old registry is a V1 registry and you have "--signature-verification=true" in docker daemon options. That prevents you from pulling the image with your 1.12.5 docker. Could you try setting:

--signature-verification=false

in /etc/sysconfig/docker, restart and re-test?

Comment 3 Jaroslav Spanko 2017-03-21 09:16:15 UTC

I will test it, why this fixed upgrade of registry to  docker-distribution-2.5.1-1.el7.x86_64 ? 
Because of V2 registry in 2.5.1 ? 
Thanks

Comment 4 Antonio Murdaca 2017-03-21 09:41:56 UTC

(In reply to Jaroslav Spanko from comment #3)
> I will test it, why this fixed upgrade of registry to 
> docker-distribution-2.5.1-1.el7.x86_64 ? 
> Because of V2 registry in 2.5.1 ? 
> Thanks

my suspicion is the old registry was a v1 registry and that prevented your reproducer from working.

Comment 5 Subodh Pachghare 2017-05-27 11:41:11 UTC

I have also faced this same error, of the image is not present or image metadata is not present in the Registry, for example, If I run a Docker Registry in a Container and I stop the created Container (not started), then I get this error. If I push the image again and pull, this error is gone.

Thanks,
Subodh Pachghare
thesubodh.com

Note You need to log in before you can comment on or make changes to this bug.