Bug 1387217

Summary: Unsupported manifest type, need a Docker schema 2 manifest
Product: Red Hat Enterprise Linux 7 Reporter: Alex Jia <ajia>
Component: skopeoAssignee: Lokesh Mandvekar <lsm5>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: high Docs Contact:
Priority: high    
Version: 7.3Keywords: Extras
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-05 10:37:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1346930    

Description Alex Jia 2016-10-20 11:23:59 UTC
Description of problem:
It seems we use the format of schema v1 image manifest in access.redhat.com registry, but atomic {pull, install, update} needs schema v2 image manifest,
the result is above atomic command doesn't work.

Version-Release number of selected component (if applicable):

[root@atomic-host-test cloud-user]# cat /etc/redhat-release 
Red Hat Enterprise Linux Atomic Host release 7.3
[root@atomic-host-test cloud-user]# atomic host status
State: idle
Deployments:
● rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard
       Version: 7.3 (2016-10-20 03:38:44)
    BaseCommit: bd5ac48f6195637c0230d9b0ab0a2e5fb843764f85bc64757106238bdf31e757
        Commit: 09816ce768bac0b3e5c09e521a4610c63be7a12f1334948aa6173c432f628dea
        OSName: rhel-atomic-host
      Packages: ltrace strace time
      Unlocked: hotfix

[root@atomic-host-test cloud-user]# rpm -q atomic skopeo docker
atomic-1.13.1-3.el7.x86_64
skopeo-0.1.17-0.4.git550a480.el7.x86_64
docker-1.10.3-56.el7.x86_64


How reproducible:
always

Steps to Reproduce:
1. atomic --debug pull registry.access.redhat.com/rhel7
or
2. atomic --debug install registry.access.redhat.com/rhel7
or
3. atomic --debug update registry.access.redhat.com/rhel7

Actual results:
[root@atomic-host-test atomic]# atomic --debug pull registry.access.redhat.com/rhel7
Image registry.access.redhat.com/rhel7 is being pulled to docker ...
Pulling registry.access.redhat.com/rhel7:latest ...
Executing: /usr/bin/skopeo --debug copy --remove-signatures docker://registry.access.redhat.com/rhel7:latest docker-daemon:registry.access.redhat.com/rhel7:latest
DEBU[0000] Using registries.d directory /etc/containers/registries.d for sigstore configuration 
DEBU[0000]  Using "default-docker" configuration        
DEBU[0000]  No signature storage configuration found for registry.access.redhat.com/rhel7:latest 
DEBU[0000] IsRunningImageAllowed for image docker:registry.access.redhat.com/rhel7:latest 
DEBU[0000]  Using default policy section                
DEBU[0000]  Requirement 0: allowed                      
DEBU[0000] Overall: allowed                             
DEBU[0000] GET https://registry.access.redhat.com/v2/   
DEBU[0001] Ping https://registry.access.redhat.com/v2/ err <nil> 
DEBU[0001] Ping https://registry.access.redhat.com/v2/ status 200 
DEBU[0001] GET https://registry.access.redhat.com/v2/rhel7/manifests/latest 
Uploading blob sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9
DEBU[0004] Downloading rhel7/blobs/sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9 
DEBU[0004] GET https://registry.access.redhat.com/v2/rhel7/blobs/sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9 
DEBU[0006] Detected compression format gzip             
 0 B / ? [--------------------------------------------------------------------=]DEBU[0006] Using original blob without modification     
DEBU[0006] docker-daemon: input with unknown size, streaming to disk first… 
 67.88 MB / ? [---------------------------------------------------------------=]DEBU[0032] … streaming done                             
DEBU[0032] Sending as tar file sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9 
 67.95 MB / ? [------------------------------------------=--------------------] 
Uploading blob sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed
DEBU[0032] Downloading rhel7/blobs/sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed 
DEBU[0032] GET https://registry.access.redhat.com/v2/rhel7/blobs/sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed 
DEBU[0034] Detected compression format gzip             
 0 B / ? [--------------------------------------------------------------------=]DEBU[0034] Using original blob without modification     
DEBU[0034] Sending as tar file sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed 

Uploading manifest to image destination
DEBU[0034] docker-daemon: Closing tar stream to abort loading 
FATA[0034] Error writing manifest: Unsupported manifest type, need a Docker schema 2 manifest 

Traceback (most recent call last):
  File "/bin/atomic", line 186, in <module>
    sys.exit(_func())
  File "/usr/lib/python2.7/site-packages/Atomic/pull.py", line 65, in pull_image
    handler()
  File "/usr/lib/python2.7/site-packages/Atomic/pull.py", line 53, in pull_docker_image
    policy_filename=self.policy_filename)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 404, in skopeo_copy
    return check_call(cmd)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 169, in check_call
    return subprocess.check_call(cmd, env=env, stdin=stdin, stderr=stderr, stdout=stdout, close_fds=True)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/skopeo', '--debug', 'copy', '--remove-signatures', 'docker://registry.access.redhat.com/rhel7:latest', 'docker-daemon:registry.access.redhat.com/rhel7:latest']' returned non-zero exit status 1

[root@atomic-host-test atomic]# atomic --debug install registry.access.redhat.com/rhel7
Uploading blob sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9
 67.95 MB / ? [------------------------------------------=--------------------] 
Uploading blob sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed
 0 B / ? [--------------------------------------------------------------------=]
Uploading manifest to image destination
FATA[0019] Error writing manifest: Unsupported manifest type, need a Docker schema 2 manifest 

Traceback (most recent call last):
  File "/bin/atomic", line 186, in <module>
    sys.exit(_func())
  File "/usr/lib/python2.7/site-packages/Atomic/install.py", line 90, in install
    self._check_if_image_present()
  File "/usr/lib/python2.7/site-packages/Atomic/install.py", line 113, in _check_if_image_present
    self.update()
  File "/usr/lib/python2.7/site-packages/Atomic/atomic.py", line 117, in update
    util.is_insecure_registry(self.d.info()['RegistryConfig'], util.strip_port(registry)))
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 404, in skopeo_copy
    return check_call(cmd)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 169, in check_call
    return subprocess.check_call(cmd, env=env, stdin=stdin, stderr=stderr, stdout=stdout, close_fds=True)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/skopeo', 'copy', '--remove-signatures', 'docker://registry.access.redhat.com/rhel7', 'docker-daemon:registry.access.redhat.com/rhel7:latest']' returned non-zero exit status 1

[root@atomic-host-test RPMs]# atomic update registry.access.redhat.com/rhel7
Uploading blob sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9
 67.95 MB / ? [------------------------------------------=--------------------] 
Uploading blob sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed
 0 B / ? [--------------------------------------------------------------------=]
Uploading manifest to image destination
FATA[0019] Error writing manifest: Unsupported manifest type, need a Docker schema 2 manifest 

[root@atomic-host-test atomic]# atomic --debug update registry.access.redhat.com/rhel7
Uploading blob sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9
 67.95 MB / ? [------------------------------------------=--------------------] 
Uploading blob sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed
 0 B / ? [--------------------------------------------------------------------=]
Uploading manifest to image destination
FATA[0018] Error writing manifest: Unsupported manifest type, need a Docker schema 2 manifest 

Traceback (most recent call last):
  File "/bin/atomic", line 186, in <module>
    sys.exit(_func())
  File "/usr/lib/python2.7/site-packages/Atomic/update.py", line 35, in update
    super(Update, self).update()
  File "/usr/lib/python2.7/site-packages/Atomic/atomic.py", line 117, in update
    util.is_insecure_registry(self.d.info()['RegistryConfig'], util.strip_port(registry)))
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 404, in skopeo_copy
    return check_call(cmd)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 169, in check_call
    return subprocess.check_call(cmd, env=env, stdin=stdin, stderr=stderr, stdout=stdout, close_fds=True)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/skopeo', 'copy', '--remove-signatures', 'docker://registry.access.redhat.com/rhel7', 'docker-daemon:registry.access.redhat.com/rhel7:latest']' returned non-zero exit status 1


Expected results:
I expect this issue is fixed on 7.3, because some basic atomic command doesn't work.

Additional info:

It works on upstream atomic.

[root@atomic-host-test atomic]# git rev-parse HEAD
61b03d3c94023e4913b05347d2b67cc2ab18c063

[root@atomic-host-test atomic]# ./atomic --debug update registry.access.redhat.com/rhel7
Using default tag: latest
Trying to pull repository registry.access.redhat.com/rhel7 ... 
latest: Pulling from registry.access.redhat.com/rhel7
30cf2e26a24f: Pull complete 
99dd41655d8a: Pull complete 
Digest: sha256:eac2421be7a6844a5c83b8f394d1f5f121b18fa4e455c5f09be940e0384a1d97
Status: Downloaded newer image for registry.access.redhat.com/rhel7:latest

Comment 2 Alex Jia 2016-10-26 07:24:55 UTC
I gave a try for upstream skopeo, it works well for me.

[root@atomic-host-test skopeo]# git rev-parse HEAD
1f655f3f0910a87f844a0ada49eada23ad9fca03

[root@atomic-host-test cloud-user]# /usr/bin/skopeo copy --remove-signatures docker://registry.access.redhat.com/rhel7 docker-daemon:registry.access.redhat.com/rhel7:latest
Copying blob sha256:30cf2e26a24f2a8426cbe8444f8af2ecb7023bd468b05c1b6fd0b2797b0f9ff9
 67.95 MB / ? [------------------------------------------=--------------------] 
Copying blob sha256:99dd41655d8a45c2fb74f9eeb73e327b3ad4796f0ff0d602c575e32e9804baed
 0 B / 700 B [-----------------------------------------------------------------]
Copying config sha256:98a88a8b722a71835dd761c88451c681a8f1bc6e577f90d4dc8b234100bd4861
 0 B / 1.64 KB [---------------------------------------------------------------]
Writing manifest to image destination
Storing signatures
 1.64 KB / 1.64 KB [===========================================================]

[root@atomic-host-test cloud-user]# docker images rhel7
REPOSITORY                         TAG                 IMAGE ID            CREATED             SIZE
registry.access.redhat.com/rhel7   latest              98a88a8b722a        7 weeks ago         201.4 MB

Comment 3 Alex Jia 2017-08-17 06:01:59 UTC
It also works well in skopeo-0.1.23-1.git1bbd87f.el7.x86_64.

Comment 5 errata-xmlrpc 2017-09-05 10:37:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2601