1499274 – Pulling image failed on Atomic 7.4.2

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1499274 - Pulling image failed on Atomic 7.4.2

Summary: Pulling image failed on Atomic 7.4.2

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	skopeo
Sub Component:
Version:	7.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Lokesh Mandvekar
QA Contact:	atomic-bugs@redhat.com
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1496267 (view as bug list)
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2017-10-06 14:30 UTC by Jiri Belka
Modified:	2017-10-19 15:21 UTC (History)
CC List:	8 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2017-10-19 15:21:11 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2017:2971	0	normal	SHIPPED_LIVE	skopeo bug fix update	2017-10-19 18:54:38 UTC

Description Jiri Belka 2017-10-06 14:30:48 UTC

Description of problem:

It seems there's a bug in atomic/skopeo

[root@10-37-138-201 ~]# rpm -q atomic skopeo
atomic-1.19.1-3.gitb39a783.el7.x86_64
skopeo-0.1.23-1.git1bbd87f.el7.x86_64
[root@10-37-138-201 ~]# atomic host status
State: idle
Deployments:
● rhel-atomic-host-ostree:rhel-atomic-host/7/x86_64/standard
                   Version: 7.4.2 (2017-10-04 02:35:38)
                    Commit: 408c3cc96af3444733fc00068bcf0c43aa6c640156b089ed3e596cb7d68a5b01
              GPGSignature: Valid signature by 567E347AD0044ADE55BA8A5F199E2F91FD431D51


# atomic --debug pull --storage=ostree brew-pulp-docker01.example.com:8888/rhev4/ovirt-guest-agent:1.0.13-34
Namespace(_class=<class 'Atomic.pull.Pull'>, assumeyes=False, debug=True, func='pull_image', ignore=False, image='brew-pulp-docker01.example.com:8888/rhev4/ovirt-guest-agent:1.0.13-34', profile=False, reg_type=None, storage='ostree')
[31mFATA[0m[0000] Invalid destination name ostree:brew-pulp-docker01.example.com:8888/rhev4/ovirt-guest-agent:1.0.13-34@/ostree/repo: invalid tag format 

Traceback (most recent call last):
  File "/usr/bin/atomic", line 185, in <module>
    sys.exit(_func())
  File "/usr/lib/python2.7/site-packages/Atomic/pull.py", line 56, in pull_image
    be.pull_image(self.args.image, remote_image_obj, debug=self.args.debug, assumeyes=self.args.assumeyes)
  File "/usr/lib/python2.7/site-packages/Atomic/backends/_ostree.py", line 113, in pull_image
    return self.syscontainers.pull_image(image)
  File "/usr/lib/python2.7/site-packages/Atomic/syscontainers.py", line 181, in pull_image
    self._pull_image_to_ostree(self._get_ostree_repo(), image or self.args.image, True)
  File "/usr/lib/python2.7/site-packages/Atomic/syscontainers.py", line 171, in _pull_image_to_ostree
    self._check_system_oci_image(repo, image, upgrade)
  File "/usr/lib/python2.7/site-packages/Atomic/syscontainers.py", line 1870, in _check_system_oci_image
    return self._check_system_oci_image_skopeo_copy(repo, img)
  File "/usr/lib/python2.7/site-packages/Atomic/syscontainers.py", line 1885, in _check_system_oci_image_skopeo_copy
    util.skopeo_copy("docker://" + img, destination, dest_ostree_tmp_dir=temp_dir, insecure=insecure)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 433, in skopeo_copy
    return check_call(cmd, env=os.environ)
  File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 198, in check_call
    return subprocess.check_call(cmd, env=env, stdin=stdin, stderr=stderr, stdout=stdout, close_fds=True)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/skopeo', 'copy', '--dest-ostree-tmp-dir', '/var/lib/containers/atomic/.5Xb9TF', 'docker://brew-pulp-docker01.example.com:8888/rhev4/ovirt-guest-agent:1.0.13-34', 'ostree:brew-pulp-docker01.example.com:8888/rhev4/ovirt-guest-agent:1.0.13-34@/ostree/repo']' returned non-zero exit status 1


Version-Release number of selected component (if applicable):
atomic-1.19.1-3.gitb39a783.el7.x86_64
skopeo-0.1.23-1.git1bbd87f.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. install ovirt GA from brew
2.
3.

Actual results:
fails

Expected results:
should work

Additional info:

Comment 3 Steve Milner 2017-10-06 14:32:54 UTC

Giuseppe noted that this is fixed upstream https://github.com/containers/image/pull/342/commits/0044db84ada95f4626305b68ca00ed6ae29f6c24

We may want to force the old way until the above fix is applied and provided.

Comment 4 Steve Milner 2017-10-06 14:54:47 UTC

Moving to skopeo for applying the fix and rebuilding.

Comment 5 Steve Milner 2017-10-06 15:18:18 UTC

PR: https://github.com/projectatomic/skopeo/pull/429

Comment 10 Micah Abbott 2017-10-06 19:29:57 UTC

I retested this with the fixed atomic/skopeo builds and it works again:

# rpm -q atomic atomic-registries ostree skopeo skopeo-containers
atomic-1.19.1-4.gitb39a783.el7.x86_64
atomic-registries-1.19.1-4.gitb39a783.el7.x86_64
ostree-2017.11-1.el7.x86_64
skopeo-0.1.24-1.dev.git28d4e08.el7.x86_64
skopeo-containers-0.1.24-1.dev.git28d4e08.el7.x86_64

# cat /run/containers/registries.conf 
REGISTRIES=" --insecure-registry brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 --add-registry registry.access.redhat.com"

# atomic --debug pull --storage=ostree http:brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhev4/ovirt-guest-agent:1.0.13-34
Namespace(_class=<class 'Atomic.pull.Pull'>, assumeyes=False, debug=True, func='pull_image', ignore=False, image='http:brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhev4/ovirt-guest-agent:1.0.13-34', profile=False, reg_type=None, storage='ostree')
Getting image source signatures
Copying blob sha256:394cb3d19d983af65c6d747f63012c553105bc14176c2ac6357a330f2dcf43ac
 71.35 MB / 71.35 MB [======================================================] 4s
Copying blob sha256:98eb325fcc1b40892bcc40ac13534a3d9d019f3241c06e3e77e3043d9d7f7fad
 1.21 KB / 1.21 KB [========================================================] 0s
Copying blob sha256:abfafb7c2ca509b3faba599ae6478cd3b39b001b9d698c937385cf420896502d
 18.16 MB / 18.16 MB [======================================================] 1s
Copying config sha256:4fa3d4f596cb3bbc6bb24c2b2fb94d2674c22bd517918703202496e1a3f3d583
 4.09 KB / 4.09 KB [========================================================] 0s
Writing manifest to image destination
Storing signatures


NOTE:  we now require using TLS with skopeo.  To use an HTTP registry (like the one in the example), you have to prepend 'http:' to the <registry>/<image> string.

Comment 14 Micah Abbott 2017-10-10 13:50:42 UTC

Marking as VERIFIED with skopeo-0.1.24-1.dev.git28d4e08.el7.x86_64

Comment 15 Ed Santiago 2017-10-12 20:03:52 UTC

*** Bug 1496267 has been marked as a duplicate of this bug. ***

Comment 17 errata-xmlrpc 2017-10-19 15:21:11 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2971

Note You need to log in before you can comment on or make changes to this bug.