Bug 1885179 - oc image append fails on push (uploading a new layer)
Summary: oc image append fails on push (uploading a new layer)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.6
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Sally
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-10-05 10:58 UTC by Ricardo Maraschini
Modified: 2021-02-24 15:23 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:22:35 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift oc pull 604 0 None closed Bug 1885179: Avoid deadlock by closing pipe reader end 2020-10-27 12:24:13 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:23:05 UTC

Description Ricardo Maraschini 2020-10-05 10:58:34 UTC
Description of problem:

oc image append fails when pushing a new layer to an image hosted in quay.io or docker.io registries.


Version-Release number of selected component (if applicable):

Client Version: 4.6.0-fc.8

How reproducible:

Always

Steps to Reproduce:
1. Create a new tar for the new layer
  1.1 mkdir layer
  1.2 cd layer
  1.2 echo test > some-file
  1.3 tar -cvf new-layer.tar some-file
2. Upload the layer:
  2.1 oc image append --from=quay.io/rmarasch/my-image:latest --to=quay.io/rmarasch/my-image:append new-layer.tar
  2.2 repository above exists

Actual results:

Upload to quay.io fails with

Uploading ... error: unable to upload new layer (0): Patch "https://quay.io/v2/rmarasch/my-image/blobs/uploads/9ecb7583-2060-4f1d-8ccc-71a5141cd462": stream error: stream ID 17; PROTOCOL_ERROR


Upload to docker.io fails with

Uploading ... error: unable to upload new layer (0): received unexpected HTTP status: 504 Gateway Time-out


Expected results:

New layer is uploaded successfully. 

Additional info:

Comment 1 Ricardo Maraschini 2020-10-06 08:45:29 UTC
It turns out that the image layer I was trying to upload was considered invalid (not a gzip'd one). The error message was misguiding, I have proposed a fix for this on https://github.com/openshift/oc/pull/604

Comment 3 RamaKasturi 2020-10-28 07:13:40 UTC
Verified with the payload below and i see that error message is clear now when uploading a layer which is not a gzip'd one.

[ramakasturinarra@dhcp35-60 openshift-client-linux-4.7.0-0.nightly-2020-10-27-051128]$ ./oc version
Client Version: 4.7.0-0.nightly-2020-10-27-051128
Server Version: 4.7.0-0.nightly-2020-10-27-051128
Kubernetes Version: v1.19.0+e67f5dc


when uploading a layer which is not a gziped one below error is seen with latest oc client

[ramakasturinarra@dhcp35-60 openshift-client-linux-4.7.0-0.nightly-2020-10-27-051128]$ sudo ./oc image append --from=quay.io/knarra/my-kube-scheduler43:latest --to=quay.io/knarra/my-kube-scheduler43:append layer/new-layer.tar
Uploading ... error: unable to upload new layer (0): Patch "https://quay.io/v2/knarra/my-kube-scheduler43/blobs/uploads/02bebf5f-9e1c-4e15-97f4-1316fe04b8bb": unable to create gzip reader layer upload: gzip: invalid header

when uploading a layer which is gziped, appened works fine with out any issues
sudo ./oc image append --from=quay.io/knarra/testcli:test --to=quay.io/knarra/testcli:append newdir.tar.gz
Uploading ... 69.42B/s
Pushed sha256:af2ae401755f2db5bd8db94f8a2066ceebf9ab751aa425072fba80ad012e2989 to quay.io/knarra/testcli:append


When uploading a layer which is not a gziped one error was misguiding in older oc client
[ramakasturinarra@dhcp35-60 openshift-client-linux-4.7.0-0.nightly-2020-10-27-051128]$ oc version
Client Version: 4.6.0-202009281501.p0-61364f0

[ramakasturinarra@dhcp35-60 openshift-client-linux-4.7.0-0.nightly-2020-10-27-051128]$ sudo oc image append --from=quay.io/knarra/my-kube-scheduler43:latest --to=quay.io/knarra/my-kube-scheduler43:append layer/new-layer.tar
Uploading ... error: unable to upload new layer (0): Patch "https://quay.io/v2/knarra/my-kube-scheduler43/blobs/uploads/13eb5d10-74de-43a6-9117-3f88b25e27e7": stream error: stream ID 17; PROTOCOL_ERROR

Based on the above moving bug to verified state.

Comment 6 errata-xmlrpc 2021-02-24 15:22:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.