Bug 2004193 - oc is crashing while mirroring registry
Summary: oc is crashing while mirroring registry
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.2.z
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: 4.8.z
Assignee: Ross Peoples
QA Contact: zhou ying
URL:
Whiteboard:
Depends On: 1786835
Blocks: 2004194
TreeView+ depends on / blocked
 
Reported: 2021-09-14 16:45 UTC by Ross Peoples
Modified: 2021-09-27 19:53 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
* Previously, there was an unchecked index operation on a slice when using the `--max components` argument. Consequently, the `oc` client returned a `panic: runtime error: slice bounds out of range` error and crashed. This update adds a check to ensure that a value is not requested for an index that is out of range. As a result, when using the `--max-components` argument, the `oc` client will no longer crash. (link:https://bugzilla.redhat.com/show_bug.cgi?id=2004193[*BZ#2004193*])
Clone Of: 1786835
: 2004194 (view as bug list)
Environment:
Last Closed: 2021-09-27 19:53:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift oc pull 926 0 None open Bug 2004193: Registry mirror panic 2021-09-14 16:48:14 UTC
Red Hat Product Errata RHBA-2021:3632 0 None None None 2021-09-27 19:53:29 UTC

Description Ross Peoples 2021-09-14 16:45:44 UTC
+++ This bug was initially created as a clone of Bug #1786835 +++

Description of problem:

oc client is crashing while creating a mirror registry on AWS

[root@mirror-registry ~]# ./oc adm -a pull-secret1.json release mirror      --from=quay.io/${PRODUCT_REPO}/${RELEASE_NAME}:${OCP_RELEASE}      --to=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}      --to-release-image=${LOCAL_REGISTRY}/${LOCAL_REPOSITORY}:${OCP_RELEASE}
panic: runtime error: slice bounds out of range

goroutine 45 [running]:
bufio.(*Reader).fill(0xc000c4a840)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/bufio/bufio.go:89 +0x211
bufio.(*Reader).WriteTo(0xc000c4a840, 0x2e12860, 0xc0000106e0, 0x7fec9d675770, 0xc000c4a840, 0x1)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/bufio/bufio.go:511 +0x106
io.copyBuffer(0x2e12860, 0xc0000106e0, 0x2e0d3e0, 0xc000c4a840, 0x0, 0x0, 0x0, 0x0, 0x7961757108000000, 0x2e6f692e)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/io/io.go:384 +0x34e
io.Copy(...)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/io/io.go:364
os/exec.(*Cmd).stdin.func1(0x463911, 0x4d03660)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/os/exec/exec.go:243 +0x67
os/exec.(*Cmd).Start.func1(0xc0010038c0, 0xc0006b7c60)
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/os/exec/exec.go:409 +0x27
created by os/exec.(*Cmd).Start
	/opt/rh/go-toolset-1.12/root/usr/lib/go-toolset-1.12-golang/src/os/exec/exec.go:408 +0x58f



Version-Release number of selected component (if applicable):

openshift-client-linux-4.2.12.tar.gz

How reproducible:

Always

Steps to Reproduce:
1. try to create a mirror registry on an AWS environment
2.
3.

Actual results:
oc is crashing

Expected results:
oc client should run seamlessly

Additional info:

--- Additional comment from Maciej Szulik on 2020-01-02 15:03:13 UTC ---

Can you provide exact steps and full dump of the errors, the current one makes it hard to figure out where and how it failed.

--- Additional comment from Maciej Szulik on 2020-03-13 15:28:34 UTC ---

I haven't heard from the author, thus closing.

--- Additional comment from  on 2021-08-04 21:56:15 UTC ---

Hi there, I believe I need to reopen this bug as I have a customer seeing the same error.

# oc version
Client Version: 4.6.7
Server Version: 4.7.21
Kubernetes Version: v1.20.0+558d959

# oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.7 example.com:5000 -a ${REG_CREDS} --insecure --filter-by-os="linux/amd64" --max-components=5
src image has index label for database path: /database/index.db
using database path mapping: /database/index.db:/tmp/861519587
wrote database to /tmp/861519587
using database at: /tmp/861519587/index.db
panic: runtime error: slice bounds out of range [:4] with capacity 2

goroutine 1 [running]:
github.com/openshift/oc/pkg/cli/admin/catalog.mappingForImages(0xc001608390, 0x2fb3672, 0x6, 0x0, 0x0, 0x0, 0x0, 0x7ffc9a2f76ca, 0x1f, 0x7ffc9a2f76ea, ...)
        /go/src/github.com/openshift/oc/pkg/cli/admin/catalog/mirrorer.go:174 +0x12ad
github.com/openshift/oc/pkg/cli/admin/catalog.(*IndexImageMirrorer).Mirror(0xc00049e1e0, 0x3, 0x3, 0xc00049e1e0)
        /go/src/github.com/openshift/oc/pkg/cli/admin/catalog/mirrorer.go:89 +0x148
github.com/openshift/oc/pkg/cli/admin/catalog.(*MirrorCatalogOptions).Run(0xc000f596c0, 0x0, 0x31c6048)
        /go/src/github.com/openshift/oc/pkg/cli/admin/catalog/mirror.go:355 +0x17a
github.com/openshift/oc/pkg/cli/admin/catalog.NewMirrorCatalog.func1(0xc0010aadc0, 0xc000761880, 0x2, 0x7)
        /go/src/github.com/openshift/oc/pkg/cli/admin/catalog/mirror.go:115 +0xbf
github.com/spf13/cobra.(*Command).execute(0xc0010aadc0, 0xc000761810, 0x7, 0x7, 0xc0010aadc0, 0xc000761810)
        /go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0xc0006cf080, 0x2, 0xc0006cf080, 0x2)
        /go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        /go/src/github.com/openshift/oc/vendor/github.com/spf13/cobra/command.go:887
main.main()
        /go/src/github.com/openshift/oc/cmd/oc/oc.go:110 +0x885


Please let me know what debugging I can provide to troubleshoot this.  Thank you!

--- Additional comment from  on 2021-08-04 22:20:37 UTC ---

Additional note, if we do not use the '--max-components=5', oc does not crash, BUT cu gets a credentials error, even though the same login works with podman.

# oc adm catalog mirror registry.redhat.io/redhat/redhat-marketplace-index:v4.7 example.com:5000 -a ${REG_CREDS} --insecure --filter-by-os="linux/amd64"
src image has index label for database path: /database/index.db
using database path mapping: /database/index.db:/tmp/636075942
wrote database to /tmp/636075942
using database at: /tmp/636075942/index.db

error: unable to retrieve source image registry.marketplace.redhat.com/rhm/cloudhedge/cloud-infra-service manifest sha256:34da7b1c709ba1478ca6cd2fe7d0ecac5f727857c0ade01eaf1144db86377bf0: Get "https://registry.marketplace.redhat.com/v2/rhm/cloudhedge/cloud-infra-service/manifests/sha256:34da7b1c709ba1478ca6cd2fe7d0ecac5f727857c0ade01eaf1144db86377bf0": unauthorized: The login credentials are not valid, or your IBM Cloud account is not active.
error mirroring image: an error occurred during planning

--- Additional comment from OpenShift Automated Release Tooling on 2021-08-12 10:28:39 UTC ---

Elliott changed bug status from MODIFIED to ON_QA.

--- Additional comment from zhou ying on 2021-08-24 11:49:40 UTC ---

can't reproduce with latest oc client:
[root@localhost registry]# oc version --client 
Client Version: 4.9.0-202108192235.p0.git.5622598.assembly.stream-5622598

[root@localhost registry]# oc adm release mirror      --from=quay.io/openshift-release-dev/ocp-release:4.8.6-x86_64  --to=localhost:5000/ocp-index --to-release-image=localhost:5000/ocp-index:48 --insecure
......
To use the new mirrored repository to install, add the following section to the install-config.yaml:

imageContentSources:
- mirrors:
  - localhost:5000/ocp-index
  source: quay.io/openshift-release-dev/ocp-release
- mirrors:
  - localhost:5000/ocp-index
  source: quay.io/openshift-release-dev/ocp-v4.0-art-dev


To use the new mirrored repository for upgrades, use the following to create an ImageContentSourcePolicy:

apiVersion: operator.openshift.io/v1alpha1
kind: ImageContentSourcePolicy
metadata:
  name: example
spec:
  repositoryDigestMirrors:
  - mirrors:
    - localhost:5000/ocp-index
    source: quay.io/openshift-release-dev/ocp-release
  - mirrors:
    - localhost:5000/ocp-index
    source: quay.io/openshift-release-dev/ocp-v4.0-art-dev

oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.7 localhost:5000 --insecure --filter-by-os="linux/amd64" --max-components=5

both run well.

--- Additional comment from errata-xmlrpc on 2021-09-07 08:59:51 UTC ---

This bug has been added to advisory RHEA-2021:81213 by OpenShift Release Team Bot (ocp-build/buildvm.openshift.eng.bos.redhat.com)

--- Additional comment from Eric Rich on 2021-09-14 15:33:01 UTC ---

@rpeoples can this be backported to 4.6?

Comment 6 zhou ying 2021-09-22 06:44:23 UTC
retested with latest oc client, can't reproduce now:

[root@localhost ~]# ./oc version --client
Client Version: 4.8.0-0.nightly-2021-09-21-220555

[root@localhost ~]# ./oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.7 localhost:5000 --insecure --filter-by-os="linux/amd64" --max-components=5
Flag --filter-by-os has been deprecated, use --index-filter-by-os instead
src image has index label for database path: /database/index.db
using database path mapping: /database/index.db:/tmp/863614584
W0922 14:35:52.190226  736818 manifest.go:442] Chose linux/amd64 manifest from the manifest list.
wrote database to /tmp/863614584
using database at: /tmp/863614584/index.db

Comment 8 errata-xmlrpc 2021-09-27 19:53:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.13 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3632


Note You need to log in before you can comment on or make changes to this bug.