Bug 2064408 - operator-sdk 1.10-ocp: scorecard impacted by docker rate limit; switch to using ubi instead of busybox
Summary: operator-sdk 1.10-ocp: scorecard impacted by docker rate limit; switch to usi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Operator SDK
Version: 4.9
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 4.9.z
Assignee: Jesus M. Rodriguez
QA Contact: xzha
URL:
Whiteboard:
Depends On: 2064413
Blocks: 2058044
TreeView+ depends on / blocked
 
Reported: 2022-03-15 18:43 UTC by Jesus M. Rodriguez
Modified: 2022-04-20 14:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Hardcoded busybox image from docker was used for scorecard. Consequence: Docker's new rate limit would cause periodic failures when running scorecard. Fix: Use ubi for scorecard: "registry.access.redhat.com/ubi8/ubi:8.4" Result: No more failures due to rate limiting.
Clone Of: 2058044
: 2064413 (view as bug list)
Environment:
Last Closed: 2022-04-20 14:49:50 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift ocp-release-operator-sdk pull 222 0 None Merged Bug 2064408: use ubi instead of busybox for untar image 2022-04-12 18:11:06 UTC
Red Hat Product Errata RHSA-2022:1363 0 None None None 2022-04-20 14:50:24 UTC

Description Jesus M. Rodriguez 2022-03-15 18:43:21 UTC
This is the 4.9 clone to fix the untar image in operator-sdk 1.10-ocp for OCP 4.9

+++ This bug was initially created as a clone of Bug #2058044 +++

Description of problem:
operator-sdk version 1.8-ocp does not contain option to --untar-image and --storage-image that is harcoded to docker.io/busybox
CU would like to override this setting and be able to use image from private repository instead of pulling from docker.io

Version-Release number of selected component (if applicable):
OCP 4.8
operator-sdk 1-8-ocp

How reproducible:
Pulling an image from docker.io fails with You have reached your pull rate limit

Steps to Reproduce:
1. operator-sdk scorecard <bundle-dir> -o text --selector=suite=olm -w 600 -x
2. scorecard pod events log:

Failed to pull image "docker.io/busybox:1.33.0": rpc error: code = Unknown desc = Error reading manifest 1.33.0 in docker.io/library/busybox: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

Actual results:


Expected results:
Use scorecard without the need to pull images from docker.io to avoid pull rate limit issue

Comment 1 Jesus M. Rodriguez 2022-03-15 19:11:42 UTC
So the final solution for this bug was to use UBI as we did in a newer release of operator-sdk. Adding a new flag is effectively adding a new feature which for such an old release did not make sense. In OCP 4.10 with operator-sdk 1.16-ocp there is a flag that can solve this if you still have a problem.

Comment 2 Jesus M. Rodriguez 2022-04-07 02:46:25 UTC
this is currently blocked by a repo configuration that is causing e2e tests to fail. A PR to fix the test infrastructure has been posted. https://github.com/openshift/release/pull/27626

Comment 4 Jesus M. Rodriguez 2022-04-12 18:17:23 UTC
We replaced the busybox image from docker with "registry.access.redhat.com/ubi8/ubi:8.4".

Comment 5 xzha 2022-04-13 06:00:43 UTC
verify:

zhaoxia@xzha-mac memcached-operator % ./operator-sdk version
operator-sdk version: "v1.10.1-ocp", commit: "8efa92dcb512196f37bdccedeadadae22f7d8fdb", kubernetes version: "v1.21", go version: "go1.17.6", GOOS: "darwin", GOARCH: "amd64"

zhaoxia@xzha-mac memcached-operator % ./operator-sdk scorecard ./bundle -o text --selector=suite=olm -w 600s -x
--------------------------------------------------------------------------------
Image:      quay.io/operator-framework/scorecard-test:v1.10.1
Entrypoint: [scorecard-test olm-crds-have-validation]
Labels:
	"suite":"olm"
	"test":"olm-crds-have-validation-test"
Results:
	Name: olm-crds-have-validation
	State: fail

	Suggestions:
		Add CRD validation for spec field `foo` in Memcached/v1alpha1
	Log:
....

zhaoxia@xzha-mac memcached-operator % oc get pod
NAME                  READY   STATUS      RESTARTS   AGE
scorecard-test-5t94   0/1     Completed   0          6m41s
scorecard-test-bqqs   0/1     Completed   0          47s
scorecard-test-bs6x   0/1     Completed   0          47s
scorecard-test-cwsf   0/1     Completed   0          47s
scorecard-test-lhjg   0/1     Completed   0          6m41s
scorecard-test-ssv7   0/1     Completed   0          47s
scorecard-test-tpkl   0/1     Completed   0          6m41s
scorecard-test-vkmz   0/1     Completed   0          6m41s
scorecard-test-vwxg   0/1     Completed   0          6m41s
scorecard-test-wtmq   0/1     Completed   0          47s
zhaoxia@xzha-mac memcached-operator % oc get pod scorecard-test-wtmq -o yaml
apiVersion: v1
kind: Pod
...
initContainers:
  - args:
    - tar
    - xvzf
    - /scorecard/bundle.tar.gz
    - -C
    - /scorecard-bundle
    image: registry.access.redhat.com/ubi8/ubi:8.4
    imagePullPolicy: IfNotPresent
    name: scorecard-untar
    resources: {}
    securityContext:
      capabilities:
        drop:
        - MKNOD
    terminationMessagePath: /dev/termination-log
...


zhaoxia@xzha-mac memcached-operator % oc get events | grep "Pulling image"  
9m28s       Normal    Pulling                  pod/scorecard-test-5t94   Pulling image "registry.access.redhat.com/ubi8/ubi:8.4"
9m14s       Normal    Pulling                  pod/scorecard-test-5t94   Pulling image "quay.io/operator-framework/scorecard-test:v1.10.1"
9m28s       Normal    Pulling                  pod/scorecard-test-lhjg   Pulling image "registry.access.redhat.com/ubi8/ubi:8.4"
9m18s       Normal    Pulling                  pod/scorecard-test-lhjg   Pulling image "quay.io/operator-framework/scorecard-test:v1.10.1"
9m28s       Normal    Pulling                  pod/scorecard-test-tpkl   Pulling image "registry.access.redhat.com/ubi8/ubi:8.4"
9m14s       Normal    Pulling                  pod/scorecard-test-tpkl   Pulling image "quay.io/operator-framework/scorecard-test:v1.10.1"
9m28s       Normal    Pulling                  pod/scorecard-test-vkmz   Pulling image "registry.access.redhat.com/ubi8/ubi:8.4"
9m7s        Normal    Pulling                  pod/scorecard-test-vkmz   Pulling image "quay.io/operator-framework/scorecard-test:v1.10.1"
9m28s       Normal    Pulling                  pod/scorecard-test-vwxg   Pulling image "registry.access.redhat.com/ubi8/ubi:8.4"
9m6s        Normal    Pulling                  pod/scorecard-test-vwxg   Pulling image "quay.io/operator-framework/scorecard-test:v1.10.1"


LGTM, verified

Comment 10 errata-xmlrpc 2022-04-20 14:49:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.9.29 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1363


Note You need to log in before you can comment on or make changes to this bug.