Bug 2071632

openshift-tests.[sig-builds][Feature:Builds] build can reference a cluster service  with a build being created from new-build should be able to run a build that references a cluster service [Skipped:Disconnected] [Suite:openshift/conformance/parallel]

is failing frequently in CI, see:
https://sippy.ci.openshift.org/sippy-ng/tests/4.11/analysis?test=openshift-tests.%5Bsig-builds%5D%5BFeature%3ABuilds%5D%20build%20can%20reference%20a%20cluster%20service%20%20with%20a%20build%20being%20created%20from%20new-build%20should%20be%20able%20to%20run%20a%20build%20that%20references%20a%20cluster%20service%20%5BSkipped%3ADisconnected%5D%20%5BSuite%3Aopenshift%2Fconformance%2Fparallel%5D

Looks like problems began around April 1.

Sample job: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-e2e-azure-upgrade-single-node/1510700562876731392

The stdout for the test failure shows at least two suspicious things:

Apr  3 22:00:34.845: INFO: Log for pod "hello-nodejs-1-build"/"sti-build"
---->
time="2022-04-03T21:59:59Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
I0403 21:59:59.140963       1 defaults.go:102] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].
Caching blobs under "/var/cache/blobs".
Trying to pull registry.redhat.io/ubi8/nodejs-16@sha256:450338f5e5a8182d90f439233bc9b869bdcd169839d2829c2c6d0685d79fb64b...
Getting image source signatures
Copying blob sha256:6c155a0c493b106911b1fa3b975be2fc7e07ef3e8619b7d5b0d169488fce15d3
Copying blob sha256:3de00bb8554b2c35c89412d5336f1fa469afc7b0160045dd08758d92c8a6b064
Copying blob sha256:a5094e3fa10135eaef0c481941463b67add30b0c65cb02e9aef14d347a7a9f40
Copying blob sha256:c530010fb61c513e7f06fa29484418fabf62924fc80cc8183ab671a3ce461a8e
Copying config sha256:3424dbb26c2d3f327621a02e7c7279ebaa332dae34b075c27b72bf674bb92e5d
Writing manifest to image destination
Storing signatures
Generating dockerfile with builder image registry.redhat.io/ubi8/nodejs-16@sha256:450338f5e5a8182d90f439233bc9b869bdcd169839d2829c2c6d0685d79fb64b
Adding transient rw bind mount for /run/secrets/rhsm
error: build error: attempting to convert BUILD_LOGLEVEL env var value "" to integer: strconv.Atoi: parsing "": invalid syntax
<----end of log for "hello-nodejs-1-build"/"sti-build"

And:

STEP: Collecting events from namespace "e2e-test-build-service-2jzm7".
STEP: Found 26 events.
Apr  3 22:00:34.979: INFO: At 2022-04-03 21:59:29 +0000 UTC - event for hello-nodejs: {deployment-controller } ScalingReplicaSet: Scaled up replica set hello-nodejs-6788d76955 to 1
Apr  3 22:00:34.979: INFO: At 2022-04-03 21:59:29 +0000 UTC - event for hello-nodejs: {buildconfig-controller } BuildConfigInstantiateFailed: error instantiating Build from BuildConfig e2e-test-build-service-2jzm7/hello-nodejs (0): Error resolving ImageStreamTag nodejs-16:latest in namespace e2e-test-build-service-2jzm7: unable to find latest tagged image
Apr  3 22:00:34.979: INFO: At 2022-04-03 21:59:29 +0000 UTC - event for hello-nodejs-6788d76955: {replicaset-controller } FailedCreate: Error creating: Pod "hello-nodejs-6788d76955-c6v6n" is invalid: spec.containers[0].image: Invalid value: " ": must not have leading or trailing whitespace


Looks like this has pretty broad impact across a large number of CI jobs, setting high severity and requesting assistance asap.