Bug 1767219 - Builds Without BuildConfigs Cause Build Controller to Have Errant Pruning Behavior
Summary: Builds Without BuildConfigs Cause Build Controller to Have Errant Pruning Beh...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Build
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.3.0
Assignee: Gabe Montero
QA Contact: wewang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-30 22:42 UTC by Steve Kuznetsov
Modified: 2020-01-23 11:10 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The build controller sometimes incorrectly assumed a build was instantiated from the build config endpoint when in fact it was instantiated directly from the build endpoint. Consequence: Confusing logging about non-existent build configs could appear in the build controller logs if a user instantiated a openshift build diretly (vs. initiating a build request off of the build config api endpoint) Fix: The build controller was update to better check whether a build was instantiated from the build config endpoint and refrain from loggin unnecessary error messages. Result: The build controller logs no longer have these confusing error messages for build instantiated directly vs. from the build config endpoint.
Clone Of:
Environment:
Last Closed: 2020-01-23 11:10:15 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift openshift-controller-manager pull 48 0 'None' closed Bug 1767218: Bug 1767219: don't enqueue non-existent buildconfigs for builds 2020-03-13 05:51:17 UTC
Red Hat Product Errata RHBA-2020:0062 0 None None None 2020-01-23 11:10:30 UTC

Description Steve Kuznetsov 2019-10-30 22:42:01 UTC
Description of problem:

If you create a Build without a BuildConfig in a Namespace, the build controller will emit the following logs into the controller-manager log stream:

E1030 18:40:37.789888       1 build_controller.go:1166] failed to prune builds for ci-op-shiv3w0f/src: buildconfig.build.openshift.io "" not found
E1030 18:40:43.447098       1 build_controller.go:1166] failed to prune builds for ci-op-p17vndip/operator-lifecycle-manager: buildconfig.build.openshift.io "" not found

This is incorrectly assuming that the Build has a BuildConfig

Comment 1 Adam Kaplan 2019-11-01 20:40:36 UTC
Per discussion with Steve on slack, this isn't a high priority bug, but did interfere with debugging recent incidents on our CI cluster.

Targeting 4.4 as this is an existing 3.11 issue that shouldn't block the 4.3 release.

Comment 2 Gabe Montero 2019-11-19 16:57:19 UTC
PR https://github.com/openshift/openshift-controller-manager/pull/48 should handle this bug as well as 1767218

Comment 4 Gabe Montero 2019-11-20 21:00:07 UTC
Testing for this would include
- create a valid build object directly in a namespace vs. creating a build config and running oc start-build
- see the build complete
- repeat this 5 times
- look at the logs for the 3 pods in the openshift-controller-manager namespace and make sure there are no build_controller logs with "failed to prune builds"

Comment 5 wewang 2019-11-21 07:43:43 UTC
I can reproduce the issue in the following  version, wait latest payload which is include the pr, will verified it.
version:
4.3.0-0.nightly-2019-11-19-122017

[root@Desktop ~]# oc logs -f pod/controller-manager-w6drh -n openshift-controller-manager |grep -i "prune builds"
E1121 07:26:36.211190       1 build_controller.go:1521] failed to prune builds for default/ruby-ex-7: builds.build.openshift.io "ruby-ex-1" not found

Comment 6 wewang 2019-11-22 08:28:41 UTC
Verified in version:
4.3.0-0.nightly-2019-11-21-122827

Steps:
 1. Create a build with buildconfig in my project
  $oc create -f build.yaml
  ----------build.yaml---------------
  apiVersion: v1
  kind: Build
  metadata:
    labels:
    name: build-without-bc-1
  spec:
    resources: {}
    source:
      git:
        uri: git://github.com/openshift/ruby-hello-world
      type: Git
    strategy:
      sourceStrategy:
        from:
          kind: ImageStreamTag
          name: ruby:2.5
          namespace: openshift
      type: Source
   -------------------------------------

2. Create 6 builds 
   $oc start-build --from-build=build-without-bc-1
   build-without-bc-1              Source   Git@7e2b285   Complete                       5 minutes ago        59s
   build-without-bc-1-7514139911   Source   Git@7e2b285   Complete                       4 minutes ago        1m18s
   build-without-bc-1-2939714793   Source   Git@7e2b285   Complete                       4 minutes ago        1m4s
   build-without-bc-1-6971280066   Source   Git@7e2b285   Complete                       4 minutes ago        1m7s
   build-without-bc-1-8970521568   Source   Git@7e2b285   Complete                       4 minutes ago        1m16s
   build-without-bc-1-0382288048   Source   Git@7e2b285   Complete                       3 minutes ago        54s

3. Check the the 3 pods log in openshift-controller-manager, no info about "failed to prune builds"

Comment 8 errata-xmlrpc 2020-01-23 11:10:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062


Note You need to log in before you can comment on or make changes to this bug.