Bug 1549832 - CRI-O runtime - cannot delete builds.build.openshift.io in the namespace
Summary: CRI-O runtime - cannot delete builds.build.openshift.io in the namespace
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Build
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 3.9.0
Assignee: Corey Daley
QA Contact: Vikas Laad
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-27 20:49 UTC by Vikas Laad
Modified: 2018-06-18 18:29 UTC (History)
4 users (show)

Fixed In Version: openshift v3.9.3
Doc Type: Bug Fix
Doc Text:
Cause: Service account did not have permission to delete builds Consequence: Builds were not being pruned correctly Fix: Added correct permission to service account Result: Builds are now pruned correctly
Clone Of:
Environment:
Last Closed: 2018-06-18 17:44:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
master logs (9.20 MB, application/zip)
2018-02-27 20:54 UTC, Vikas Laad
no flags Details

Description Vikas Laad 2018-02-27 20:49:29 UTC
Description of problem:
I see lot of these errors in master logs. I am noticing it as part of Reliability tests run for 3.9. Reliability tests use quickstart apps to create/delete/build/scale apps over a period of time.

Feb 27 17:10:24 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:10:24.673624   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-201-72-335-124": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-201-72-335-124"
Feb 27 17:12:32 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:12:32.715105   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-201-72-335-124": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-201-72-335-124"
Feb 27 17:48:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:48:45.812548   15884 buildconfig_controller.go:103] builds.build.openshift.io "django-psql-example-1" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "django-psql-u-234-83-368-135": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "django-psql-u-234-83-368-135"
Feb 27 17:48:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:48:45.812643   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-49-18-351-130": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-49-18-351-130"
Feb 27 17:58:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:58:45.812275   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-49-18-351-130": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-49-18-351-130"
Feb 27 17:58:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 17:58:45.812324   15884 buildconfig_controller.go:103] builds.build.openshift.io "django-psql-example-1" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "django-psql-u-234-83-368-135": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "django-psql-u-234-83-368-135"
Feb 27 18:08:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:08:45.812832   15884 buildconfig_controller.go:103] builds.build.openshift.io "django-psql-example-1" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "django-psql-u-234-83-368-135": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "django-psql-u-234-83-368-135"
Feb 27 18:08:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:08:45.812880   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-49-18-351-130": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-49-18-351-130"
Feb 27 18:18:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:18:45.812790   15884 buildconfig_controller.go:103] builds.build.openshift.io "django-psql-example-1" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "django-psql-u-234-83-368-135": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "django-psql-u-234-83-368-135"
Feb 27 18:18:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:18:45.812844   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-49-18-351-130": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-49-18-351-130"
Feb 27 18:28:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:28:45.814070   15884 buildconfig_controller.go:103] builds.build.openshift.io "dancer-mysql-example-2" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "dancer-mysql-u-49-18-351-130": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "dancer-mysql-u-49-18-351-130"
Feb 27 18:28:45 ip-172-31-58-173.us-west-2.compute.internal atomic-openshift-master-controllers[15884]: E0227 18:28:45.814070   15884 buildconfig_controller.go:103] builds.build.openshift.io "django-psql-example-1" is forbidden: User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in the namespace "django-psql-u-234-83-368-135": User "system:serviceaccount:openshift-infra:build-config-change-controller" cannot delete builds.build.openshift.io in project "django-psql-u-234-83-368-135"

Version-Release number of selected component (if applicable):
openshift v3.9.0-0.53.0
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.8


How reproducible:
I see it the error 209 times in 1 day

Steps to Reproduce:
1. create/build/delete/modify lot of quickstart apps
2. watch master logs

Actual results:
Errors in master logs.

Expected results:
No errors in logs.

Additional info:
Master logs attached.

Comment 1 Vikas Laad 2018-02-27 20:54:04 UTC
Created attachment 1401531 [details]
master logs

Comment 2 Corey Daley 2018-02-28 20:37:51 UTC
Submitted pull request: https://github.com/openshift/origin/pull/18781

Comment 3 Ben Parees 2018-03-01 14:06:20 UTC
PR for 3.9: https://github.com/openshift/origin/pull/18782

Comment 4 Ben Parees 2018-03-06 15:00:18 UTC
Corey please don't forget to move your bugs to modified when your PRs merge.

Comment 6 Vikas Laad 2018-03-08 22:38:58 UTC
Verified on following version, did not see these errors

openshift v3.9.3
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.16


Note You need to log in before you can comment on or make changes to this bug.