Bug 1492999 - Enabling admission plugins with configurations fails by using DefaultAdmissionConfig
Summary: Enabling admission plugins with configurations fails by using DefaultAdmissio...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.7.0
Assignee: Avesh Agarwal
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-19 06:55 UTC by DeShuai Ma
Modified: 2017-11-28 22:11 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-28 22:11:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:3188 0 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-29 02:34:54 UTC

Description DeShuai Ma 2017-09-19 06:55:02 UTC
Description of problem:
No matter set 'disable=true' or 'disable=false' for PodTolerationRestriction admission controller, it always can inject toleration to pod.
This parameter can take effect for other admission controller, eg:'AlwaysPullImages'

Version-Release number of selected component (if applicable):
openshift v3.7.0-0.126.4
kubernetes v1.7.0+80709908fd
etcd 3.2.1

How reproducible:
Always

Steps to Reproduce:
1. Configure PodTolerationRestriction in master-config.yaml and make sure disable=true, then restart atomic-openshift-master-api
----
admissionConfig:
  pluginConfig:
    PodTolerationRestriction:
      configuration:
        apiVersion: podtolerationrestriction.admission.k8s.io/v1alpha1
        kind: Configuration
        disable: true
        default:
         - key: key1
           operator: Equal
           value: value1
        whitelist:
         - key: key1
           operator: Equal
           value: value1
         - key: key2
           value: value2
-bash-4.2# systemctl restart atomic-openshift-master-api

2. Create a pod without tolerations
oc create -f https://raw.githubusercontent.com/mdshuai/testfile-openshift/master/k8s/hello-pod.yaml
-bash-4.2# oc get pod
NAME        READY     STATUS    RESTARTS   AGE
hello-pod   1/1       Running   0          9m

3. When pod becomes running, check the tolerations of pod
oc describe pod hello-pod | grep -i tolerations

Actual results:
3. The cluster default toleration inject to the pod
-bash-4.2# oc describe pod hello-pod | grep -i tolerations
Tolerations:	key1=value1


Expected results:
3. The pod shouldn't have any toleration, as I have set 'disable: true'

Additional info:

Comment 1 Avesh Agarwal 2017-09-20 18:36:28 UTC
Correct way to disable PodTolerationRestriction admission plugin is as follows:

admissionConfig:
  pluginConfig:
    PodTolerationRestriction: 
      configuration:
        kind: DefaultAdmissionConfig
        apiVersion: v1
        disable: true


Once you do above, and checked the master log at log level 2, it will show following:

I0920 14:31:41.048629   16359 register.go:134] Admission plugin PodTolerationRestriction is not enabled.  It will not be started.

Since the plugin PodTolerationRestriction is off by default, it does not make much sense to disable it explicitly.

Comment 2 DeShuai Ma 2017-09-21 09:47:30 UTC
Thanks for your explain. But there is one question make me confused, why? when enable PodTolerationRestriction, The configuration is:
admissionConfig:
  pluginConfig:
    PodTolerationRestriction:
      configuration:
        apiVersion: podtolerationrestriction.admission.k8s.io/v1alpha1
        kind: Configuration
        default:
         - key: key1
           operator: Equal
           value: value1
        whitelist:
         - key: key1
           operator: Equal
           value: value1
         - key: key2
           value: value2

But when disable the configuration is: 
admissionConfig:
  pluginConfig:
    PodTolerationRestriction: 
      configuration:
        kind: DefaultAdmissionConfig
        apiVersion: v1
        disable: true

kind & apiVersion is total different, thanks.

Comment 3 DeShuai Ma 2017-09-21 09:49:33 UTC
Also I can't enable PodTolerationRestriction by:
admissionConfig:
  pluginConfig:
    PodTolerationRestriction:
      configuration:
        kind: DefaultAdmissionConfig
        apiVersion: v1
        disable: false
        default:
         - key: key1
           operator: Equal
           value: value1
        whitelist:
         - key: key1
           operator: Equal
           value: value1
         - key: key2
           value: value2

It will be error as below:
Sep 21 09:48:57 ip-172-18-0-234.ec2.internal atomic-openshift-master-api[12584]: F0921 09:48:57.987322       1 start_api.go:67] Couldn't init admission plugin "PodTolerationRestriction": no kind "DefaultAdmissionConfig" is registered for version "v1"

Comment 4 Avesh Agarwal 2017-09-22 13:46:33 UTC
I have sent https://github.com/openshift/origin/pull/16505 to address this issue. But in general, the enabling or disabling of admission plugins work as follows (other than their default on and off mechanism): 

1. Via DefaultAdmissionConfig
2. Via their own configurations if they have one

In the past, admission plugins with configuration have been enabled by passing their own configurations, and so not able to enable these plugins via DefaultAdmissionConfig has not been an issue it seems. 

But anyway, I have sent https://github.com/openshift/origin/pull/16505 to addresses issue with admission plugins with configuration and DefaultAdmissionConfig.

Comment 5 Avesh Agarwal 2017-09-22 13:49:55 UTC
Also this issue is not specific to to PodTolerationRestriction plugin and can happen with any plugins with configurations such as ResourceQuota etc, so changing the title would be good.

Comment 6 Avesh Agarwal 2017-09-22 13:52:18 UTC
I have changed the title of this bug to correctly reflect the issue, feel free to change further as needed.

Comment 7 DeShuai Ma 2017-09-25 04:28:10 UTC
Thanks for the fix

Comment 8 Seth Jennings 2017-10-05 13:13:48 UTC
New Origin PR:
https://github.com/openshift/origin/pull/16639

Comment 9 DeShuai Ma 2017-10-13 08:01:38 UTC
Verify on openshift v3.7.0-0.147.0 when configure like below then restart atomic-openshift-master-api.service, now there is no the error.
admissionConfig:
  pluginConfig:
    PodTolerationRestriction:
      configuration:
        kind: DefaultAdmissionConfig
        apiVersion: v1
        disable: false
        default:
         - key: key1
           operator: Equal
           value: value1
        whitelist:
         - key: key1
           operator: Equal
           value: value1
         - key: key2
           value: value2

Comment 10 DeShuai Ma 2017-10-13 08:02:12 UTC
Could you help move to ON_QA, thanks

Comment 11 Seth Jennings 2017-10-13 13:35:43 UTC
DeShuai,

Unclear what needs to be done.  Sounds like you verified the fix.  If so, please just move this to VERIFIED state.

Comment 15 errata-xmlrpc 2017-11-28 22:11:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188


Note You need to log in before you can comment on or make changes to this bug.