Bug 1324280 - Fluentd pod refused to start up due to SCC constrain
Summary: Fluentd pod refused to start up due to SCC constrain
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.1.0
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Luke Meyer
QA Contact: chunchen
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-06 03:56 UTC by Xia Zhao
Modified: 2016-09-30 02:17 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-15 02:47:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Xia Zhao 2016-04-06 03:56:07 UTC
Problem description: 
Fluentd pod refused to start up due to SCC forbidden, get this error from event:
2m          4s         8         logging-fluentd-1              ReplicationController                                                 FailedCreate        {replication-controller }                        Error creating: Pod "logging-fluentd-1-" is forbidden: unable to validate against any security context constraint: [provider restricted: .spec.containers[0].securityContext.privileged: invalid value 'true', Details: Privileged containers are not allowed provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'varlog', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'varlibdockercontainers', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'dockerhostname', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.containers.0.hostPort: invalid value '1095', Details: Host ports are not allowed to be used provider hostmount-anyuid: .spec.containers[0].securityContext.privileged: invalid value 'true', Details: Privileged containers are not allowed provider hostmount-anyuid: .spec.containers[0].securityContext.containers.0.hostPort: invalid value '1095', Details: Host ports are not allowed to be used]

Version-Release number of selected component (if applicable):
logging-deployment      3.1.1-12            1889baecfc21
logging-fluentd         3.1.1-9             6a4bfd80f3eb
logging-elasticsearch   3.1.1-9             c0901c52554b
logging-kibana          3.1.1-7             3ce38d905617
logging-auth-proxy      latest              3d6792a3aeed

How reproducible:
Always

Steps to Reproduce:
1. Login openshift, create a project
oc new-project logging
2. Create supporting service account and deployer secrets
oc create -f - <<API
apiVersion: v1
kind: ServiceAccount
metadata:
 name: logging-deployer
secrets:
- name: logging-deployer
API
oc secrets new logging-deployer nothing=/dev/null

#Login master node and run below commands:
oadm policy add-role-to-user edit system:serviceaccount:logging:logging-deployer
oadm policy add-scc-to-user privileged system:serviceaccount:logging:aggregated-logging-fluentd
oadm policy add-cluster-role-to-user cluster-reader system:serviceaccount:logging:aggregated-logging-fluentd

3. Go back to oc client command line, make sure you are in logging project, then run the deployer:
wget https://raw.githubusercontent.com/openshift/origin-aggregated-logging/master/deployment/deployer.yaml
oc process -f deployer.yaml -v IMAGE_PREFIX=${image_prefix},\
              KIBANA_HOSTNAME=${kibana_route},\
              PUBLIC_MASTER_URL=https://${master_dns}:8443,\
              ES_INSTANCE_RAM=1024M,\
              ES_CLUSTER_SIZE=1,\
              | oc create -f -
4. Wait for deployer pod completed
5. Run "oc process logging-support-template | oc create -f -"
6. After es and kibana pod is running, scale up fluentd rc:
oc scale rc logging-fluentd-1 --replicas=1

Actual Result:
Fluentd pod refused to start up due to SCC forbidden, get this error from event:
2m          4s         8         logging-fluentd-1              ReplicationController                                                 FailedCreate        {replication-controller }                        Error creating: Pod "logging-fluentd-1-" is forbidden: unable to validate against any security context constraint: [provider restricted: .spec.containers[0].securityContext.privileged: invalid value 'true', Details: Privileged containers are not allowed provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'varlog', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'varlibdockercontainers', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.VolumeMounts: invalid value 'dockerhostname', Details: Host Volumes are not allowed to be used provider restricted: .spec.containers[0].securityContext.containers.0.hostPort: invalid value '1095', Details: Host ports are not allowed to be used provider hostmount-anyuid: .spec.containers[0].securityContext.privileged: invalid value 'true', Details: Privileged containers are not allowed provider hostmount-anyuid: .spec.containers[0].securityContext.containers.0.hostPort: invalid value '1095', Details: Host ports are not allowed to be used]

Expected Result:
Fluentd pod should be able to startup

Additional info:
1.Issue did not exit on 3.2 images
2.Issue repro when using hostmount-anyuid scc:
$ oadm policy add-scc-to-user hostmount-anyuid system:serviceaccount:logging:aggregated-logging-fluentd

Comment 2 Xia Zhao 2016-04-06 05:46:17 UTC
This is proved to be a script issue. I'm actually using 
$ oadm policy add-scc-to-user hostmount-anyuid system:serviceaccount:logging:aggregated-logging-fluentd
instead of
$ oadm policy add-scc-to-user privileged system:serviceaccount:logging:aggregated-logging-fluentd
which caused the issue. Did not repro after scirpt is updated. Closing.

Comment 3 Lutz Lange 2016-05-23 12:52:10 UTC
I tried both of the above command to fix my fluentd not starting up, but had no luck. I'm running OSE 3.2.

This is my error :

May 23 12:50:51 tmaster atomic-openshift-master: E0523 12:50:51.429178     913 replication_controller.go:442] unable to create pods: pods "logging-fluentd-6-" is forbidden: unable to validate against any security context constraint: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed spec.containers[0].securityContext.volumes[0]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.volumes[1]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.volumes[3]: Invalid value: "hostPath": hostPath volumes are not allowed to be used spec.containers[0].securityContext.containers[0].hostPort: Invalid value: 1095: Host ports are not allowed to be used]

Comment 4 Luke Meyer 2016-05-23 13:22:02 UTC
(In reply to Lutz Lange from comment #3)
> I tried both of the above command to fix my fluentd not starting up, but had
> no luck. I'm running OSE 3.2.

Does "both of the above" mean:
$ oadm policy add-scc-to-user hostmount-anyuid system:serviceaccount:logging:aggregated-logging-fluentd

and

$ oadm policy add-scc-to-user privileged system:serviceaccount:logging:aggregated-logging-fluentd

?

Is your logging deployment in the `logging` namespace or somewhere else?

The errors you list indicate that the serviceaccount in your namespace has not been added to either indicated SCC (need only the `privileged` one above, at least for now).

If you can't get it working, it may help to attach the yaml:

$ oc get sa,dc,scc -o yaml

Comment 5 Lutz Lange 2016-05-23 13:35:46 UTC
This runs in the openshift project like the documentation point me to.

Please find my yamls attached.

Comment 7 Lutz Lange 2016-05-23 15:28:29 UTC
The problem is our documentation that had me creating this in the "openshift" project.

Comment 8 Luke Meyer 2016-05-23 15:36:41 UTC
Can you link to which documentation points you to using the "openshift" project?

It's fine to use that, you just need to adjust the commands to apply the SCC and roles to the service accounts in the right project. So, for example,

$ oadm policy add-scc-to-user privileged system:serviceaccount:openshift:aggregated-logging-fluentd

See step 7 under https://docs.openshift.com/enterprise/3.2/install_config/aggregate_logging.html#pre-deployment-configuration

Comment 9 Luke Meyer 2016-06-15 02:47:47 UTC
Closing this for now, docs bugs welcome.


Note You need to log in before you can comment on or make changes to this bug.