Bug 1420538 - Unable to set Supplemental Groups or fsGroup for Cassandra, via metrics deployer.
Summary: Unable to set Supplemental Groups or fsGroup for Cassandra, via metrics deplo...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.4.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Jeff Cantrill
QA Contact: Peng Li
Depends On:
TreeView+ depends on / blocked
Reported: 2017-02-08 22:27 UTC by Ryan Howe
Modified: 2017-07-24 14:11 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Last Closed: 2017-04-12 19:01:11 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0903 0 normal SHIPPED_LIVE OpenShift Container Platform atomic-openshift-utils bug fix and enhancement 2017-04-12 22:45:42 UTC

Description Ryan Howe 2017-02-08 22:27:35 UTC
Description of problem:

There is not easy way with via the deployer to set Supplemental Groups or fsGroup. These would need to be manually added after a deploy. In which case the deploy would fail because it does not have permissions to write to the PV that is attached. 


Comment 1 Jeff Cantrill 2017-02-09 13:35:39 UTC
@Matt, if he is using the deployer then the 'easy' way would be to edit the deployer template to include the supplemental group before executing the 'oc new-app' step.  Is that correct?

Comment 2 Matt Wringe 2017-02-09 14:20:41 UTC
@jeff: no, that is not how it works. This doesn't need to be applied to the deployer template, but to the Cassandra template. The Cassandra template is not added until the deployment starts, which in this case will fail because there isn't a PV for it to use.

So they need to deploy metrics, then scale everything to 0, then modify the cassandra template, then scale everything back up.

And when they go to update or reinstall, then they need to do these steps all over again.

Trying to handle everything as parameters passed to the deployer or ansible is probably not a good idea. There are too many options people will want, we can't just continue to add in more and more options. Eventually we are going to run into hundreds of parameters.

Its also extremely confusing for users. They will know they want to have option X that OpenShift provides, then they need to read the docs to see what that corresponding option is for ansible/deployer and then figure out how to add it as a parameter (or most likely complain that we don't expose that option).

Comment 3 Ryan Howe 2017-02-09 14:38:14 UTC
All the templates that directly deploy the pods for each metric component come from within the deployer image. The template used to deploy the metrics does not allow the user to make direct changes to the any of the components unless we have added a parameter that can be passed to the deploy, where the deployer will then pass it to the components.

The parameter that can be set can be seen here: 

The templates for each component are here and these can only be customized if you build a new deployer image, which is not supported.

#  docker run --entrypoint /bin/bash -it openshift3/metrics-deployer  -c "/bin/cat templates/*"

Comment 4 Jeff Cantrill 2017-02-09 15:15:57 UTC
We could make a change to the deployer similiar to what logging is doing: https://github.com/openshift/origin-aggregated-logging/blob/v1.4.1/deployer/templates/es.yaml#L48

Comment 7 Jeff Cantrill 2017-02-09 19:20:19 UTC
fixed in https://github.com/openshift/openshift-ansible/pull/3320

Comment 8 Matt Wringe 2017-02-09 20:02:30 UTC
@jeff: do we also need a change in the deployer? I guess not since I don't think we need this for the 'oc up' command.

Comment 9 Jeff Cantrill 2017-02-09 20:38:10 UTC
I dont anticipate this being back ported to the deployer unless we are directed from PM

Comment 10 openshift-github-bot 2017-02-10 14:06:32 UTC
Commits pushed to master at https://github.com/openshift/openshift-ansible

bug 1420538. Allow users to set supplementalGroup for Cassandra

Merge pull request #3320 from jcantrill/bz_1420538_metrics_supplemental_groups

bug 1420538. Allow users to set supplementalGroup for Cassandra

Comment 13 Peng Li 2017-03-30 09:55:27 UTC


1. specify 'openshift_metrics_cassandra_storage_group' in the inventory
#cat ~/my_inventory

2. run the playbook on ansible node
# ansible-playbook  -v  -i  ~/my_inventory  playbooks/common/openshift-cluster/openshift_metrics.yml

3. check deploy succeed
# oc get pod -n oepnshift-infra
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-9j37p   1/1       Running   0          7m
hawkular-metrics-ztkd6       1/1       Running   0          7m
heapster-fqcx7               1/1       Running   0          7m

3. check the rc
# oc get rc/hawkular-cassandra-1 -o yaml
        - 65531

4. check the pod
# oc get pod hawkular-cassandra-1-9j37p -o yaml
    fsGroup: 1000000000
      level: s0:c1,c0
    - 65531

Version info:
# rpm -qa openshift-ansible*

Comment 18 errata-xmlrpc 2017-04-12 19:01:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.