Bug 1311657 - [Docs] "Configuring Ceph with Custom Config Settings"
[Docs] "Configuring Ceph with Custom Config Settings"
Status: CLOSED CURRENTRELEASE
Product: Red Hat OpenStack
Classification: Red Hat
Component: documentation (Show other bugs)
8.0 (Liberty)
All Linux
high Severity high
: ---
: 9.0 (Mitaka)
Assigned To: Dan Macpherson
RHOS Documentation Team
: Documentation, FutureFeature, ZStream
: 1319328 (view as bug list)
Depends On:
Blocks: 1261979 1335593
  Show dependency treegraph
 
Reported: 2016-02-24 12:00 EST by Alan Bishop
Modified: 2017-03-13 11:37 EDT (History)
21 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Add support for configuring more Ceph parameters
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-03-13 09:55:37 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Alan Bishop 2016-02-24 12:00:49 EST
Description of problem:

The TripleO heat templates for Ceph only support a limited number of parameters for inclusion in the ceph.conf file. The following parameters need to be supported in order to properly manage and tune the Ceph cluster:

  max open files
  osd op threads
  osd mkfs options
  osd mount options
  filestore max sync interval
  filestore xattr use omap

Ideally, the heat template syntax would support adding any parameter to the ceph.conf file. That way the heat template itself wouldn't need to change, but additional entries could be added to the ceph.conf file by adding the data to the ceph.yaml hieradata file.

Expected results:

Additional parameters can be added to ceph.conf
Comment 2 Mike Burns 2016-04-07 17:11:06 EDT
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 4 Giulio Fidente 2016-04-29 17:11:33 EDT
It should be possible to provide arbitary config params for ceph by passing the following via environment file:

parameter_defaults:
  ExtraConfig:
    ceph_classes:
      - ceph::confg
    ceph::config:
      DEFAULT/my_ceph_parameter:
        value: my_value
      DEFAULT/another_parameter:
        value: another_value

which should result in ceph.conf being populated with something like:

[DEFAULT]
my_ceph_parameter: my_value
another_parameter: another_value
Comment 5 Alan Bishop 2016-05-10 13:21:31 EDT
This isn't working for me. Here is what I added to an environment file that is part of our solution:

parameter_defaults:
  ExtraConfig:
    ceph_classes:
      - ceph::conf
    ceph::conf:
      global/max_open_files:
        value: 131072

Note: Deployments failed until I determined the proper ceph_classes name is "ceph::conf" (not "ceph::confg" or "ceph::config").

The overcloud deployment succeeds, but no additional entry appears in ceph.conf. The hiera data appears to be correct:

[heat-admin@overcloud-cephstorage-0 ~]$ sudo hiera ceph_classes
["ceph::conf"]
[heat-admin@overcloud-cephstorage-0 ~]$ sudo hiera ceph::conf
{"global/max_open_files"=>{"value"=>131072}}

I expected to find a "max_open_files" entry in the [global] section of ceph.conf, but there isn't one.
Comment 6 Giulio Fidente 2016-05-20 00:43:22 EDT
hi Alan, please excuse me but the syntax I suggested in comment #4 is indeed wrong; if I remember correctly we wrote it down together during a session at the summit but I didn't test it. My bad.

The correct syntax is:

parameter_defaults:
  ExtraConfig:
    ceph_classes:
      - ceph::conf
    ceph::conf::args:
      DEFAULT/my_ceph_parameter:
        value: my_value
      DEFAULT/another_parameter:
        value: another_value

which in your use case ( comment #6 ) would become:

parameter_defaults:
  ExtraConfig:
    ceph_classes:
      - ceph::conf
    ceph::conf:args:
      global/max_open_files:
        value: 131072

I tested it this time and it works as intended.

As a side note, with OSP8 templates we also don't need to pass ::conf as an additional class (in ceph_classes) because it's included by default so one could remove:

    ceph_classes:
      - ceph::conf
Comment 7 Giulio Fidente 2016-05-20 01:49:49 EDT
(In reply to Alan Bishop from comment #5)
> This isn't working for me. Here is what I added to an environment file that
> is part of our solution:
> 
> parameter_defaults:
>   ExtraConfig:
>     ceph_classes:
>       - ceph::conf
>     ceph::conf:
>       global/max_open_files:
>         value: 131072

this should be:

parameter_defaults:
  ExtraConfig:
    ceph_classes:
      - ceph::conf
    ceph::conf::args:
      global/max_open_files:
        value: 131072
Comment 8 Alan Bishop 2016-05-24 10:13:17 EDT
Giulio, I confirmed the corrected syntax you provided works as you describe. Thanks!

On one hand, this RFE is not needed because the capability already exists. On the other hand, I have not seen any documentation that explains how to use the capability.

I recommend adding a write-up to an appropriate section in the Red Hat OpenStack Platform Partner Integration guide (https://access.redhat.com/documentation/en/red-hat-openstack-platform/8/partner-integration/partner-integration).
Comment 9 Giulio Fidente 2016-05-31 12:10:15 EDT
Alan, I am adding some docs in tripleo.org to document this feature [1].

Do you think we can move the BZ to docs and do something similar with the official RH OSP documentation?

1. https://review.openstack.org/#/c/323464/
Comment 10 Alan Bishop 2016-05-31 13:39:42 EDT
That sounds like a good idea to me.
Comment 12 Karl Hastings 2016-07-14 20:22:46 EDT
Since the feature exists, and this is now a docs bug, I'm changing the flags back to rhos-9.0? to match the Target Release "9.0 (Mitaka)"
Comment 13 Sean Cohen 2016-08-16 12:56:57 EDT
*** Bug 1319328 has been marked as a duplicate of this bug. ***
Comment 14 Dan Macpherson 2017-03-03 00:12:45 EST
It looks like this BZ fell through the cracks.

There's a page for this content here:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/9/html/red_hat_ceph_storage_for_the_overcloud/creation#Configuring_Ceph_Storage_Cluster_Settings

Same content appears on OSP10 and OSP11 branches.

@Giulio, is there anything further we need for this content?
Comment 15 Giulio Fidente 2017-03-13 09:29:33 EDT
thanks Dan, I think the example in the link you posted is fine, Alan do you think it is sufficient? Maybe we can close this as CURRENTRELEASE?
Comment 16 Alan Bishop 2017-03-13 09:55:37 EDT
I reviewed the documentation at the link Dan provided, and feel it does a very good job of explaining things. I agree this BZ can be closed.
Comment 17 Dan Macpherson 2017-03-13 11:37:16 EDT
Thanks, guys!

Note You need to log in before you can comment on or make changes to this bug.