Bug 1256893 - RFE: The Ceph Mon Network Should Be Configurable
RFE: The Ceph Mon Network Should Be Configurable
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 10.0 (Newton)
Assigned To: Hugh Brock
Shai Revivo
: FutureFeature
Depends On:
Blocks: 1335593 1356451
  Show dependency treegraph
Reported: 2015-08-25 12:43 EDT by Dan Sneddon
Modified: 2016-12-08 17:26 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-08-11 13:12:11 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Dan Sneddon 2015-08-25 12:43:01 EDT
Description of problem:
The storage management network is separate from the storage network, and it is used for Ceph back-end. The storage management network is attached to the controllers, and we do network validation of that network during deployment and then use that network for the Ceph Mons running on the Controllers. We have at least one customer who doesn't want to attach the storage management network to the controllers.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Deploy OSP 7

Actual results:
There is no way to use the Storage network for the Ceph Mons on the Controllers. If you change the CephClusterNetwork from storage_mgmt to storage, then it affects both the controllers and the ceph nodes.

Expected results:
There should be a CephMonNetwork entry in the ServiceNetMap that affects only the controllers. This would allow an installation where the storage management network was only on the controllers.

Additional info:
Comment 5 Mike Burns 2016-01-28 14:29:45 EST
Dan, can you clear up if this is something that changed in this?  Dell is reporting that the statements in the summary are incorrect.
Comment 6 Dan Sneddon 2016-01-28 14:34:37 EST
This feature has been implemented, I believe in OSP 7.1:

    CephClusterNetwork: storage_mgmt
    CephPublicNetwork: storage
Comment 8 Mike Burns 2016-04-07 16:47:27 EDT
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.
Comment 10 Karl Hastings 2016-07-14 20:54:04 EDT
Dan or Mike,

Could you offer a little clarification regarding Comment 6.  If the feature was added upstream in OpenStack 7.1 shouldn't we have inherited it for OSP8+ ?

Or was it not actually fixed?  Is the CephMonNetwork item still needed?
Comment 11 Dan Sneddon 2016-07-26 14:32:06 EDT
(In reply to Dan Sneddon from comment #6)
> This feature has been implemented, I believe in OSP 7.1:
> ServiceNetMap:
>   defaults:
>     [...]
>     CephClusterNetwork: storage_mgmt
>     CephPublicNetwork: storage

It turns out that these two parameters are insufficient for the following use case:

Ceph storage public network is Storage
Ceph clustering network is Storage Management
Storage Management network is not on the controllers
Ceph Mon monitoring is done through the Storage network

So we need an additional parameter for CephMonNetwork. This will require a 2-pronged approach, hitting both tripleo-heat-templates and the Puppet manifests that process the hieradata output and create the Ceph configurations. This should be specified in an upstream bug, which I will link to this ticket.
Comment 12 Bob Fournier 2016-07-27 18:28:05 EDT
In OSP8, ceph-mon (run on the controller) actually uses the CephPublicNetwork, not the CephClusterNetwork, so its not necessary to add a separate configurable CephMon network.  In this case the controllers do not need to be attached to the StorageMgmt network, just the Storage network that is used for the CephPublicNetwork.

In tripleo-heat-templates, the ceph public_addr is set to this hieredata:
puppet/controller.yaml: ceph::profile::params::public_addr: {get_input: ceph_public_ip}

ceph_public_ip comes from CephPublicNetwork:
puppet/controller.yaml: ceph_public_ip: {get_attr: [NetIpMap, net_ip_map, {get_param: [ServiceNetMap, CephPublicNetwork]}]}

This is then used on the controller to set the address that cep-mon is bound to:
/etc/puppet/modules/ceph/manifests/profile/mon.pp:    public_addr         => $ceph::profile::params::public_addr,

So it should not be necessary to make the CephMon network configurable and we can close this RFE.

Note You need to log in before you can comment on or make changes to this bug.