Bug 1418335 - [RFE] Easy heterogeneous cluster management
Summary: [RFE] Easy heterogeneous cluster management
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 12.0 (Pike)
Hardware: x86_64
OS: Linux
Target Milestone: Upstream M2
: 12.0 (Pike)
Assignee: Saravanan KR
QA Contact: Yariv
: 1404208 1465547 (view as bug list)
Depends On:
Blocks: 1458798 1465537 1389435 1442136 1463145 1469642 1550658
TreeView+ depends on / blocked
Reported: 2017-02-01 15:12 UTC by Franck Baudin
Modified: 2018-03-01 17:47 UTC (History)
16 users (show)

Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170616123155.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1463145 1550658 (view as bug list)
Last Closed: 2017-12-13 21:06:26 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
OpenStack gerrit 437956 0 'None' MERGED Add role specific information to the service template 2020-07-17 14:25:49 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Franck Baudin 2017-02-01 15:12:11 UTC
Description of problem:
Not all servers are identical: some have more RAM than other, some have more CPUs than others. NFV configuration however requires very specific parameters to be set, like core list to run OVS-DPDK. This list will obviously vary if not all the compute nodes are identical. So, for a given role, we would like to be able to override the template values for a given set of machines to match their HW characteristics.


How do we set a parameter, specific to a role? Like, I want to specific NeutronDpdkSocketMemory for Compute Role 1 as "2014,2014" and Compute Role 2 as "0,2048".
This is possible to configure parameter per role basis, with a workaround, which is explained below.
Creating Custom Role:
The document OVS DPDK Role with RHOSP10 (composable roles)  gives the steps involved in creating custom role for deploying OVS-DPDK as new role ComputeOvsDpdk. Additionally, if we experimented configuring a parameter only specific to this role. The same parameter is defined globally but it is overridden for this new role to validate that role-specific parameters are achievable.
Configuring Parameter Specific to Custom Role:
Below addition has been done in the environment file to achieve role specific parameter update.
  # Existing paramter (applied for all nodes)
  NeutronDpdkSocketMemory: "'2048,2048'"

  # Role Specific Parameter (overriden)
    vswitch::dpdk::socket_memory: "'0,2048'"

Expected results:
Ideally, it should be possible to provide THT parameter per role to reflect the differentiation as needed, which is NOT possible with RHOSP10.
Instead, the overriding can be done with "hiera" data, for that role.
In above case, the actual parameter name is NeutronDpdkSocketMemory but the overriding has been done to its corresponding hiera value which is vswitch::dpdk::socket_memory. The assignment to this parameter happens at tripleo-heat-templates/neutron-ovs-dpdk-agent.yaml at stable/newton · openstack/tripleo-heat-templates · GitHub

Comment 1 Franck Baudin 2017-02-01 15:15:01 UTC
Related also to overcloud image customization per set of servers: https://review.openstack.org/#/c/423304/

Comment 2 Vijay Chundury 2017-02-03 13:39:49 UTC
With OSP-10 it is possible to set node-specific hiera-data is possible.
To be specific, with a heat environment file we can customise the "settingss" for a specific node regardless of the resource group.
It is possible to customize multiple and different keys for multiple nodes (UUID mapping).

The catch is the operator has to run introspection for the specific node (capture the UUID) and then create a template, then using extraconfig he can key in a value specific value to that target node. 
This should be possible with OSP-10.

Sanjay can you please test this scenario and update us please.

Comment 3 Sanjay Upadhyay 2017-02-21 08:43:17 UTC
Created a blog at https://mojo.redhat.com/groups/openstack-community-of-practice/blog/2017/02/10/tripleo-composables-roles-and-passing-parameters-per-role, which shows how we can pass specific *Custom Role* specific parameters -

#Very specific a compute Role specific parameter is sent via ExtraConfigs and needs to be a hiera data format 
    vswitch::dpdk::memory_channels: "'0,2048'"

^^ above ComputeOvsDPDK is a custom role, and we are passing the parameters for memory channels.

Comment 4 Sanjay Upadhyay 2017-02-21 09:04:18 UTC
*** Bug 1404208 has been marked as a duplicate of this bug. ***

Comment 5 Saravanan KR 2017-03-15 05:56:42 UTC
This bug is wrongly closed. The requirement of this bug is to provide easily configurable parameters per role. Currently we are able to provide role specific parameters using hiera variables, which is not correct approach for operators as they need to know 2 names for the same variable. 

We are working on role-specific parameters for each service. This has been discussed in PTG, please refer etherpad link below for more details.


Comment 7 Saravanan KR 2017-05-25 04:42:28 UTC
Then service plugin interface change has been merged. For our usecase, we need to modify following templates:
1) neutron-ovs-dpdk-agent: Done (https://review.openstack.org/#/c/446093/)
2) nova-compute: In progress (https://review.openstack.org/#/c/465481/)
3) PreNetworkConfig: In progress (https://review.openstack.org/#/c/447051/)
4) neutron-sriov-agent: Not Started

All SR-IOV and DPDK parameters will be covered with this.

Comment 8 Saravanan KR 2017-06-16 07:51:55 UTC
All the DPDK and SR-IOV parameters can be role-specific - Documentation @https://docs.openstack.org/developer/tripleo-docs/advanced_deployment/role_specific_parameters.html

Comment 10 Ollie Walsh 2017-07-26 01:56:50 UTC
*** Bug 1465547 has been marked as a duplicate of this bug. ***

Comment 11 Ziv Greenberg 2017-10-23 11:38:02 UTC
Heterogeneous cloud features have been verified.

Comment 14 errata-xmlrpc 2017-12-13 21:06:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.