Bug 1469435 - [RFE][Swift] - Implement Swift ring rebalance as an Ansible-based Mistral workflow [NEEDINFO]
[RFE][Swift] - Implement Swift ring rebalance as an Ansible-based Mistral wor...
Status: VERIFIED
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common (Show other bugs)
13.0 (Queens)
Unspecified Unspecified
unspecified Severity unspecified
: Upstream M1
: 13.0 (Queens)
Assigned To: Christian Schwede (cschwede)
Mike Abrams
Kim Nylander
: FutureFeature, Triaged
Depends On:
Blocks: 1469441 1469452
  Show dependency treegraph
 
Reported: 2017-07-11 05:26 EDT by Christian Schwede (cschwede)
Modified: 2018-06-19 22:53 EDT (History)
7 users (show)

See Also:
Fixed In Version: openstack-tripleo-common-8.6.1-3.el7ost
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
knylande: needinfo? (cschwede)


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1759311 None None None 2018-03-27 11:54 EDT
OpenStack gerrit 510884 None master: MERGED tripleo-common: Add Ansible playbook/workflow to rebalance Swift rings (I7bb1d7d4f45bee36df1c435a11bda5d4cdaae896) 2018-04-24 17:55 EDT
OpenStack gerrit 556928 None master: MERGED tripleo-common: Fix missing permissions on Swift rebalance playbook (Ia766bc44a647fec15ff662f1ef9ffb67860b155b) 2018-04-24 17:55 EDT
OpenStack gerrit 560337 None stable/queens: NEW tripleo-common: Fix parameter indentation on Swift rebalance playbook (I31db903787feded6acecd67dc98ef10b6bf26ea8) 2018-04-24 17:56 EDT

  None (edit)
Description Christian Schwede (cschwede) 2017-07-11 05:26:51 EDT
As an operator, I want to trigger a Swift ring rebalance without executing a full overcloud update. 

Today rebalances are only executed if there is a change in the devices (IP address, device added etc). It is executed during an overcloud update. This is a huge overhead if one only wants to rebalance the rings. To make things worse, rebalances might need to be executed multiple times - there should be at least an option to automate this.

Some recent changes in Tripleo/Mistral make it possible to execute Ansible playbooks within a workflow. Implementing the rebalance as a workflow has multiple benefits:

1. It can be executed by an operator as a standalone operation on the undercloud
2. It can be executed automatically by Mistral multiple times (eg when there is a bigger rebalance required)

The workflow itself could be nearly the same as today:

1. Download most recent rings from the undercloud
2. Rebalance
3. Upload updated rings to the overcloud

It could be also done in a more lightweight manner, for example the above workflow is executed only on a single node, and once the updated rings are available they will be fetched by all nodes.

There should be an additional sanity check that uses the swift-dispersion-report tool to check if the dispersion already reached a given level (for example 95%) and also ensure that at least one replication pass finished since the last rebalance (using swift-recon data).

Expected outcome
----------------
Operator can trigger a Mistral workflow that rebalances Swift rings in a safe manner and distributes them. 

Work items
----------
1. Add option to swift-ring-builder to limit rebalance operations to a maximum of X percent
2. Make swift-dispersion-report importable
3. Write Ansible playbook & library to execute descibed workflow
Comment 2 Christian Schwede (cschwede) 2017-12-21 12:32:34 EST
Upstream patch merged, moving to POST.

Note You need to log in before you can comment on or make changes to this bug.