Bug 1297060

Summary: Scaling out with an additional Swift node fails
Product: Red Hat OpenStack Reporter: Marius Cornea <mcornea>
Component: rhosp-directorAssignee: Dougal Matthews <dmatthew>
Status: CLOSED CURRENTRELEASE QA Contact: Marius Cornea <mcornea>
Severity: high Docs Contact:
Priority: urgent    
Version: 8.0 (Liberty)CC: achernet, hbrock, jcoufal, jslagle, mburns, rhel-osp-director-maint
Target Milestone: gaKeywords: TestOnly, Triaged
Target Release: 8.0 (Liberty)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-0.8.13-1.el7ost Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-20 11:22:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
swift scale out error none

Description Marius Cornea 2016-01-08 22:14:58 UTC
Created attachment 1113030 [details]
swift scale out error

Description of problem:
Adding an additional Swift node to an existing deployment fails with:
No partitions could be reassigned.
Either none need to be or none can be due to min_part_hours [1].

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-0.8.7-2.el7ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. Deploy overcloud with 3 x Swift nodes:
openstack overcloud deploy --templates ~/templates/my-overcloud  \
    --control-scale 1 --compute-scale 1 --swift-storage-scale 3 \
    -e ~/templates/my-overcloud/environments/network-isolation.yaml \
    -e ~/templates/network-environment.yaml \
    -e ~/templates/swift.yaml

2. Run the deploy command with an additional swift node:
openstack overcloud deploy --templates ~/templates/my-overcloud  \
    --control-scale 1 --compute-scale 1 --swift-storage-scale 4 \
    -e ~/templates/my-overcloud/environments/network-isolation.yaml \
    -e ~/templates/network-environment.yaml \
    -e ~/templates/swift.yaml

Actual results:
Deployment fails:
Deploying templates in the directory /home/stack/templates/my-overcloud
Stack failed with status: resources.ObjectStorageNodesPostDeployment: resources.StorageRingbuilderDeployment_Step2: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6
Heat Stack update failed.


Expected results:
Deployment succeeds.

Additional info:
Attaching the puppet error.

Comment 2 Dougal Matthews 2016-03-07 13:33:27 UTC
There is no support for scaling Swift in t-h-t yet. This is to do with complex requirements and Swift's ring-builder.

Comment 3 James Slagle 2016-03-15 18:22:23 UTC
dougal, going to assign this one back to you. i think we can call it "fixed" once the patch to optionally disable the ring build is landed, and the docs patch is landed.

Comment 9 Marius Cornea 2016-04-19 17:21:33 UTC
I'm going to test the patch attached to the BZ which disables the ring build by Director. Swift nodes scale out is going to be supported with manual ring management(outside Director).

After passing the following environment file:

parameter_defaults:
  SwiftRingBuild: False
  RingBuild: False

We get the following contens in /etc/swift:

[root@overcloud-controller-0 ~]# ls  /etc/swift/
account-server  account-server.conf  container-reconciler.conf  container-server  container-server.conf  object-expirer.conf  object-server  object-server.conf  proxy-server  proxy-server.conf  swift.conf


[heat-admin@overcloud-objectstorage-0 ~]$ ls  /etc/swift/
account-server  account-server.conf  container-reconciler.conf  container-server  container-server.conf  object-expirer.conf  object-server  object-server.conf  proxy-server  proxy-server.conf  swift.conf


[heat-admin@overcloud-objectstorage-1 ~]$ ls  /etc/swift/
account-server  account-server.conf  container-reconciler.conf  container-server  container-server.conf  object-expirer.conf  object-server  object-server.conf  proxy-server  proxy-server.conf  swift.conf

[heat-admin@overcloud-objectstorage-2 ~]$ ls /etc/swift/
account-server  account-server.conf  container-reconciler.conf  container-server  container-server.conf  object-expirer.conf  object-server  object-server.conf  proxy-server  proxy-server.conf  swift.conf