Bug 1437499

Summary:	TripleO/Director deployment fails if Swift rings rebalanced within within min_part_hours
Product:	Red Hat OpenStack	Reporter:	Lukas Bezdicka <lbezdick>
Component:	puppet-swift	Assignee:	Christian Schwede (cschwede) <cschwede>
Status:	CLOSED ERRATA	QA Contact:	Mike Abrams <mabrams>
Severity:	unspecified	Docs Contact:
Priority:	urgent
Version:	11.0 (Ocata)	CC:	cschwede, derekh, dmacpher, jjoyce, jschluet, michele, pmyers, slinaber, srevivo, tvignaud, yprokule, zaitcev
Target Milestone:	rc
Target Release:	11.0 (Ocata)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	puppet-swift-10.3.0-2.el7ost	Doc Type:	No Doc Update
Doc Text:	undefined	Story Points:	---
Clone Of:
Clones:	1465731 (view as bug list)		Environment:
Last Closed:	2017-05-17 20:16:33 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1443090
Bug Blocks:	1394025, 1465731

Description Lukas Bezdicka 2017-03-30 12:15:19 UTC

Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1e:%PORT%/d1]/Ring_object_device[[fd00:fd00:fd00:4000::1e]:6000/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1e:%PORT%/d1]/Ring_container_device[[fd00:fd00:fd00:4000::1e]:6001/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1e:%PORT%/d1]/Ring_account_device[[fd00:fd00:fd00:4000::1e]:6002/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1b:%PORT%/d1]/Ring_object_device[[fd00:fd00:fd00:4000::1b]:6000/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1b:%PORT%/d1]/Ring_container_device[[fd00:fd00:fd00:4000::1b]:6001/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Tripleo::Profile::Base::Swift::Add_devices[r1z1-fd00:fd00:fd00:4000::1b:%PORT%/d1]/Ring_account_device[[fd00:fd00:fd00:4000::1b]:6002/d1]/ensure: created
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]/returns: Reassigned 1024 (100.00%) partitions. Balance is now 16.67.  Dispersion is now 0.00
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]/returns: NOTE: Balance of 16.67 indicates you should push this 
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]/returns:       ring, wait at least 1 hours, and rebalance/repush.
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]/returns: Reassigned 1024 (100.00%) partitions. Balance is now 16.67.  Dispersion is now 0.00
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]/returns: NOTE: Balance of 16.67 indicates you should push this 
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]/returns:       ring, wait at least 1 hours, and rebalance/repush.
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]/returns: Reassigned 1024 (100.00%) partitions. Balance is now 16.67.  Dispersion is now 0.00
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]/returns: -------------------------------------------------------------------------------
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]/returns: NOTE: Balance of 16.67 indicates you should push this 
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]/returns:       ring, wait at least 1 hours, and rebalance/repush.
Notice: /Stage[main]/Tripleo::Profile::Base::Swift::Ringbuilder/Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]/returns: -------------------------------------------------------------------------------

Comment 1 Christian Schwede (cschwede) 2017-03-30 12:28:10 UTC

Could you share the /etc/swift/*.builder files please? That would be very helpful to check why there is an inbalance.

On the error itself: the problem is that puppet-swift fails, but swift-ring-builder just warns - it should continue. Will propose a patch.

Comment 2 Christian Schwede (cschwede) 2017-03-30 12:40:09 UTC

Proposed upstream patch: https://review.openstack.org/451793

And upstream bugreport: https://bugs.launchpad.net/tripleo/+bug/1677588

Comment 3 Christian Schwede (cschwede) 2017-04-02 16:17:37 UTC

Backported patch merged: https://review.openstack.org/#/c/452535/
Changing state to POST.

Comment 4 Jon Schlueter 2017-04-03 15:31:57 UTC

dropping upstream master since stable/ocata external tracker is merged

Comment 6 Christian Schwede (cschwede) 2017-04-10 13:30:39 UTC

*** Bug 1385483 has been marked as a duplicate of this bug. ***

Comment 18 errata-xmlrpc 2017-05-17 20:16:33 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245