Bug 1314840 - Upgrade from OSP 7.0.3 to 7.0.4 breaks if there is no partitions for swift to rebalance.
Summary: Upgrade from OSP 7.0.3 to 7.0.4 breaks if there is no partitions for swift to...
Keywords:
Status: CLOSED DUPLICATE of bug 1321088
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: async
: 7.0 (Kilo)
Assignee: Angus Thomas
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-04 16:24 UTC by Jeremy
Modified: 2023-09-14 03:18 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-03 16:49:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jeremy 2016-03-04 16:24:35 UTC
Description of problem:

During an update test,  the process was interrupted and marked in failed status[1] when we look at the heat resource output we can see a rebalance was tried but it failed because it runs during the minimum number of hours before a partition can be reassigned[2]  see more  details in attached file [see attached file:  stack_update_heat_resource_output.txt]

Expect to see:
- update stack works without issues
- if rebalance cannot happen because it is not necessary or if the update runs during the minimum number of hours, the update process should be able to determine that and do not fail.


[1]
 [stack@716091-lab1-director ~]$ openstack overcloud update stack overcloud -i --templates ~/templates/my-overcloud -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e ~/templates/network-environment.yaml -e ~/templates/my-overcloud/environments/storage-environment.yaml 
starting package update on stack overcloud
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
FAILED
update finished with status FAILED


[2]
Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]: Failed to call refresh: swift-ring-builder /etc/swift/account.builder rebalance 999 returned 1 instead of one of [0]
Swift::Ringbuilder::Rebalance[account]/Exec[rebalance_account]: swift-ring-builder /etc/swift/account.builder rebalance 999 returned 1 instead of one of [0]
Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]: Failed to call refresh: swift-ring-builder /etc/swift/container.builder rebalance 999 returned 1 instead of one of [0]
Swift::Ringbuilder::Rebalance[container]/Exec[rebalance_container]: swift-ring-builder /etc/swift/container.builder rebalance 999 returned 1 instead of one of [0]
Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]: Failed to call refresh: swift-ring-builder /etc/swift/object.builder rebalance 999 returned 1 instead of one of [0]
Swift::Ringbuilder::Rebalance[object]/Exec[rebalance_object]: swift-ring-builder /etc/swift/object.builder rebalance 999 returned 1 instead of one of [0]



Version-Release number of selected component (if applicable):
openstack-swift-2.3.0-4.el7ost.noarch 

Steps to reproduce: 
1. Deploy overcloud using OSP 7.0.3 from a director running OSP-D 7.2
2. Update director to 7.3 
3. Run openstack overcloud update stack with respective templates in interactive mode

NOTE: The important part here is that we deployed the 7.2 cloud, and 20-30 mins later we were running the update, but if you have a 7.0.3 overcloud running for more than that, try to rebalance before trying to update to 7.0.4 you can see the same rebalance results and the update might fail. [see attached file swift_rebalance_manual_test.txt]


Actual results:
fail

Expected results:
success install

Additional info:

1. We initially had undercloud OSP-D 7.2 and deployed an overcloud with OSP 7.0.3 or in other words, 7.2 images,  that worked without issues
2. Then we updated the undercloud to (OSP-D 7.3) and registered the overcloud nodes to our Satellite network.
3. We updated the undercloud to OSP-D 7.3  per the documentation,  no issues
4 We finally ran the stack update to update the current overcloud (this way the overcloud running OSP 7.0.3 would be updated to OSP 7.0.4) , that's then the error happens.

It's important to mention that we ran other tests and the updates were successful, it is just when there is nothing to rebalance that we got a failure.

Comment 3 Pete Zaitcev 2016-03-10 20:03:43 UTC
If OSPd insists on running rebalances before the minimum number of
hours has expired, it needs to supply the "-f" flag.

Comment 7 Dave Maley 2016-05-03 16:49:30 UTC
customer is on osp7 and thus the fix provided in bug 1321088 will resolve this.

*** This bug has been marked as a duplicate of bug 1321088 ***

Comment 8 Red Hat Bugzilla 2023-09-14 03:18:58 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.