Bug 1302613 - Replicator does not delete handoff copies, thus leaving multiple copies in cluster and disk usage increases
Summary: Replicator does not delete handoff copies, thus leaving multiple copies in cl...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-swift
Version: 7.0 (Kilo)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: 7.0 (Kilo)
Assignee: Pete Zaitcev
QA Contact: Mike Abrams
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-01-28 09:33 UTC by Christian Schwede (cschwede)
Modified: 2019-10-10 11:01 UTC (History)
7 users (show)

Fixed In Version: openstack-swift-2.3.0-5.el7ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-05 19:15:43 UTC
Target Upstream Version:
cschwede: needinfo+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1503152 0 None None None 2016-01-28 09:33:19 UTC
OpenStack gerrit 232696 0 'None' 'MERGED' 'Fix replicator intersection exception when sync data to remote regions.' 2019-11-18 01:26:19 UTC
Red Hat Product Errata RHBA-2016:2028 0 normal SHIPPED_LIVE openstack-swift bug fix advisory 2016-10-05 23:11:52 UTC

Description Christian Schwede (cschwede) 2016-01-28 09:33:19 UTC
Description of problem:

The swift object replicator does not delete a handoff copy that is successfully replicated to its primary nodes. Thus data is never deleted when a rebalance changes the cluster layout; for example when adding new disks data is replicated to the new disks, but not removed from the existing ones. Therefore a rebalance will actually not rebalance the data distribution. In fact total data usage will increase without adding new data.

Most likely happens with geo-replicated clusters.

Version-Release number of selected component (if applicable):

2.3
2.5

openstack-swift-object-2.3.0-2.el7ost.noarch

How reproducible:

Always.

Steps to Reproduce:
1. Create a Swift cluster with multiple regions
2. Upload data
3. Add or remove a node or disk from cluster and rebalance
4. Data is left on the old disks and not removed, even if it is a handoff node.

Actual results:

Data is NOT removed from a handoff node after successful replication.

Expected results:

Data is removed from a handoff node after successful replication.

Additional info:

Fixed upstream: https://github.com/openstack/swift/commit/d01cd425094c2e56e4e89dbf3eaf887815dd5b62

Also affects Swift 2.5 (as stated in the linked Launchpad bug entry).

Error message from object-replicator:

Jan 28 11:13:17 ******* object-replicator: Error syncing handoff partition: #012Traceback (most recent call last):#012  File "/usr/lib/python2.7/site-packages/swift/obj/replicator.py", line 269, in update_deleted#012    delete_objs = delete_objs.intersection(cand_objs)#012AttributeError: 'list' object has no attribute 'intersection'

Comment 4 Christian Schwede (cschwede) 2016-02-02 07:46:58 UTC
This has been also fixed upstream in stable/kilo: https://review.openstack.org/#/c/232696/

Comment 12 errata-xmlrpc 2016-10-05 19:15:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2028.html


Note You need to log in before you can comment on or make changes to this bug.