Bug 1039569 - [Functional] Object Expirer doesn't clean up objects yet it accepts headers X-Delete-After & X-Delete-At
Summary: [Functional] Object Expirer doesn't clean up objects yet it accepts headers X...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: gluster-swift
Version: 2.1
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: RHGS 3.0.0
Assignee: Luis Pabón
QA Contact: Bhaskar Bandari
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-12-09 13:48 UTC by pushpesh sharma
Modified: 2016-11-08 22:25 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, headers X-Delete-At and X-Delete-After were accepted although object expiration feature was not fully implemented, thus leading to confusion. With this fix, the X-Delete-At and X-Delete-After headers are not accepted.
Clone Of:
Environment:
Last Closed: 2014-09-22 19:30:09 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2014:1278 0 normal SHIPPED_LIVE Red Hat Storage Server 3.0 bug fix and enhancement update 2014-09-22 23:26:55 UTC

Description pushpesh sharma 2013-12-09 13:48:36 UTC
Description of problem:

A 404 is returned for a GET on an expired object. As "object-expirer" daemon is not run in gluster-swift environment, the files however are not deleted from gluster volume.

Version-Release number of selected component (if applicable):
RHS2.1-GA & RHS2.1-U1

How reproducible:


Steps to Reproduce:
Few study updates: 
Gluster-Swift does not support object expiration yet.

Object expiration is a two fold task.
1) Store the expiration time to the object metadata. Return 404 when such time elapsed.
2) special service "object-expirer" should run to delete such objects and their metadata <wherever stored>.

Gluster-Swift currently does not store X-Delete-At or X-Delete-After header in it's xattr. It has no code to perform cross-checks with current epoch and header's mentioned epoch to return 404 if it has elapsed. This is verified and failed with latest Gluster-Swift code. So this #1 can be implemented in Gluster-Swift with some code changes. Needs more investigation.

2) Interesting thing is in plain swift environment, Openstack Swift stores expiration related metadata of those objects in a special account. SWAuth could be useful in here when we maintain multiple accounts per volume.

Openstack Swift's object expirer does walk across such metadata and removes those objects internally.

Current account = volume would not work in there. It would need multiple accounts per volume which SWAuth can privide. Just a suggestion. Need to investigate on this further.


The "object-expirer" daemon, as of now, depends on a separate special account that keeps track of objects to be deleted. As accounts=volume in gluster-swift, we may have to wait till multiple accounts per volume are implemented OR we could investigate overriding some parts of object-expirer if we want the feature sooner.

Actual results:


Expected results:


Additional info:

Comment 2 Luis Pabón 2013-12-12 03:47:12 UTC
New fix does not allow X-Delete-After or X-Delete-At headers to be used in POST and PUT commands.  The fix has been merged into the havana branch:

http://review.gluster.org/#/c/6444/

Comment 3 Luis Pabón 2013-12-12 03:48:04 UTC
Upstream RPMs available here:
https://launchpad.net/gluster-swift/havana/1.10.0-1

Comment 4 pushpesh sharma 2014-06-06 06:48:07 UTC
]# curl -i -v -X PUT -T anaconda-ks.cfg -H 'X-Delete-After: 20' http://10.65.202.205:8080/v1/AUTH_test/cont/obj-exp 
* About to connect() to 10.65.202.205 port 8080 (#0)
*   Trying 10.65.202.205... connected
* Connected to 10.65.202.205 (10.65.202.205) port 8080 (#0)
> PUT /v1/AUTH_test/cont/obj-exp HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 10.65.202.205:8080
> Accept: */*
> X-Delete-After: 20
> Content-Length: 1190
> Expect: 100-continue
> 
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< Content-Length: 52
Content-Length: 52
< Content-Type: text/plain
Content-Type: text/plain
< X-Trans-Id: tx7623e5c3b3b84071a6178-005391b417
X-Trans-Id: tx7623e5c3b3b84071a6178-005391b417
< Date: Fri, 06 Jun 2014 12:29:11 GMT
Date: Fri, 06 Jun 2014 12:29:11 GMT

< 
* Connection #0 to host 10.65.202.205 left intact
* Closing connection #0
x-delete-at,x-delete-after headers are not supported[root@RHSS2 ~]# 
[root@RHSS2 ~]# curl -i -v -X PUT -T anaconda-ks.cfg -H 'X-Delete-At: 87920' http://10.65.202.205:8080/v1/AUTH_test/cont/obj-exp 
* About to connect() to 10.65.202.205 port 8080 (#0)
*   Trying 10.65.202.205... connected
* Connected to 10.65.202.205 (10.65.202.205) port 8080 (#0)
> PUT /v1/AUTH_test/cont/obj-exp HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: 10.65.202.205:8080
> Accept: */*
> X-Delete-At: 87920
> Content-Length: 1190
> Expect: 100-continue
> 
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< Content-Length: 52
Content-Length: 52
< Content-Type: text/plain
Content-Type: text/plain
< X-Trans-Id: txb58a0e573ab249718a5fc-005391b432
X-Trans-Id: txb58a0e573ab249718a5fc-005391b432
< Date: Fri, 06 Jun 2014 12:29:38 GMT
Date: Fri, 06 Jun 2014 12:29:38 GMT

< 
* Connection #0 to host 10.65.202.205 left intact
* Closing connection #0
x-delete-at,x-delete-after headers are not supported[root@RHSS2 ~]# 
[root@RHSS2 ~]# 


Verified On:-
#rpm -qa|grep swift
openstack-swift-account-1.10.0-3.el6ost.noarch
openstack-swift-object-1.10.0-3.el6ost.noarch
swiftonfile-1.10.0-6.el6rhs.noarch
openstack-swift-plugin-swift3-1.0.0-0.20120711git.1.el6ost.noarch
openstack-swift-proxy-1.10.0-3.el6ost.noarch
python-swiftclient-1.8.0-1.el6ost.noarch
openstack-swift-container-1.10.0-3.el6ost.noarch
openstack-swift-1.10.0-3.el6ost.noarch

Comment 5 Prashanth Pai 2014-06-18 10:34:57 UTC
Updated doc text accordingly.

Comment 6 Pavithra 2014-07-23 10:06:56 UTC
Hi ppai,

Please review the doc text for technical accuracy and sign off.

Comment 7 Prashanth Pai 2014-07-23 10:19:26 UTC
The doc text meant slightly different from the actual bug fix. Have updated the same.

Comment 11 errata-xmlrpc 2014-09-22 19:30:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1278.html


Note You need to log in before you can comment on or make changes to this bug.