Bug 1351620

Summary:	[RHEL-7] Unlink() is called on non-existent path for DELETE requests
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	Prashanth Pai <ppai>
Component:	gluster-swift	Assignee:	Prashanth Pai <ppai>
Status:	CLOSED ERRATA	QA Contact:	surabhi <sbhaloth>
Severity:	low	Docs Contact:
Priority:	low
Version:	rhgs-3.1	CC:	amukherj, rcyriac, rhinduja, rhs-bugs, thiago
Target Milestone:	Development Freeze
Target Release:	RHGS 3.2.0
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	swiftonfile-2.3.0-4.el7rhgs	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1378422 (view as bug list)		Environment:
Last Closed:	2017-03-23 05:13:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1351522, 1378422

Description Prashanth Pai 2016-06-30 12:45:15 UTC

Description of problem:
When client sends DELETE request for an object which does not exist, gluster-swift internally calls unlink() despite determining earlier that the file does not exist.

Version-Release number of selected component (if applicable):
RHGS 3.1.3
swiftonfile-2.3.0-3.el7rhgs


How reproducible:
Send a DELETE request on a non-existent object.


Steps to Reproduce:
1. Create a container
2. Send DELETE for a non-existing object

curl -i http://localhost:8080/v1/AUTH_test/c1/blah_blah -X DELETE
HTTP/1.1 404 Not Found
Content-Length: 70
Content-Type: text/html; charset=UTF-8
X-Trans-Id: txb18a11352229424c9a7eb-005775136a
Date: Thu, 30 Jun 2016 12:41:14 GMT

<html><h1>Not Found</h1><p>The resource could not be found.</p></html>

Before sending DELETE, attach strace to object-server process and monitor syscalls.

root# ps aux | grep object-server
root      1398  0.7  1.2 266960 26268 ?        Ss   18:10   0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
root      1427  0.0  1.1 1413920 22756 ?       Sl   18:10   0:00 /usr/bin/python /usr/bin/swift-object-server /etc/swift/object-server.conf
root      1510  0.0  0.1 112672  2100 pts/1    S+   18:10   0:00 grep --color=auto object-server

root# strace -ff -p 1398,1427
Process 1398 attached
Process 1427 attached with 21 threads

[pid  1427] getxattr("/mnt/gluster-object/test/c1/f1", "user.swift.metadata", 0x0, 0) = -1 ENOENT (No such file or directory)
[pid  1427] getxattr("/mnt/gluster-object/test/c1/f1", "user.swift.metadata", 0x0, 0) = -1 ENOENT (No such file or directory)
[pid  1427] unlink("/mnt/gluster-object/test/c1/f1") = -1 ENOENT (No such file or directory)


Actual results:
unlink() is called on the non-existent path.


Expected results:
unlink() should not be called non-existent path.

Comment 2 Prashanth Pai 2016-07-28 13:27:54 UTC

Upstream patch: http://review.gluster.org/#/c/14501/

Comment 4 Prashanth Pai 2016-09-13 07:43:28 UTC

Upstream change has been merged.

Comment 5 Prashanth Pai 2016-09-19 10:56:12 UTC

Downstream changes have been submitted and merged.

rhel7 - https://code.engineering.redhat.com/gerrit/#/c/85036/
rhel6 - https://code.engineering.redhat.com/gerrit/#/c/85055/

Comment 8 surabhi 2016-12-15 06:48:08 UTC

Verified When a DELETE is sent on a non-existent object , there was a unlink call which gluster.swift was making.
With the fix, when a DELETE is sent on a non-existent object , gluster swift doesn't call unlink.Tried with deleting multiple objects (10000)

curl -i httv1/AUTH_test/dir1/fake -X DELETE
strace -ff -p 11860,11883

[pid 11883] poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}, {fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 2, 60000) = 1 ([{fd=4, revents=POLLIN}])
[pid 11883] accept(4, {sa_family=AF_INET, sin_port=htons(35514), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5
[pid 11883] fcntl(5, F_GETFL)           = 0x2 (flags O_RDWR)
[pid 11883] fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 11883] fcntl(5, F_GETFL)           = 0x802 (flags O_RDWR|O_NONBLOCK)
[pid 11883] fcntl(5, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid 11883] accept(4, 0x7ffda869aa90, [16]) = -1 EAGAIN (Resource temporarily unavailable)
[pid 11883] recvfrom(5, "DELETE /test/0/AUTH_test/dir1/fa"..., 65536, 0, NULL, NULL) = 422
[pid 11883] getsockname(5, {sa_family=AF_INET, sin_port=htons(6010), sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
[pid 11883] lstat("/mnt/gluster-object/test", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 11883] lstat("/mnt/gluster-object/test/..", {st_mode=S_IFDIR|0755, st_size=63, ...}) = 0
[pid 11883] getxattr("/mnt/gluster-object/test/dir1/fake", "user.swift.metadata", 0x0, 0) = -1 ENOENT (No such file or directory)
[pid 11883] sendto(5, "HTTP/1.1 404 Not Found\r\nContent-"..., 251, 0, NULL, 0) = 251
[pid 11883] shutdown(5, SHUT_RDWR)      = 0
[pid 11883] close(5)                    = 0
[pid 11883] poll([{fd=12, events=POLLIN|POLLPRI|POLLERR|POLLHUP}, {fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 2, 60000


Marking the BZ verified.

Comment 10 errata-xmlrpc 2017-03-23 05:13:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0488.html