Bug 960999

Summary: G4S: Ranged GET requests failes to respond properly to some byte range requests
Product: [Community] Gluster-Swift Reporter: pushpesh sharma <psharma>
Component: Proxy-serverAssignee: Nobody <nobody>
Status: VERIFIED --- QA Contact: David J. Mac Donald <david.macdonald>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 0.1CC: bugs, david.macdonald
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description pushpesh sharma 2013-05-08 13:45:11 UTC
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:

1.) byte ranges works perfectly with some of the some ranges like :- 1-50 ,-50 
 
[root@dhcp207-5 swift]# curl -v -X GET -H 'X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963' -H 'Range: bytes=-50' -k https://10.65.207.5:443/v1/AUTH_test2/dir2/test1K.txt
* About to connect() to 10.65.207.5 port 443 (#0)
*   Trying 10.65.207.5...
* connected
* Connected to 10.65.207.5 (10.65.207.5) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 08 15:24:51 2013 GMT
* 	expire date: Jun 07 15:24:51 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> GET /v1/AUTH_test2/dir2/test1K.txt HTTP/1.1
> User-Agent: curl/7.27.0
> Host: 10.65.207.5
> Accept: */*
> X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963
> Range: bytes=-50
> 
< HTTP/1.1 206 Partial Content
< Last-Modified: Wed, 08 May 2013 18:25:12 GMT
< Content-Range: bytes 1164-1213/1214
< Etag: d55c7dbeb35f435f522c299ba91a60f9
< X-Timestamp: 1368037512.85567
< Accept-Ranges: bytes
< Content-Length: 50
< Content-Type: text/plain
< Date: Wed, 08 May 2013 18:56:21 GMT
< 
wift#memcache
memcache_servers = 127.0.0.1:11211

* Connection #0 to host 10.65.207.5 left intact
* Closing connection #0

2.) It failed to raise error for the following illegal request:-
As per :-
# RFC 2616 14.35.1
                # "If a syntactically valid byte-range-set includes ... at
                # least one suffix-byte-range-spec with a NON-ZERO
                # suffix-length, then the byte-range-set is satisfiable.
                # Otherwise, the byte-range-set is unsatisfiable.


[root@dhcp207-5 swift]# curl -v -X GET -H 'X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963' -H 'Range: bytes=-0' -k https://10.65.207.5:443/v1/AUTH_test2/dir2/test1K.txt
* About to connect() to 10.65.207.5 port 443 (#0)
*   Trying 10.65.207.5...
* connected
* Connected to 10.65.207.5 (10.65.207.5) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 08 15:24:51 2013 GMT
* 	expire date: Jun 07 15:24:51 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> GET /v1/AUTH_test2/dir2/test1K.txt HTTP/1.1
> User-Agent: curl/7.27.0
> Host: 10.65.207.5
> Accept: */*
> X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963
> Range: bytes=-0
> 
< HTTP/1.1 206 Partial Content
< Last-Modified: Wed, 08 May 2013 18:25:12 GMT
< Content-Range: bytes 0-1213/1214
< Etag: d55c7dbeb35f435f522c299ba91a60f9
< X-Timestamp: 1368037512.85567
< Accept-Ranges: bytes
< Content-Length: 1214
< Content-Type: text/plain
< Date: Wed, 08 May 2013 18:57:07 GMT
< 
[DEFAULT]
#bind_port = 8080
user = root
log_facility = LOG_LOCAL1
bind_port = 443
cert_file = /etc/swift/cert.crt
key_file = /etc/swift/cert.key
bind_ip = 10.65.207.5

[pipeline:main]
pipeline = healthcheck cache tempauth proxy-server

[app:proxy-server]
use = egg:gluster_swift_ufo#proxy
allow_account_management = true
account_autocreate = true

[filter:tempauth]
use = egg:swift#tempauth
# Here you need to add users explicitly. See the OpenStack Swift Deployment
# Guide for more information. The user and user64 directives take the
# following form:
#     user_<account>_<username> = <key> [group] [group] [...] [storage_url]
#     user64_<account_b64>_<username_b64> = <key> [group] [group] [...] [storage_url]
# Where you use user64 for accounts and/or usernames that include underscores.
#
# NOTE (and WARNING): The account name must match the device name specified
# when generating the account, container, and object build rings.
#
# E.g.
#     user_ufo0_admin = abc123 .admin
user_test_tester = testing .admin
user_test2_tester2 = testing2 .admin
user_test_tester3 = testing3

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = 127.0.0.1:11211

* Connection #0 to host 10.65.207.5 left intact
* Closing connection #0


3.) This should not raise error(416)
  # RFC 2616 14.35.1
        # "If the entity is shorter than the specified suffix-length, the
        # entire entity-body is used."

[root@dhcp207-5 swift]# curl -v -X GET -H 'X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963' -H 'Range: bytes=-1500' -k https://10.65.207.5:443/v1/AUTH_test2/dir2/test1K.txt
* About to connect() to 10.65.207.5 port 443 (#0)
*   Trying 10.65.207.5...
* connected
* Connected to 10.65.207.5 (10.65.207.5) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* 	subject: O=Default Company Ltd,L=Default City,C=XX
* 	start date: May 08 15:24:51 2013 GMT
* 	expire date: Jun 07 15:24:51 2013 GMT
* 	common name: (nil)
* 	issuer: O=Default Company Ltd,L=Default City,C=XX
> GET /v1/AUTH_test2/dir2/test1K.txt HTTP/1.1
> User-Agent: curl/7.27.0
> Host: 10.65.207.5
> Accept: */*
> X-Auth-Token: AUTH_tk0efc970d210444c09ccfa5209ef0f963
> Range: bytes=-1500
> 
< HTTP/1.1 416 Requested Range Not Satisfiable
< Content-Length: 44
< Content-Type: text/html; charset=UTF-8
< Date: Wed, 08 May 2013 19:11:38 GMT
< 
* Connection #0 to host 10.65.207.5 left intact
Requested range not satisfiable: bytes=-1500* Closing connection #0
 


  
Actual results:
Other byte ranges cases are perfecly working like:-

file_length+1000 - file_length+2000 ==> raises error Requested Range Not Satisfiable (416) 

file_length-1000 - file_length+2000 ==>Return data till end,Partial Content(206)

Range: 0-4 :- HTTP/1.1 206 Partial Content


Expected results:


Additional info:

Comment 1 pushpesh sharma 2013-05-08 13:47:31 UTC
Test Code:-


    def testRangedGets(self):
        file_length = 10000
        range_size = file_length / 10
        file = self.env.container.file(Utils.create_name())
        data = file.write_random(file_length)

        for i in range(0, file_length, range_size):
            range_string = 'bytes=%d-%d' % (i, i + range_size - 1)
            hdrs = {'Range': range_string}
            self.assert_(data[i: i + range_size] == file.read(hdrs=hdrs),
                         range_string)

            range_string = 'bytes=-%d' % (i)
            hdrs = {'Range': range_string}
            if i == 0:
                # RFC 2616 14.35.1
                # "If a syntactically valid byte-range-set includes ... at
                # least one suffix-byte-range-spec with a NON-ZERO
                # suffix-length, then the byte-range-set is satisfiable.
                # Otherwise, the byte-range-set is unsatisfiable.
                self.assertRaises(ResponseError, file.read, hdrs=hdrs)
                self.assert_status(416)
            else:
                self.assertEquals(file.read(hdrs=hdrs), data[-i:])

            range_string = 'bytes=%d-' % (i)
            hdrs = {'Range': range_string}
            self.assert_(file.read(hdrs=hdrs) == data[i - file_length:],
                         range_string)

        range_string = 'bytes=%d-%d' % (file_length + 1000, file_length + 2000)
        hdrs = {'Range': range_string}
        self.assertRaises(ResponseError, file.read, hdrs=hdrs)
        self.assert_status(416)

        range_string = 'bytes=%d-%d' % (file_length - 1000, file_length + 2000)
        hdrs = {'Range': range_string}
        self.assert_(file.read(hdrs=hdrs) == data[-1000:], range_string)

        hdrs = {'Range': '0-4'}
        self.assert_(file.read(hdrs=hdrs) == data, range_string)

        # RFC 2616 14.35.1
        # "If the entity is shorter than the specified suffix-length, the
        # entire entity-body is used."
        range_string = 'bytes=-%d' % (file_length + 10)
        hdrs = {'Range': range_string}
        self.assert_(file.read(hdrs=hdrs) == data, range_string)

Comment 2 pushpesh sharma 2013-05-08 13:47:57 UTC
[root@dhcp207-5 swift]# rpm -qa|grep swift
glusterfs-swift-3.3.1-13.fc18.noarch
glusterfs-swift-container-3.3.1-13.fc18.noarch
glusterfs-swift-account-3.3.1-13.fc18.noarch
glusterfs-swift-object-3.3.1-13.fc18.noarch
glusterfs-swift-proxy-3.3.1-13.fc18.noarch
python-swiftclient-1.2.0-3.fc18.noarch
[root@dhcp207-5 swift]#

Comment 3 Luis Pabón 2013-06-11 18:54:19 UTC
I can no longer reproduce it using http://build.gluster.org/job/gluster-swift-builds-cent6/lastSuccessfulBuild/artifact/build/glusterfs-openstack-swift-1.8.0-27.el6.noarch.rpm .

Please confirm and close if there is no longer an issue.

Verified:
[root@heka-client-10 swift]# nosetests --exe test/functional/tests.py:TestFile.testRangedGets
testRangedGets (test.functional.tests.TestFile) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.410s

OK

Comment 4 pushpesh sharma 2013-06-12 14:05:17 UTC
Verified on gluster-swift
glusterfs-openstack-swift-1.8.0-28.el6.noarch.rpm	

Issue no longer seen on the build:- 

[root@dhcp207-140 ~]# nosetests --exe ~/swift/test/functional/tests.py:TestFile.testRangedGets
.
----------------------------------------------------------------------
Ran 1 test in 0.732s

OK

Verified using curl request:-

#curl -v -X GET -H 'X-Auth-Token: AUTH_tkf9a56394870d4b6ea218809394abdf7f' https://10.65.207.140:443/v1/AUTH_test/dir/proxy-server.conf -k
HTTP/1.1 200 OK
< Content-Length: 2347
< Accept-Ranges: bytes
< Last-Modified: Wed, 12 Jun 2013 19:26:56 GMT
< Etag: 05b44fa17b60b24f34159052c4de23bb
< X-Timestamp: 1371065216.90306
< Content-Type: application/octet-stream
< Date: Wed, 12 Jun 2013 19:27:19 GMT
< 
[DEFAULT]
#bind_port = 8080
..

# curl -v -X GET -H 'X-Auth-Token: AUTH_tkf9a56394870d4b6ea218809394abdf7f' -H 'Range: bytes=-0' https://10.65.207.140:443/v1/AUTH_test/dir/proxy-server.conf -k

 HTTP/1.1 416 Requested Range Not Satisfiable
< Content-Length: 0
< Accept-Ranges: bytes
< Last-Modified: Wed, 12 Jun 2013 19:26:56 GMT
< Etag: 05b44fa17b60b24f34159052c4de23bb
< X-Timestamp: 1371065216.90306
< Content-Type: application/octet-stream
< Date: Wed, 12 Jun 2013 19:28:07 GMT
< 
* Connection #0 to host 10.65.207.140 left intact
* Closing connection #0


#curl -v -X GET -H 'X-Auth-Token: AUTH_tkf9a56394870d4b6ea218809394abdf7f' -H 'Range: bytes=-3000' https://10.65.207.140:443/v1/AUTH_test/dir/proxy-server.conf -k

< HTTP/1.1 206 Partial Content
< Content-Length: 2347
< Accept-Ranges: bytes
< Last-Modified: Wed, 12 Jun 2013 19:26:56 GMT
< Content-Range: bytes 0-2346/2347
< Etag: 05b44fa17b60b24f34159052c4de23bb
< X-Timestamp: 1371065216.90306
< Content-Type: application/octet-stream
< Date: Wed, 12 Jun 2013 19:29:23 GMT
< 
[DEFAULT]
#bind_port = 8080
user = root
# Consider using 1 worker per CPU
workers = 1
bind_port = 

.....