Description of problem: I used openstack-swift 1.7.7 version installed from sources of its master branch, and gluster-swift-ufo-1.1-4 on rhel 6.2 When downloading a container, It reports a 503 error. Version-Release number of selected component (if applicable): master branch How reproducible: Steps to Reproduce: 1.# swift -A http://ip:8080/auth/v1.0 -U service:swift -K secrete upload bucket6 /root/install.log root/install.log 2.# swift -A http://ip:8080/auth/v1.0 -U service:swift -K secrete download bucket6 3. Actual results: 2. root/install.log [headers 0.016s, total 0.017s, 3.735s MB/s] Object GET failed: http://10.1.83.82:8080/v1/AUTH_service/bucket6/root 503 Internal Server Error [first 60 chars of response] <html><h1>Service Unavailable</h1><p>The server is currently Logs in /var/log/messages as below: proxy-server ERROR with Object server 127.0.0.1:6010/service re: Trying to GET /v1/AUTH_service/bucket6/root: #012Traceback (most recent call last):#012 File "/usr/lib/python2.6/site-packages/swift-1.7.7-py2.6.egg/swift/proxy/controllers/base.py", line 732, in GETorHEAD_base#012 possible_source = conn.getresponse()#012 File "/usr/lib/python2.6/site-packages/swift-1.7.7-py2.6.egg/swift/common/bufferedhttp.py", line 102, in getresponse#012 response = HTTPConnection.getresponse(self)#012 File "/usr/lib64/python2.6/httplib.py", line 994, in getresponse#012 response.begin()#012 File "/usr/lib64/python2.6/httplib.py", line 395, in begin#012 version, status, reason = self._read_status()#012 File "/usr/lib64/python2.6/httplib.py", line 356, in _read_status#012 raise BadStatusLine(line)#012BadStatusLine (txn: tx1d1e0985647540fba8e3d5850173a100) Expected results: download the full container Additional info:
Log message with newlines added back: proxy-server ERROR with Object server 127.0.0.1:6010/service re: Trying to GET /v1/AUTH_service/bucket6/root: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/swift-1.7.7-py2.6.egg/swift/proxy/controllers/base.py", line 732, in GETorHEAD_base possible_source = conn.getresponse() File "/usr/lib/python2.6/site-packages/swift-1.7.7-py2.6.egg/swift/common/bufferedhttp.py", line 102, in getresponse response = HTTPConnection.getresponse(self) File "/usr/lib64/python2.6/httplib.py", line 994, in getresponse response.begin() File "/usr/lib64/python2.6/httplib.py", line 395, in begin version, status, reason = self._read_status() File "/usr/lib64/python2.6/httplib.py", line 356, in _read_status raise BadStatusLine(line) BadStatusLine (txn: tx1d1e0985647540fba8e3d5850173a100)
Is that all there is in the logs? Are there any other messages?
(In reply to comment #2) > Is that all there is in the logs? Are there any other messages? Yes, that's all the log message when download a container. However, it successed downloading a container while using CyberDuck.
The URL "/v1/AUTH_service/bucket6/root" seems to be an object reference. The swift account is "AUTH_service", the container is "bucket6", and the object is "root". On the server, what does this command return: ls -ld /mnt/gluster-object/AUTH_service/bucket6/root
(In reply to comment #4) > The URL "/v1/AUTH_service/bucket6/root" seems to be an object reference. The > swift account is "AUTH_service", the container is "bucket6", and the object > is "root". On the server, what does this command return: ls -ld > /mnt/gluster-object/AUTH_service/bucket6/root ls -ld /mnt/gluster-object/service/bucket6/root/install.log -rwxr-xr-x 1 root root 64601 Mar 21 05:06 /mnt/gluster-object/service/bucket6/root/install.log # ls -ld /mnt/gluster-object/service/bucket6/root drwxr-xr-x 2 root root 4102 Mar 21 05:06 /mnt/gluster-object/service/bucket6/root
Try doing a GET on /v1/AUTH_service/bucket6 ... It sounds like the bug is with getting a listing for an object that is a directory and not a container.
With latest of G4S code (master, HAVANA based) and with openstack swift 1.9.* (HAVANA) Below is the setup of the directory and directory types of objects. [root@vm2 ~]# ls -al /mnt/gluster-object/test2/container1/ total 0 drwxr-xr-x. 3 root root 30 Sep 16 05:10 . drwxr-xr-x. 4 root root 40 Sep 16 05:01 .. -rwxr-xr-x. 1 root root 0 Sep 16 05:02 dir1 drwxr-xr-x. 2 root root 6 Sep 16 05:10 newdir .. [root@vm2 ~]# curl -v -X GET http://127.0.0.1:8080/v1/AUTH_test2/container1/dir1 * About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... * connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > GET /v1/AUTH_test2/container1/dir1 HTTP/1.1 > User-Agent: curl/7.27.0 > Host: 127.0.0.1:8080 > Accept: */* > < HTTP/1.1 200 OK < Content-Length: 0 < Accept-Ranges: bytes < Last-Modified: Mon, 16 Sep 2013 09:02:09 GMT < Etag: d41d8cd98f00b204e9800998ecf8427e < X-Timestamp: 1379322129.73821 < Content-Type: application/octet-stream < X-Trans-Id: tx1829056739894efea36f2-00523827b9 < Date: Tue, 17 Sep 2013 09:58:17 GMT < * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 and now a GET on the directory named(newdir) manually created on the gluster volume. [root@vm2 ~]# curl -v -X GET http://127.0.0.1:8080/v1/AUTH_test2/container1/newdir * About to connect() to 127.0.0.1 port 8080 (#0) * Trying 127.0.0.1... * connected * Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0) > GET /v1/AUTH_test2/container1/newdir HTTP/1.1 > User-Agent: curl/7.27.0 > Host: 127.0.0.1:8080 > Accept: */* > < HTTP/1.1 200 OK < Content-Length: 0 < Accept-Ranges: bytes < Last-Modified: Mon, 16 Sep 2013 09:10:39 GMT < Etag: d41d8cd98f00b204e9800998ecf8427e < X-Timestamp: 1379322639.30244 < Content-Type: application/directory < X-Trans-Id: tx3e08236dd93b4c1b9c45e-00523827ed < Date: Tue, 17 Sep 2013 09:59:09 GMT < * Connection #0 to host 127.0.0.1 left intact * Closing connection #0 So both the cases are fixed in master branch of G4S, and with openstack swift (HAVANA). Issue reported is still present with grizzly. But marking it as fixed now with latest master G4S code base with below change http://review.gluster.org/#/c/5511/
This is not reproducible with HAVANA. But bug is present in grizzly. I think we can mark it as ON_QA as latest HAVANA code is out in downstream.
Not reproducible with latest upstream G4S(havana based). I used headers 'Content-type: application/directory' 'Content-Length: 0' for creating sub-directory. GET on a sub directory currently does not return list of content in it(already logged as a known issue), However there are no errors. GET on a container works fine, as expected. Marking this as verified. Log a new issue on latest version of G4S, if you still see the problem.