Bug 960889

Summary: G4S: PUT/GET/HEAD/DELETE request where file and containers are named in UTF-8 format fails
Product: [Community] GlusterFS Reporter: pushpesh sharma <psharma>
Component: object-storageAssignee: Junaid <junaid>
Status: CLOSED CURRENTRELEASE QA Contact: pushpesh sharma <psharma>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.3.1CC: bbandari, gluster-bugs, lpabon, vagarwal
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-24 18:02:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/log/messages none

Description pushpesh sharma 2013-05-08 09:13:01 UTC
I ran the functional tests of swift on G4S setup and got these results,related to UTF-8.

However the point here to note is that some TCs failes both for ASCII and UTF-8 encoding here i am mentioning only those failures which are specific to UTF-8.Some cases here results in Error(i.e 503 Internal Server Error) which also points to more severe bugs :- 

I will try to provide logs(/var/log/messages) for indiviual failed/Error TCs, but for now attaching logs for entire Test suite. 
  
ERROR: testLastContainerMarker (test.functional.tests.TestAccountUTF8)
ERROR: testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8)
ERROR: testFileListingLimitMarkerPrefix (test.functional.tests.TestContainerUTF8)
ERROR: testLastFileMarker (test.functional.tests.TestContainerUTF8)
ERROR: testMarkerLimitFileList (test.functional.tests.TestContainerUTF8)
FAIL: testInvalidUTF8Path (test.functional.tests.TestAccount)
FAIL: testInvalidUTF8Path (test.functional.tests.TestAccountUTF8)

FAIL: testPrefixAndLimit (test.functional.tests.TestContainerUTF8)

 
[psharma@dhcp193-66 ~]$ nosetests --with-xunit --xunit-file functional-all-nosetests.xml /home/psharma/git/swift/test/functional/tests.py
......F..............FE.E.........F...F...........FFF......F...F..E..EEF...................F..F.....E...F...............................F..F.....E...F.....
======================================================================
ERROR: testLastContainerMarker (test.functional.tests.TestAccountUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 319, in testLastContainerMarker
    parms={'format': format, 'marker': containers[-1]})
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 347, in containers
    raise ResponseError(self.conn.response)
ResponseError: 503: Internal Server Error

======================================================================
ERROR: testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 333, in testMarkerLimitContainerList
    parms={'format': format, 'marker': marker, 'limit': limit})
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 347, in containers
    raise ResponseError(self.conn.response)
ResponseError: 503: Internal Server Error

======================================================================
ERROR: testFileListingLimitMarkerPrefix (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 457, in testFileListingLimitMarkerPrefix
    self.assert_(cont.files(parms={'limit': j, 'marker': f}) ==
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 447, in files
    raise ResponseError(self.conn.response)
ResponseError: 503: Internal Server Error

======================================================================
ERROR: testLastFileMarker (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 585, in testLastFileMarker
    parms={'format': format, 'marker': files[-1]})
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 447, in files
    raise ResponseError(self.conn.response)
ResponseError: 503: Internal Server Error

======================================================================
ERROR: testMarkerLimitFileList (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 613, in testMarkerLimitFileList
    'limit': limit})
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 447, in files
    raise ResponseError(self.conn.response)
ResponseError: 503: Internal Server Error

======================================================================
ERROR: testNameLimit (test.functional.tests.TestFile)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 1040, in testNameLimit
    self.assert_(file.write())
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 725, in write
    raise ResponseError(self.conn.response)
ResponseError: 400: Bad Request

======================================================================
ERROR: testNameLimit (test.functional.tests.TestFileUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 1040, in testNameLimit
    self.assert_(file.write())
  File "/home/psharma/git/swift/test/functional/swift_test_client.py", line 725, in write
    raise ResponseError(self.conn.response)
ResponseError: 400: Bad Request

======================================================================
FAIL: testInvalidUTF8Path (test.functional.tests.TestAccount)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 215, in testInvalidUTF8Path
    self.assert_body('Invalid UTF8 or contains NULL')
  File "/home/psharma/git/swift/test/functional/tests.py", line 151, in assert_body
    'Body returned: %s' % (response_body))
AssertionError: Body returned: Invalid UTF8

======================================================================
FAIL: testInvalidUTF8Path (test.functional.tests.TestAccountUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 215, in testInvalidUTF8Path
    self.assert_body('Invalid UTF8 or contains NULL')
  File "/home/psharma/git/swift/test/functional/tests.py", line 151, in assert_body
    'Body returned: %s' % (response_body))
AssertionError: Body returned: Invalid UTF8


======================================================================
FAIL: testContainerNameLimit (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 426, in testContainerNameLimit
    self.assert_status(201)
  File "/home/psharma/git/swift/test/functional/tests.py", line 158, in assert_status
    (self.env.conn.response.status, status_or_statuses))
AssertionError: Status returned: 202 Expected: 201

======================================================================
FAIL: testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 561, in testDeleteOnContainerThatDoesNotExist
    self.assert_status(404)
  File "/home/psharma/git/swift/test/functional/tests.py", line 158, in assert_status
    (self.env.conn.response.status, status_or_statuses))
AssertionError: Status returned: 503 Expected: 404

======================================================================
FAIL: testPrefixAndLimit (test.functional.tests.TestContainerUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 466, in testPrefixAndLimit
    self.assert_(cont.create())
AssertionError: False is not true

======================================================================
FAIL: testFileSizeLimit (test.functional.tests.TestFileUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 1212, in testFileSizeLimit
    file.write, cfg={'set_content_length': i})
AssertionError: ResponseError not raised

======================================================================
FAIL: testHead (test.functional.tests.TestFileUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 1309, in testHead
    self.assertEquals(info['content_length'], self.env.file_size)
AssertionError: 13 != 128

======================================================================
FAIL: testRangedGets (test.functional.tests.TestFileUTF8)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/psharma/git/swift/test/functional/tests.py", line 1157, in testRangedGets
    self.assertRaises(ResponseError, file.read, hdrs=hdrs)
AssertionError: ResponseError not raised

----------------------------------------------------------------------
Ran 155 tests in 602.703s

FAILED (errors=7, failures=16)

Comment 1 pushpesh sharma 2013-05-08 09:18:40 UTC
Created attachment 745140 [details]
/var/log/messages

Comment 2 Luis Pabón 2013-06-20 03:34:58 UTC
testLastContainerMarker (test.functional.tests.TestAccount) ... ok
testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok
testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok
testLastFileMarker (test.functional.tests.TestContainer) ... ok
testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok
testInvalidUTF8Path (test.functional.tests.TestAccount) ... ok
testInvalidUTF8Path (test.functional.tests.TestAccountUTF8) ... ok

All tests are now working.  Using http://build.gluster.org/job/gluster-swift-builds-cent6/lastSuccessfulBuild/artifact/build/glusterfs-openstack-swift-1.8.0-29.el6.noarch.rpm

Comment 3 pushpesh sharma 2013-07-24 11:59:45 UTC
Verified:-

nosetests --with-xunit --xunit-file functional-all-nosetests.xml --with-html-output --html-out-file=functional-all-nosetests.html ~/gluster-swift/test/functional/tests.py
..................................................SSS......................................S............................................S..................
----------------------------------------------------------------------