Bug 846657 - gluster-object: 400 Bad request syntax
gluster-object: 400 Bad request syntax
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: gluster-swift (Show other bugs)
2.0
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Vivek Agarwal
pushpesh sharma
:
Depends On:
Blocks: 858439
  Show dependency treegraph
 
Reported: 2012-08-08 07:03 EDT by Saurabh
Modified: 2016-02-17 19:03 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 858439 (view as bug list)
Environment:
Last Closed: 2013-09-23 18:32:20 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Saurabh 2012-08-08 07:03:25 EDT
Description of problem:

the below messages are seen after a ConnectionTimeout(5.0s), 
usually Bad request happens when the container is not available, but in this case the container is available.

Aug  8 10:27:39 gqac028 proxy-server ERROR with Container server 127.0.0.1:6011/sdb1 re: Trying to get container info for /AUTH_test2/cont1865: ConnectionTimeout (5.0s) (txn: txe60c25161a2c45e9b917680cfcca643d) (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR WSGI: code 400, message Bad request syntax ('=\x16\xd2p.W\xa2\x93\xb7\x86\xf3w\x93`h@\xe1$\x80#\xfe\xc8\xc3\r\x12\xc5\tp\x0e\xf9\xc8\xb3\xda\xcf\x10\x9e\x19\xdc\xa493\xac\x80\xb3\xa5\xd8m\x9a\xa56\x03V\r\t\x1c\x1c\x86C\xd8\x13\x14\xdc\xc8\x01I\x0e\xbb\x8b\xe6d2\xfb\xe3&\x7ff\x08\x94\x1e\xfb$:\x90\xca\xb7\xd1Tw\x8d\xd7\xb6\xbe\xe4\xeb\xf3\xceb\xf0\x0f=\xd2Q"\xd3\x99\x99b.n\xa1\x05xQ\rz#M\x8b\xd2\x9f\xd0\xa4\xf0\x02\x1a2\xd0\xac\xb6\xfb\xf3\x81-\xc9\xe0,s\xba\x15(\x08\xb8v\xb6b9\x06b0@\xa7\xfeNX\x8f\xaf5[\x95\xa7\x1c\x1d\xf1\xa0\xb7\xa1\xef\x90\x19\xfe\x7f\x83~b\xcb\xe4i\xa5\x13\xc2x\xa7`\x87\x16\xc3E|-\x94\xb3\x93.QC\xa5\xbe\x80\xa2>\xd25\x9b\xb8pL\x020\xa4\xf8.\xc0\xa2\x98\\\xc5\x8b\xbb\xd1\'Y\xb1JF\x8c\xaa\xc8d\xdcR \xd9\xb1\xee \x94\x07\x06\xe4\x02b\x80\xfbX\\\xfa\x84$\xd0h*\x13\x8b\xd8\xcdIT\x9c\xee\xdf\xc3\xda\x87\x7fGG\xd7\xf2\r\\\x8d\x8c\xddv\x08\x86\xef"K.\xf0@\x82\x8fI\xd0V_\xa4f\xdc\xcf\x0e\x97\x00\r)!D\x08\xc2\x91\xd8\x1c\xf0\xde+\xa9\xae_z\x8c\xad\xefH\x93\xd0\x10\xe6o\xd0.\x82!\x01o\xd5F\x80\xd7z\x8d\x90\xf8\x0f\x9b\x13') (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR WSGI: code 400, message Bad request syntax ('=\x16\xd2p.W\xa2\x93\xb7\x86\xf3w\x93`h@\xe1$\x80#\xfe\xc8\xc3\r\x12\xc5\tp\x0e\xf9\xc8\xb3\xda\xcf\x10\x9e\x19\xdc\xa493\xac\x80\xb3\xa5\xd8m\x9a\xa56\x03V\r\t\x1c\x1c\x86C\xd8\x13\x14\xdc\xc8\x01I\x0e\xbb\x8b\xe6d2\xfb\xe3&\x7ff\x08\x94\x1e\xfb$:\x90\xca\xb7\xd1Tw\x8d\xd7\xb6\xbe\xe4\xeb\xf3\xceb\xf0\x0f=\xd2Q"\xd3\x99\x99b.n\xa1\x05xQ\rz#M\x8b\xd2\x9f\xd0\xa4\xf0\x02\x1a2\xd0\xac\xb6\xfb\xf3\x81-\xc9\xe0,s\xba\x15(\x08\xb8v\xb6b9\x06b0@\xa7\xfeNX\x8f\xaf5[\x95\xa7\x1c\x1d\xf1\xa0\xb7\xa1\xef\x90\x19\xfe\x7f\x83~b\xcb\xe4i\xa5\x13\xc2x\xa7`\x87\x16\xc3E|-\x94\xb3\x93.QC\xa5\xbe\x80\xa2>\xd25\x9b\xb8pL\x020\xa4\xf8.\xc0\xa2\x98\\\xc5\x8b\xbb\xd1\'Y\xb1JF\x8c\xaa\xc8d\xdcR \xd9\xb1\xee \x94\x07\x06\xe4\x02b\x80\xfbX\\\xfa\x84$\xd0h*\x13\x8b\xd8\xcdIT\x9c\xee\xdf\xc3\xda\x87\x7fGG\xd7\xf2\r\\\x8d\x8c\xddv\x08\x86\xef"K.\xf0@\x82\x8fI\xd0V_\xa4f\xdc\xcf\x0e\x97\x00\r)!D\x08\xc2\x91\xd8\x1c\xf0\xde+\xa9\xae_z\x8c\xad\xefH\x93\xd0\x10\xe6o\xd0.\x82!\x01o\xd5F\x80\xd7z\x8d\x90\xf8\x0f\x9b\x13') (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR with Container server 127.0.0.1:6011/sdb1 re: Trying to get container info for /AUTH_test2/cont1868: ConnectionTimeout (5.0s) (txn: txd49185f087eb45b8b6bc4a1543633116) (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR WSGI: code 400, message Bad request syntax ('=\x16\xd2p.W\xa2\x93\xb7\x86\xf3w\x93`h@\xe1$\x80#\xfe\xc8\xc3\r\x12\xc5\tp\x0e\xf9\xc8\xb3\xda\xcf\x10\x9e\x19\xdc\xa493\xac\x80\xb3\xa5\xd8m\x9a\xa56\x03V\r\t\x1c\x1c\x86C\xd8\x13\x14\xdc\xc8\x01I\x0e\xbb\x8b\xe6d2\xfb\xe3&\x7ff\x08\x94\x1e\xfb$:\x90\xca\xb7\xd1Tw\x8d\xd7\xb6\xbe\xe4\xeb\xf3\xceb\xf0\x0f=\xd2Q"\xd3\x99\x99b.n\xa1\x05xQ\rz#M\x8b\xd2\x9f\xd0\xa4\xf0\x02\x1a2\xd0\xac\xb6\xfb\xf3\x81-\xc9\xe0,s\xba\x15(\x08\xb8v\xb6b9\x06b0@\xa7\xfeNX\x8f\xaf5[\x95\xa7\x1c\x1d\xf1\xa0\xb7\xa1\xef\x90\x19\xfe\x7f\x83~b\xcb\xe4i\xa5\x13\xc2x\xa7`\x87\x16\xc3E|-\x94\xb3\x93.QC\xa5\xbe\x80\xa2>\xd25\x9b\xb8pL\x020\xa4\xf8.\xc0\xa2\x98\\\xc5\x8b\xbb\xd1\'Y\xb1JF\x8c\xaa\xc8d\xdcR \xd9\xb1\xee \x94\x07\x06\xe4\x02b\x80\xfbX\\\xfa\x84$\xd0h*\x13\x8b\xd8\xcdIT\x9c\xee\xdf\xc3\xda\x87\x7fGG\xd7\xf2\r\\\x8d\x8c\xddv\x08\x86\xef"K.\xf0@\x82\x8fI\xd0V_\xa4f\xdc\xcf\x0e\x97\x00\r)!D\x08\xc2\x91\xd8\x1c\xf0\xde+\xa9\xae_z\x8c\xad\xefH\x93\xd0\x10\xe6o\xd0.\x82!\x01o\xd5F\x80\xd7z\x8d\x90\xf8\x0f\x9b\x13') (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR with Container server 127.0.0.1:6011/sdb1 re: Trying to get container info for /AUTH_test2/cont1866: ConnectionTimeout (5.0s) (txn: tx3fb1e8c648d442fab8bfcb8da52cee42) (client_ip: 10.16.159.238)
Aug  8 10:27:39 gqac028 proxy-server ERROR WSGI: code 400, message Bad request syntax ('=\x16\xd2p.W\xa2\x93\xb7\x86\xf3w\x93`h@\xe1$\x80#\xfe\xc8\xc3\r\x12\xc5\tp\x0e\xf9\xc8\xb3\xda\xcf\x10\x9e\x19\xdc\xa493\xac\x80\xb3\xa5\xd8m\x9a\xa56\x03V\r\t\x1c\x1c\x86C\xd8\x13\x14\xdc\xc8\x01I\x0e\xbb\x8b\xe6d2\xfb\xe3&\x7ff\x08\x94\x1e\xfb$:\x90\xca\xb7\xd1Tw\x8d\xd7\xb6\xbe\xe4\xeb\xf3\xceb\xf0\x0f=\xd2Q"\xd3\x99\x99b.n\xa1\x05xQ\rz#M\x8b\xd2\x9f\xd0\xa4\xf0\x02\x1a2\xd0\xac\xb6\xfb\xf3\x81-\xc9\xe0,s\xba\x15(\x08\xb8v\xb6b9\x06b0@\xa7\xfeNX\x8f\xaf5[\x95\xa7\x1c\x1d\xf1\xa0\xb7\xa1\xef\x90\x19\xfe\x7f\x83~b\xcb\xe4i\xa5\x13\xc2x\xa7`\x87\x16\xc3E|-\x94\xb3\x93.QC\xa5\xbe\x80\xa2>\xd25\x9b\xb8pL\x020\xa4\xf8.\xc0\xa2\x98\\\xc5\x8b\xbb\xd1\'Y\xb1JF\x8c\xaa\xc8d\xdcR \xd9\xb1\xee \x94\x07\x06\xe4\x02b\x80\xfbX\\\xfa\x84$\xd0h*\x13\x8b\xd8\xcdIT\x9c\xee\xdf\xc3\xda\x87\x7fGG\xd7\xf2\r\\\x8d\x8c\xddv\x08\x86\xef"K.\xf0@\x82\x8fI\xd0V_\xa4f\xdc\xcf\x0e\x97\x00\r)!D\x08\xc2\x91\xd8\x1c\xf0\xde+\xa9\xae_z\x8c\xad\xefH\x93\xd0\x10\xe6o\xd0.\x82!\x01o\xd5F\x80\xd7z\x8d\x90\xf8\x0f\x9b\x13') (client_ip: 10.16.159.238)

Version-Release number of selected component (if applicable):

[root@gqac028 ~]# glusterfs -V
glusterfs 3.3.0rhs built on Jul 25 2012 11:21:57
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.


How reproducible:
seen happening this time

Steps to Reproduce:
1. send request to create 2000 objects in parallel to different containers.
2.
3.
  
Actual results:
the Bad request message seems to be misguiding at this stage

Expected results:


Additional info:
Comment 2 Junaid 2013-01-22 01:15:11 EST
This should be fixed in gluster-swift-1.7.4 rpms. Reopen if seen again.
Comment 3 pushpesh sharma 2013-07-17 06:37:15 EDT
This BZ has been verified using catalyst workload on RHS2.1.It seems to be fixed, as new PDQ performance related changes are merged to RHS2.1. 

[root@dhcp207-9 ~]# rpm -qa|grep gluster
gluster-swift-object-1.8.0-6.3.el6rhs.noarch
vdsm-gluster-4.10.2-22.7.el6rhs.noarch
gluster-swift-plugin-1.8.0-2.el6rhs.noarch
glusterfs-geo-replication-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-1.8.0-6.3.el6rhs.noarch
glusterfs-server-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-proxy-1.8.0-6.3.el6rhs.noarch
gluster-swift-account-1.8.0-6.3.el6rhs.noarch
glusterfs-rdma-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-container-1.8.0-6.3.el6rhs.noarch


All performance related tests(From QE perspective) will be done using catalyst workload(If required in future may be ssbench).Which has 15 runs of 10000 requests(PUT/GET/HEAD/DELETE) each distributed among 10 threads.These comprehensive test include all file formats and varied sizes.These test executed on a machine with following configuration:-

RAM:- 7500Gb
CPU:- 1
Volume Info:-

All bricks are created as a logical volume(on localhost) of 10G each, and each volume has 4 of such bricks.

[root@dhcp207-9 ~]# gluster volume info
 
Volume Name: test
Type: Distribute
Volume ID: 440fdac0-a3bd-4ab1-a70c-f4c390d97100
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: localhost:/mnt/lv1/lv1
Brick2: localhost:/mnt/lv2/lv2
Brick3: localhost:/mnt/lv3/lv3
Brick4: localhost:/mnt/lv4/lv4
 
Volume Name: test2
Type: Distribute
Volume ID: 6d922203-6657-4ed3-897a-069ef6c396bf
Status: Started
Number of Bricks: 4
Transport-type: tcp
Bricks:
Brick1: localhost:/mnt/lv5/lv5
Brick2: localhost:/mnt/lv6/lv6
Brick3: localhost:/mnt/lv7/lv7
Brick4: localhost:/mnt/lv8/lv8


PS: Performance Engineering will be responsible for all large scale test , which will be done on BAGL cluster.
Comment 5 Scott Haines 2013-09-23 18:32:20 EDT
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html

Note You need to log in before you can comment on or make changes to this bug.