.AWS SDK for Golang applications work as expected with the Ceph Object Gateway
A bug in the URL processing in the Civetweb HTTP server caused certain kinds of Simple Storage Service (S3) requests to fail. The affected requests included for example a number of requests generated by clients of the Amazon Web Services (AWS) Software Development Kit (SDK) for Golang. Consequently, S3 applications written for AWS SDK for Golang did not interact correctly with the Ceph Object Gateway. This update fixes the handling of absolute URIs is Civetweb, and the AWS SDK for Golang applications work as expected with the Ceph Object Gateway.
Description of problem:
Upstream PR https://github.com/ceph/ceph/pull/7675 is integrated in RHCS 2.0 but still unable to use GO AWS SDK to access with RadosGW since issue looks to be with absolute URI in civetweb 'process_new_connection()' call defined in src/civetweb/src/civetweb.c.
PR 7675 updates: This PR only affect code civetweb_callback, however process_new_connection in civetweb still checks uri and failed on absolute uri.
Version-Release number of selected component (if applicable):
Red Hat Ceph Storage 2.0
Based on the original reproducer for this bz and its cognate reproducer in upstream Ceph tracker, it appears sufficient to verify that
1. RGW properly handles absolute URIs
2. we would like to verify that RGW handles absolute URIs generated by minio's "mc" S3 client for golang
I verified these assertions as follows:
1. configured minio mc S3 client with host "lemon" such that mc is likely to generate absolute URIs:
"accessKey": "<access key>",
"secretKey": "<secret key>",
2. ran simple requests (which succeed)
[mbenjamin@lemon mc]$ mc ls lemon
[2017-09-30 13:02:31 EDT] 0B bu_0/
[2017-09-30 13:02:34 EDT] 0B bu_1/
[2017-09-30 13:02:35 EDT] 0B bu_2/
[2017-09-30 13:02:35 EDT] 0B bu_3/
[2017-09-30 13:02:35 EDT] 0B bu_4/
[2017-09-30 13:02:35 EDT] 0B bu_5/
[2017-09-30 13:02:35 EDT] 0B bu_6/
[2017-09-30 13:02:35 EDT] 0B bu_7/
[2017-09-30 13:02:35 EDT] 0B bu_8/
[2017-09-30 13:02:36 EDT] 0B bu_9/
[2017-09-30 13:48:20 EDT] 0B buck1/
[2017-10-06 16:47:23 EDT] 0B buck2/
[2017-10-12 09:16:16 EDT] 0B buck3/
[2017-10-23 16:54:34 EDT] 0B buck5/
[2017-10-23 11:28:34 EDT] 0B buck_ver1/
[2017-09-30 13:49:36 EDT] 0B new_dir/
[2017-10-12 09:23:26 EDT] 0B nfs/
[2017-10-16 16:41:58 EDT] 0B nfsroot/
[2017-10-17 15:46:32 EDT] 0B shay/
[mbenjamin@lemon mc]$ mc ls lemon/nfs
[2017-10-25 18:08:30 EDT] 0B level00/
[2017-10-25 18:08:30 EDT] 0B level01/
3. using wireshark, verified that HTTP protocol decode contains absolute URIs we would expect--screenshot attached, note the value of "Full Request URI" in the middle panel
Created attachment 1343444 [details]
wireshark screenshot, HTTP request detail packet 2132
screenshot of wireshark output from verified baseline
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.