Bug 1387437 - Object Gateway does not support absolute URL
Summary: Object Gateway does not support absolute URL
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RGW
Version: 2.0
Hardware: x86_64
OS: Linux
Target Milestone: rc
: 3.0
Assignee: Matt Benjamin (redhat)
QA Contact: Matt Benjamin (redhat)
Bara Ancincova
Depends On:
Blocks: 1494421
TreeView+ depends on / blocked
Reported: 2016-10-20 21:21 UTC by Mike Hackett
Modified: 2020-03-11 15:19 UTC (History)
16 users (show)

Fixed In Version: RHEL: ceph-12.1.2-1.el7cp Ubuntu: ceph_12.1.2-2redhat1xenial
Doc Type: Bug Fix
Doc Text:
.AWS SDK for Golang applications work as expected with the Ceph Object Gateway A bug in the URL processing in the Civetweb HTTP server caused certain kinds of Simple Storage Service (S3) requests to fail. The affected requests included for example a number of requests generated by clients of the Amazon Web Services (AWS) Software Development Kit (SDK) for Golang. Consequently, S3 applications written for AWS SDK for Golang did not interact correctly with the Ceph Object Gateway. This update fixes the handling of absolute URIs is Civetweb, and the AWS SDK for Golang applications work as expected with the Ceph Object Gateway.
Clone Of:
Last Closed: 2017-12-05 23:31:14 UTC
Target Upstream Version:

Attachments (Terms of Use)
wireshark screenshot, HTTP request detail packet 2132 (195.96 KB, image/png)
2017-10-25 22:15 UTC, Matt Benjamin (redhat)
no flags Details

System ID Priority Status Summary Last Updated
Ceph Project Bug Tracker 12917 None None None 2016-10-20 21:21:37 UTC
Ceph Project Bug Tracker 17657 None None None 2017-08-08 22:06:25 UTC
Red Hat Product Errata RHBA-2017:3387 normal SHIPPED_LIVE Red Hat Ceph Storage 3.0 bug fix and enhancement update 2017-12-06 03:03:45 UTC

Description Mike Hackett 2016-10-20 21:21:37 UTC
Description of problem:

Upstream PR https://github.com/ceph/ceph/pull/7675 is integrated in RHCS 2.0 but still unable to use GO AWS SDK to access with RadosGW since issue looks to be with absolute URI in civetweb 'process_new_connection()' call defined in src/civetweb/src/civetweb.c.

PR 7675 updates: This PR only affect code civetweb_callback, however process_new_connection in civetweb still checks uri and failed on absolute uri.





Version-Release number of selected component (if applicable):

Red Hat Ceph Storage 2.0

Comment 16 Matt Benjamin (redhat) 2017-10-25 22:13:31 UTC
Based on the original reproducer for this bz and its cognate reproducer in upstream Ceph tracker, it appears sufficient to verify that 

1. RGW properly handles absolute URIs

in particular, 

2. we would like to verify that RGW handles absolute URIs generated by minio's "mc" S3 client for golang

I verified these assertions as follows:

1. configured minio mc S3 client with host "lemon" such that mc is likely to generate absolute URIs:

cat ~/.mc/config.json

		"lemon": {
			"url": "http://lemon:80",
			"accessKey": "<access key>",
			"secretKey": "<secret key>",
			"api": "S3v4"

2. ran simple requests (which succeed)
[mbenjamin@lemon mc]$  mc ls lemon
[2017-09-30 13:02:31 EDT]     0B bu_0/
[2017-09-30 13:02:34 EDT]     0B bu_1/
[2017-09-30 13:02:35 EDT]     0B bu_2/
[2017-09-30 13:02:35 EDT]     0B bu_3/
[2017-09-30 13:02:35 EDT]     0B bu_4/
[2017-09-30 13:02:35 EDT]     0B bu_5/
[2017-09-30 13:02:35 EDT]     0B bu_6/
[2017-09-30 13:02:35 EDT]     0B bu_7/
[2017-09-30 13:02:35 EDT]     0B bu_8/
[2017-09-30 13:02:36 EDT]     0B bu_9/
[2017-09-30 13:48:20 EDT]     0B buck1/
[2017-10-06 16:47:23 EDT]     0B buck2/
[2017-10-12 09:16:16 EDT]     0B buck3/
[2017-10-23 16:54:34 EDT]     0B buck5/
[2017-10-23 11:28:34 EDT]     0B buck_ver1/
[2017-09-30 13:49:36 EDT]     0B new_dir/
[2017-10-12 09:23:26 EDT]     0B nfs/
[2017-10-16 16:41:58 EDT]     0B nfsroot/
[2017-10-17 15:46:32 EDT]     0B shay/
[mbenjamin@lemon mc]$  mc ls lemon/nfs
[2017-10-25 18:08:30 EDT]     0B level00/
[2017-10-25 18:08:30 EDT]     0B level01/

3. using wireshark, verified that HTTP protocol decode contains absolute URIs we would expect--screenshot attached, note the value of "Full Request URI" in the middle panel

Comment 17 Matt Benjamin (redhat) 2017-10-25 22:15:10 UTC
Created attachment 1343444 [details]
wireshark screenshot, HTTP request detail packet 2132

screenshot of wireshark output from verified baseline

Comment 22 errata-xmlrpc 2017-12-05 23:31:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.