Bug 1253891

Summary: keystone / rgw interaction -- system frozen.
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Warren <wusui>
Component: RGWAssignee: Yehuda Sadeh <yehuda>
Status: CLOSED NOTABUG QA Contact: ceph-qe-bugs <ceph-qe-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 1.3.0CC: cbodley, ceph-eng-bugs, kbader, mbenjamin, owasserm, smanjara, sweil, wusui
Target Milestone: rc   
Target Release: 1.3.2   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-08 18:01:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Warren 2015-08-15 05:10:56 UTC
Description of problem:

While trying to test rgw that uses keystone for authentication, I believe that I managed to set up rgw to use keystone.

I tried a few invalid curl commands, and the machine now does not respond to anything.

Version-Release number of selected component (if applicable):
1.2.3 update

How reproducible:
It happened once

Steps to Reproduce:

[root@magna068 ubuntu]# curl -s -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "87b21dc623e84ddf"}}}' -H 'Content-type: application/json' http://localhost:5000/v3.0
{"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}[root@magna068 ubuntu]# ^C
[root@magna068 ubuntu]# curl -s -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "87b21dc623e84ddf"}}}' -H 'Content-type: application/json' http://localhost:5000/v3.

At this point, nothing happens on the screen.  ssh to magna068 from another window gets a port 22: Connection timed out 


Actual results:

System stuck

Expected results:

Anything but stuck

Additional Information:

This happened on magna068.  I will leave it in that state in case someone wants to look at it.

Comment 2 Warren 2015-08-15 05:25:17 UTC
After several minutes, I got:

[root@magna068 ubuntu]# curl -s -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "87b21dc623e84ddf"}}}' -H 'Content-type: application/json' http://localhost:5000/v3.Write failed: Broken pipe


When I try to connect again:

wusui@magna002:~$ ssh -X -A magna068
ssh: connect to host magna068 port 22: No route to host

Comment 3 Yehuda Sadeh 2015-08-15 06:09:52 UTC
I'm trying to figure out what you did and why you think this is rgw bug. From what I can tell you contacted keystone through curl and the system stopped responding. If any that would be a bug with keystone. Maybe check what the system logs are showing?

Note that you used v3 auth, whereas we only support v2, but that's really unrelated.

Comment 4 shilpa 2015-08-15 07:28:24 UTC
Hi Warren,

Is this about BZ 1213999? I think what you are trying to do is get keystone tokens? Try this with v2:

#curl -s -d '{"auth": {"tenantName": "admin", "passwordCredentials": {"username": "admin", "password": "87b21dc623e84ddf"}}}' -H 'Content-type: application/json' http://localhost:5000/v2.0/tokens

If that command is successful, you should get a token ID and using that ID(which is a value for "X-Auth-Token"), you could contact RGW node to make curl transactions.

# curl -v -H "X-Auth-Token: 38abd6XXXX" http://rgwnode.com/swift/v1

If the configuration is correct, the you should see an output like this:

*   Trying 10.X.X.X...
* Connected to rgwnode.com (10.X.X.X) port 80 (#0)
> GET /swift/v1 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: rgwnode.com
> Accept: */*
> X-Auth-Token: 38abd6842e294e56b65a15883764f841

But if you are using downstream RHELOSP 7 and you got that error mentioned in c#2, then it should be a RHELOSP bug. The curl request you made does attempt to make any contact with rgw.

Comment 5 shilpa 2015-08-15 07:34:05 UTC
> But if you are using downstream RHELOSP 7 and you got that error mentioned
> in c#2, then it should be a RHELOSP bug. The curl request you made does not
> attempt to make any contact with rgw.

Or it could just be a keystone configuration issue.

Comment 6 Yehuda Sadeh 2015-08-26 18:34:18 UTC
Can you check and verify what's actually happening on this one?