Bug 1284446

Summary: S3 and EC2 services are returning empty responses on HA
Product: Red Hat OpenStack Reporter: Daniel Mellado <dmellado>
Component: python-rdomanager-oscpluginAssignee: RHOS Maint <rhos-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Shai Revivo <srevivo>
Severity: high Docs Contact:
Priority: low    
Version: 8.0 (Liberty)CC: apevec, aschultz, chkumar, dmellado, hbrock, jcoufal, jslagle, mburns, rhel-osp-director-maint, sgordon, svanders, tkammer
Target Milestone: ---Keywords: Automation, AutomationBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-04-21 12:43:30 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Daniel Mellado 2015-11-23 11:04:37 UTC
Description of problem:
When running tempest on an ospd installation, the boto section is missing. At first, due to it was missing, it was trying to connect localhost, which led to connecting refused. As the tempest-config script is disconvering the services, this lead to the following investigation:

Controller 0
============

[heat-admin@overcloud-controller-0 ~]$ sudo egrep -r '(ec2|s3)_listen' /etc/nova
/etc/nova/nova.conf:#s3_listen=0.0.0.0
/etc/nova/nova.conf:#s3_listen_port=3333
/etc/nova/nova.conf:#ec2_listen=0.0.0.0
/etc/nova/nova.conf:ec2_listen=10.19.94.14
/etc/nova/nova.conf:#ec2_listen_port=8773
/etc/nova/nova.conf:ec2_listen_port=8773
[heat-admin@overcloud-controller-0 ~]$ sudo netstat -ntlp |egrep '(3333|8773)'
tcp        0      0 10.19.94.11:8773        0.0.0.0:*               LISTEN      29279/haproxy       
tcp        0      0 10.19.184.180:8773      0.0.0.0:*               LISTEN      29279/haproxy       

When using the VIP for the HAProxy, the response is empty:
(py27)[stack@undercloud tempest]$ curl http://10.19.184.180:8773/services/Cloud
curl: (52) Empty reply from server



Version-Release number of selected component (if applicable):


How reproducible:
Add the proper boto section to the tempest.conf file
[boto]
ec2_url = http://10.19.184.180:8773/services/Cloud
s3_url = http://10.19.184.180:3333

Launch any ec2 tests
  File "/home/stack/tempest/.tox/py27/lib/python2.7/site-packages/httplib2/__init__.py", line 1307, in _conn_request
    response = conn.getresponse()
  File "/usr/lib64/python2.7/httplib.py", line 1051, in getresponse
    response.begin()
  File "/usr/lib64/python2.7/httplib.py", line 415, in begin
    version, status, reason = self._read_status()
  File "/usr/lib64/python2.7/httplib.py", line 379, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''


Actual results:
Empty response


Expected results:
To get a valid response


Additional info:

Comment 2 Mike Burns 2016-04-07 20:57:01 UTC
This bug did not make the OSP 8.0 release.  It is being deferred to OSP 10.

Comment 6 James Slagle 2016-10-14 15:15:10 UTC
Removing from OSP10, not a blocker/regression

Comment 7 Sven Anderson 2017-02-10 14:05:56 UTC
AFAIK S3 support in EC2-API was always experimental and not meant for production use.

See for example here: https://review.openstack.org/#/c/354732/

So, does this problem exist independently of S3?

Comment 8 Sven Anderson 2017-02-10 14:14:29 UTC
Actually, that bug is so old, does it actually apply for OSP11? OSP10 didn't have EC2API support in OSP Director.

Comment 9 Daniel Mellado 2017-02-27 12:06:57 UTC
@Seven, nah, this shouldn't apply at all to OSP10/OSP11. Even the S3 tests from tempest were removed.