Bug 966094

Summary: Openstack Installer: packstack fail to configure horizon to work with SSL
Product: Red Hat OpenStack Reporter: Nir Magnezi <nmagnezi>
Component: openstack-packstackAssignee: Ivan Chavero <ichavero>
Status: CLOSED ERRATA QA Contact: Nir Magnezi <nmagnezi>
Severity: medium Docs Contact:
Priority: high    
Version: 3.0CC: aortega, apevec, derekh, ichavero, jkt, mlopes, mmagr
Target Milestone: z1Keywords: Regression, TestOnly, ZStream
Target Release: 4.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-packstack-2013.2.1-0.14.dev919.el6ost Doc Type: Known Issue
Doc Text:
Dashboard SSL configuration is not applied when specified in the Packstack answer file. This behaviour is due to Nagios deleting Apache's SSL configuration. The workaround is to disable Nagios at installation time: "packstack --os-horizon-ssl=y --nagios-install=n --allinone"
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-23 14:21:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 955316, 1054498, 1061689    
Attachments:
Description Flags
packstack logs (DEBUG)
none
ssl dashboard screenshot none

Description Nir Magnezi 2013-05-22 13:05:39 UTC
Description of problem:
=======================
packstack fail to configure horizon to work with SSL.
Horizon accepts connections on port 80 instead of port 443.

Version-Release number of selected component (if applicable):
=============================================================
openstack-packstack-2013.1.1-0.6.dev538.el6ost.noarch

How reproducible:
=================
1/1

Steps to Reproduce:
===================
1. Use packstack to install openstack:
   answers file:  CONFIG_HORIZON_SSL=y
   CONFIG_GLANCE_INSTALL=y
   CONFIG_CINDER_INSTALL=y
   CONFIG_NOVA_INSTALL=y
   CONFIG_HORIZON_INSTALL=y
   CONFIG_SWIFT_INSTALL=y
   CONFIG_CLIENT_INSTALL=y
   CONFIG_NTP_SERVERS=<IP_Address>
   CONFIG_NAGIOS_INSTALL=y
2.

Actual results:
===============
httpd listen to port 80

Expected results:
=================
httpd should listen to port 443

Comment 6 Nir Magnezi 2013-11-11 14:48:21 UTC
Tested NVR: openstack-packstack-2013.2.1-0.9.dev840.el6ost.noarch

Followed Comment #0

Result:
=======
httpd listen both to port 80 and 443

# netstat -ntpl | grep httpd
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      3362/httpd          
tcp        0      0 :::443                      :::*                        LISTEN      3362/httpd

Comment 7 Ivan Chavero 2013-11-12 07:37:21 UTC
SSL is not enabled by default in horizon, you have to set CONFIG_HORIZON_SSL=y or --os-horizon-ssl=y

Comment 8 Nir Magnezi 2013-11-12 08:09:18 UTC
(In reply to Ivan Chavero from comment #7)
> SSL is not enabled by default in horizon, you have to set
> CONFIG_HORIZON_SSL=y or --os-horizon-ssl=y

I did.

Comment 9 Ivan Chavero 2013-11-12 08:53:49 UTC
I found the problem, fixing it right now

Comment 10 Ivan Chavero 2013-11-12 18:19:23 UTC
Tested with this command line: '--allinone --use-epel=y --os-horizon-ssl=y' twice and everything works fine.

did you try to enter the dashboard via web browser?

an nmap scan gives me this:

 nmap 192.168.100.197

Starting Nmap 6.40 ( http://nmap.org ) at 2013-11-12 11:16 MST
Nmap scan report for 192.168.100.197
Host is up (0.71s latency).
Not shown: 997 filtered ports
PORT     STATE  SERVICE
22/tcp   open   ssh
443/tcp  open   https
8080/tcp closed http-proxy

Comment 11 Nir Magnezi 2013-11-13 12:03:15 UTC
(In reply to Ivan Chavero from comment #10)
> Tested with this command line: '--allinone --use-epel=y --os-horizon-ssl=y'
> twice and everything works fine.
> 
> did you try to enter the dashboard via web browser?

yup.

> 
> an nmap scan gives me this:
> 
>  nmap 192.168.100.197
> 
> Starting Nmap 6.40 ( http://nmap.org ) at 2013-11-12 11:16 MST
> Nmap scan report for 192.168.100.197
> Host is up (0.71s latency).
> Not shown: 997 filtered ports
> PORT     STATE  SERVICE
> 22/tcp   open   ssh
> 443/tcp  open   https
> 8080/tcp closed http-proxy


This is different from what I saw.
I tried to reproduce this once again (with the same packstack version) and now I see httpd only listen to port 80


How I installed it: packstack --allinone --os-horizon-ssl=y -d
I did not use EPEL since I installed RHOS.

Comment 14 Nir Magnezi 2013-11-13 13:27:01 UTC
I ran packstack for the second time, using the same answer file as a used in Comment #11 (allinone) only this time i changed to IP of horizon to a different node.

Results are the same as in Comment #6


# netstat -ntpl | grep httpd
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      4782/httpd          
tcp        0      0 :::443                      :::*                        LISTEN      4782/httpd  


both http and https are available via browser.

Comment 15 Alvaro Lopez Ortega 2013-11-13 18:55:14 UTC
The 443 port seems to have been bound only to the IPv6 interface.

Comment 16 Ivan Chavero 2013-11-14 10:56:57 UTC
yes, the netstat command gives me just the ipv6 address also, but all my tests tell me that the server is listening on port 443 on ipv4.

can you try this command?

openssl s_client -connect 192.168.100.165:443

you should get an output like this:

CONNECTED(00000003)
depth=0 C = --, ST = State, L = City, O = openstack, OU = packstack, CN = rhel6.5, emailAddress = admin
verify error:num=18:self signed certificate
verify return:1
depth=0 C = --, ST = State, L = City, O = openstack, OU = packstack, CN = rhel6.5, emailAddress = admin
verify return:1
---
Certificate chain


I noticed that the generated configuration in /etc/httpd/conf.d/ssl.conf is this


SSLCertificateKeyFile /etc/pki/tls/private/ssl_ps_server.key
Listen 443
SSLEngine on
SSLCertificateFile /etc/pki/tls/certs/ssl_ps_server.crt


The server will listen in all interfaces so that's why it could be showing the ipv6 interface.

I can change the Listen line to: Listen 0.0.0.0:443 to make it just listen to ipv4.

Comment 17 Ivan Chavero 2013-11-14 21:22:43 UTC
The puppet horizon package is adding a Listen 0.0.0.0:80 that's why it's appearing in the netstat even through we're commenting a Listen 80 line in packstack's manifests. I'm correcting this behaviour. Letting the Listen 443 so it listens in all interfaces

Comment 18 Alvaro Lopez Ortega 2013-12-03 17:23:37 UTC
Merged

Comment 21 Nir Magnezi 2013-12-17 09:43:30 UTC
Tested NVR: openstack-packstack-2013.2.1-0.20.dev936.el6ost.noarch

Configured CONFIG_HORIZON_SSL=y , yet I ended up with httpd that listens to port 80

[root@puma03 ~]# netstat -ntpl | grep httpd
tcp        0      0 :::80                       :::*                        LISTEN      7074/httpd

Comment 22 Nir Magnezi 2013-12-17 09:48:26 UTC
Created attachment 837613 [details]
packstack logs (DEBUG)

packstack logs

Comment 25 Ivan Chavero 2013-12-18 14:53:45 UTC
Succesfully tested with openstack-packstack-2013.2.1-0.18.dev934.el6ost.noarch.rpm 
testing again with: openstack-packstack-2013.2.1-0.20.dev936.el6ost.noarch

Comment 28 Nir Magnezi 2014-01-14 15:13:46 UTC
re-opening.

Tested NVR: openstack-packstack-2013.2.1-0.22.dev956.el6ost.noarch
Puddle: 2014-01-13.5

Test Steps:
===========
1. Configured packstack answer file to enable horizon SSL

# grep -i HORIZON_SSL ANSWER_FILE 
CONFIG_HORIZON_SSL=y

2. ran packstack using that answer file

Result:
=======

# netstat -ntpl
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name 
tcp        0      0 :::80                       :::*                        LISTEN      6912/httpd

Comment 29 Ivan Chavero 2014-01-15 05:45:59 UTC
Created attachment 850336 [details]
ssl dashboard screenshot

Comment 30 Ivan Chavero 2014-01-15 19:57:02 UTC
Installed: openstack-packstack-2013.2.1-0.22.dev956.el6ost

created answer file:

packstack --gen-answer-file

edit and set: CONFIG_HORIZON_SSL=y

run packstack -d --answer-file=ans.txt

after a succesful installation check:

$ curl --insecure https://192.168.100.169
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://192.168.100.169/dashboard/">here</a>.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at 192.168.100.169 Port 443</address>
</body></html>
[ichavero@cloud qpid]$ curl --insecure https://192.168.100.169/dashboard


<!DOCTYPE html>
<html lang="en" xml:lang="en">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
    <title>Login - OpenStack Dashboard</title>
    

<link rel="stylesheet" href="/static/dashboard/css/291184af72d2.css" type="text/css" media="screen" />

<link rel="shortcut icon" href="/static/dashboard/img/favicon.ico"/>

  </head>
  <body id="splash">
    <div class="container">
      <div class="row large-rounded">
        <div id="" class="login ">
  <div class="modal-header">
    
    <h3>Log In</h3>
  </div>
  
  <form id="" autocomplete="on" class="" action="/dashboard/auth/login/" method="POST"  ><div style='display:none'><input type='hidden' name='csrfmiddlewaretoken' value='ItVQzMDqRVeS55fPCHy6Sk6iQmeE5BW5' /></div>
    <div class="modal-body clearfix">
    
  <fieldset>
    
    
    
    
  <input type="hidden" name="region" value="http://192.168.100.169:5000/v2.0" id="id_region" />



  <div class="control-group form-field clearfix ">
    <label for="id_username">User Name</label>
    
    <span class="help-block"></span>
    <div class="input">
      <input type="text" name="username" id="id_username" />
    </div>
  </div>

  <div class="control-group form-field clearfix ">
    <label for="id_password">Password</label>
    
    <span class="help-block"></span>
    <div class="input">
      <input type="password" name="password" id="id_password" />
    </div>
  </div>


  </fieldset>

    </div>
    <div class="modal-footer">
  <button type="submit" class="btn btn-primary pull-right">Sign In</button>
</div>
  </form>
</div>


      </div>
    </div>
  </body>
</html>

Comment 31 Nir Magnezi 2014-01-16 12:05:27 UTC
re-opening.

Tested NVR: openstack-packstack-2013.2.1-0.22.dev956.el6ost.noarch
RHOS Puddle: 2014-01-13.5


Tested as follows:

1. packstack --os-horizon-ssl=y --allinone -d
2. post installation, Verified the created answer file:

# grep HORIZON_SSL packstack-answers-20140116-130956.txt 
CONFIG_HORIZON_SSL=y

3. check the port that httpd listens to: (should be 443)
# netstat -ntpl | grep httpd
tcp        0      0 :::80                       :::*                        LISTEN      23468/httpd    

4. Followed a step from comment #30:

checked https (failed):
[root@nmagnezi-os-allinone ~]# curl --insecure https://<IP_ADDRESS>
curl: (7) couldn't connect to host

checked http:
[root@nmagnezi-os-allinone ~]# curl --insecure http://<IP_ADDRESS>
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://<IP_ADDRESS>/dashboard/">here</a>.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at <IP_ADDRESS> Port 80</address>
</body></html>


I installed 4 setups and got the same results (port 80 and not 443)

Comment 33 Ivan Chavero 2014-01-16 17:24:37 UTC
Testing the same way as #31

Comment 34 Martin Magr 2014-01-16 18:12:22 UTC
Finalizing...                                          [ DONE ]

 **** Installation completed successfully ******


Additional information:
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * File /root/keystonerc_admin has been created on OpenStack client host 192.168.122.25. To use the command line tools you need to source the file.
 * Copy of keystonerc_admin file has been created for non-root user in /home/para.
 * NOTE : A certificate was generated to be used for ssl, You should change the ssl certificate configured in /etc/httpd/conf.d/ssl.conf on 192.168.122.25 to use a CA signed cert.
 * To access the OpenStack Dashboard browse to https://192.168.122.25/dashboard.
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * The installation log file is available at: /var/tmp/packstack/20140116-183214-3C8SYf/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20140116-183214-3C8SYf/manifests
[para@virtual-rhel ~]$ curl --insecure https://192.168.122.25
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://192.168.122.25/dashboard/">here</a>.</p>
<hr>
<address>Apache/2.2.15 (Red Hat) Server at 192.168.122.25 Port 443</address>
</body></html>
[para@virtual-rhel ~]$ curl --insecure https://192.168.122.25/dashboard


<!DOCTYPE html>
<html lang="en" xml:lang="en">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8" />
    <title>Login - OpenStack Dashboard</title>
    

<link rel="stylesheet" href="/static/dashboard/css/291184af72d2.css" type="text/css" media="screen" />

<link rel="shortcut icon" href="/static/dashboard/img/favicon.ico"/>
....
[para@virtual-rhel ~]$ sudo netstat -ntpl | grep httpd
tcp        0      0 :::443                      :::*                        LISTEN      16984/httpd         
[para@virtual-rhel ~]$ 


I don't know what you are doing with your VM, but SSL works for me too.

Comment 35 Martin Magr 2014-01-16 18:14:47 UTC
Forgot to put version.

[para@virtual-rhel ~]$ rpm -q openstack-packstack
openstack-packstack-2013.2.1-0.29.dev956.el6.noarch

Comment 36 Lon Hohberger 2014-01-16 20:35:50 UTC
The disconnect with how it's tested.

I tested with:
   packstack --allinone --os-horizon-ssl=y
   -> failed, as Nir shows.

I took the answer-file from the above run, reimaged the VM and ran this after re-imaging:
   packstack --answer-file=p-a.txt
   -> worked, as Ivan and Martin show

Very peculiar.

Comment 37 Lon Hohberger 2014-01-16 20:45:53 UTC
Actually, I was incorrect.

I didn't recheck the initial case (packstack --allinone --os...), but the latter case was back to port 80 and not 443 after reboot.

Comment 38 Lon Hohberger 2014-01-16 22:10:19 UTC
I rechecked the initial case (just packstack --allinone --os-horizon-ssl=y) - definitely only listening on port 80.

I'm rechecking using the answer-file now.

Comment 39 Lon Hohberger 2014-01-16 23:11:20 UTC
Found it.  It was a disconnect as to how it was tested, but not what I originally thought:

* Horizon correctly sets up SSL
* Nagios tears it down.

Thus, presently, SSL with Horizon is mutually exclusive with enabling Nagios.

If you run packstack with Horizon SSL enabled and Nagios disabled, Horizon gets to keep SSL.

Comment 40 Lon Hohberger 2014-01-16 23:23:31 UTC
I filed bug 1054498 to account for the Horizon/Nagios apache configuration conflict.

The following should work fine to verify this specific issue:

  packstack --os-horizon-ssl=y --nagios-install=n --allinone

Comment 42 Ivan Chavero 2014-01-18 00:37:02 UTC
Added doc text as known issue, fixing it in: bug 1054498

Comment 43 Nir Magnezi 2014-01-19 07:27:29 UTC
(In reply to Lon Hohberger from comment #40)
> I filed bug 1054498 to account for the Horizon/Nagios apache configuration
> conflict.
> 
> The following should work fine to verify this specific issue:
> 
>   packstack --os-horizon-ssl=y --nagios-install=n --allinone

Verified with openstack-packstack-2013.2.1-0.22.dev956.el6ost.noarch
Followed Lon's steps.

Comment 46 Lon Hohberger 2014-02-04 17:19:07 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2014-0046.html