Bug 1131980

Summary: rubygem-staypuft: When choosing deployment with QPID - Rabbit is being installed on the controller, with bad configuration that cause deployment to fail.
Product: Red Hat OpenStack Reporter: Omri Hochman <ohochman>
Component: openstack-foreman-installerAssignee: Jason Guiditta <jguiditt>
Status: CLOSED ERRATA QA Contact: Alexander Chuzhoy <sasha>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 5.0 (RHEL 6)CC: jguiditt, mburns, mlopes, morazi, ohochman, rhos-maint, yeylon
Target Milestone: z2   
Target Release: Installer   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-foreman-installer-2.0.29-1.el6ost Doc Type: Bug Fix
Doc Text:
Previously, a misplaced puppet parameter would occur when selecting Qpid as the AMQP provider. As a result, the Qpid service would not install, and OpenStack Networking would fail to start. With this update, the amqp_provider parameter is correctly passed through. Consequently, the Qpid server is installed on the controller, and OpenStack Networking services start as expected.
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-04 17:02:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Omri Hochman 2014-08-20 13:04:29 UTC
rubygem-staypuft: When choosing deployment with QPID - Rabbit is being installed on the controller, with bad configuration that cause deployment to fail. 
 

Environment :
-------------
ruby193-rubygem-staypuft-0.2.5-1.el6ost.noarch
rhel-osp-installer-0.1.10-2.el6ost.noarch
puppet-3.6.2-1.1.el6.noarch
puppet-server-3.6.2-1.1.el6.noarch
openstack-puppet-modules-2014.1-20.2.el6ost.noarch


Steps: 
-------
(1) Attempt to create using staypuft Non-HA-neutron - choose to use QPID. 
(2) run the deployment . 

Results:
---------
- Deployment failed on the controller machine because neutron-server service failed to start  - the puppet agent Error shows 

neutron-server failed to start : 
 Error: Could not start Service[neutron-server]: Execution of '/usr/bin/systemctl start neutron-server' returned 1: Job for neutron-server.service failed. See 'systemctl status neutron-server.service' and 'journalctl -xn' for details.
Wrapped exception:
Execution of '/usr/bin/systemctl start neutron-server' returned 1: Job for neutron-server.service failed. See 'systemctl status neutron-server.service' and 'journalctl -xn' for details.
Error: /Stage[main]/Neutron::Server/Service[neutron-server]/ensure: change from stopped to running failed: Could not start Service[neutron-server]: Execution of '/usr/bin/systemctl start neutron-server' returned 1: Job for neutron-server.service failed. See 'systemctl status neutron-server.service' and 'journalctl -xn' for details.


neurton-server.log 
(failed to communicate with qpid) 
-------------------
2014-08-20 10:39:56.590 16504 INFO neutron.common.config [-] Logging enabled!
2014-08-20 10:39:56.596 16504 INFO neutron.common.config [-] Config paste file: /usr/share/neutron/api-paste.ini
2014-08-20 10:39:56.683 16504 INFO neutron.manager [-] Loading core plugin: neutron.plugins.ml2.plugin.Ml2Plugin
2014-08-20 10:39:56.786 16504 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['vxlan', 'flat', 'vlan', 'gre', 'local']
2014-08-20 10:39:56.790 16504 INFO neutron.plugins.ml2.drivers.type_flat [-] Arbitrary flat physical_network names allowed
2014-08-20 10:39:56.793 16504 INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: {'physnet-external': []}
2014-08-20 10:39:56.794 16504 INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete
2014-08-20 10:39:56.818 16504 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'vlan', 'local', 'gre', 'vxlan']
2014-08-20 10:39:56.819 16504 INFO neutron.plugins.ml2.managers [-] Registered types: ['flat', 'vlan', 'local', 'gre', 'vxlan']
2014-08-20 10:39:56.819 16504 INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vxlan']
2014-08-20 10:39:56.820 16504 INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'l2population']
2014-08-20 10:39:56.822 16504 INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'l2population']
2014-08-20 10:39:56.822 16504 INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'l2population']
2014-08-20 10:39:56.836 16504 WARNING neutron.openstack.common.db.sqlalchemy.session [-] This application has not enabled MySQL traditional mode, which means silent data corruption may occur. Please encourage the application developers to enable this mode.
2014-08-20 10:39:56.904 16504 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat'
2014-08-20 10:39:56.904 16504 INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete
2014-08-20 10:39:56.904 16504 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'vlan'
2014-08-20 10:39:56.947 16504 INFO neutron.plugins.ml2.drivers.type_vlan [-] VlanTypeDriver initialization complete
2014-08-20 10:39:56.948 16504 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'local'
2014-08-20 10:39:56.948 16504 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'gre'
2014-08-20 10:39:56.948 16504 INFO neutron.plugins.ml2.drivers.type_tunnel [-] gre ID ranges: [(10, 1000)]
2014-08-20 10:39:57.203 16504 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'vxlan'
2014-08-20 10:39:57.204 16504 INFO neutron.plugins.ml2.drivers.type_tunnel [-] vxlan ID ranges: [(10, 1000)]
2014-08-20 10:39:57.457 16504 INFO neutron.plugins.ml2.managers [-] Initializing mechanism driver 'openvswitch'
2014-08-20 10:39:57.457 16504 INFO neutron.plugins.ml2.managers [-] Initializing mechanism driver 'l2population'
2014-08-20 10:39:57.462 16504 INFO neutron.plugins.ml2.plugin [req-592c575d-9976-4425-a08c-bcf35daed324 None] Modular L2 Plugin initialization complete
2014-08-20 10:39:57.463 16504 WARNING neutron.agent.securitygroups_rpc [req-592c575d-9976-4425-a08c-bcf35daed324 None] Driver configuration don't match with enable_security_group
2014-08-20 10:39:57.464 16504 INFO neutron.manager [req-592c575d-9976-4425-a08c-bcf35daed324 None] Loading Plugin: neutron.services.l3_router.l3_router_plugin.L3RouterPlugin
2014-08-20 10:39:57.652 16504 ERROR neutron.openstack.common.rpc.impl_qpid [req-592c575d-9976-4425-a08c-bcf35daed324 None] Unable to connect to AMQP server: client: 0-10, server: 9-1. Sleeping 1 seconds
2014-08-20 10:39:58.655 16504 ERROR neutron.openstack.common.rpc.impl_qpid [req-592c575d-9976-4425-a08c-bcf35daed324 None] Unable to connect to AMQP server: client: 0-10, server: 9-1. Sleeping 2 seconds
2014-08-20 10:40:00.658 16504 ERROR neutron.openstack.common.rpc.impl_qpid [req-592c575d-9976-4425-a08c-bcf35daed324 None] Unable to connect to AMQP server: client: 0-10, server: 9-1. Sleeping 3 seconds
2014-08-20 10:40:03.661 16504 ERROR neutron.openstack.common.rpc.impl_qpid [req-592c575d-9976-4425-a08c-bcf35daed324 None] Unable to connect to AMQP server: client: 0-10, server: 9-1. Sleeping 4 seconds
2014-08-20 10:40:07.665 16504 ERROR neutron.openstack.common.rpc.impl_qpid [req-592c575d-9976-4425-a08c-bcf35daed324 None] Unable to connect to AMQP server

looking at yum.log :
--------------------
Aug 20 10:30:10 Installed: python-qpid-0.18-12.el7.noarch 

QPID server is not installed on the controller . but only qpid client is installed.  


[root@maca25400702875 ~]# rpm -qa | grep rabbit
rabbitmq-server-3.1.5-6.3.el7ost.noarch


ps -ef | grep rabbit :
(rabbit is running on the machine)
-------------------------
[root@maca25400702875 ~]# ps -ef | grep rabbit
rabbitmq 13525     1  0 10:36 ?        00:00:12 /usr/lib64/erlang/erts-5.10.4/bin/beam -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/../ebin -noshell -noinput -s rabbit boot -sname rabbit@maca25400702875 -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -rabbit tcp_listeners [{"auto",5672}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit"} -rabbit enabled_plugins_file "/etc/rabbitmq/enabled_plugins" -rabbit plugins_dir "/usr/lib/rabbitmq/lib/rabbitmq_server-3.1.5/sbin/../plugins" -rabbit plugins_expand_dir "/var/lib/rabbitmq/mnesia/rabbit@maca25400702875-plugins-expand" -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@maca25400702875"
rabbitmq 13619 13525  0 10:36 ?        00:00:00 inet_gethost 4
rabbitmq 13620 13619  0 10:36 ?        00:00:00 inet_gethost 4
root     16882 16863  0 12:57 pts/0    00:00:00 grep --color=auto rabbit

Comment 4 Mike Burns 2014-10-01 19:24:30 UTC
staypuft currently sets a parameter to switch between rabbit and qpid, so this appears to be a problem in the puppet classes.

Comment 5 Jason Guiditta 2014-10-02 19:52:42 UTC
Patch posted:
https://github.com/redhat-openstack/astapor/pull/380

Omri, can you verify you hit this only on non-HA neutron controller?

Comment 6 Jason Guiditta 2014-10-07 13:06:24 UTC
merged

Comment 9 Alexander Chuzhoy 2014-10-16 17:13:58 UTC
Verified: 
rhel-osp-installer-0.4.4-1.el6ost.noarch
openstack-puppet-modules-2014.1-24.el6ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el6ost.noarch
openstack-foreman-installer-2.0.30-1.el6ost.noarch


The deployment completes successfully.
The neutron-server is up ad running on the controller.

Comment 11 Jason Guiditta 2014-10-24 14:32:18 UTC
Doc text added

Comment 13 errata-xmlrpc 2014-11-04 17:02:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2014-1800.html