Bug 1284121 - rhel-osp-director: 7.2 - HA overcloud deployment with netiso fails Error: resources.ControllerNodesPostDeployment.resources.ControllerPostPuppet.resources.ControllerPostPuppetRestartDeployment.resources[0]: Deployment to server failed"
Summary: rhel-osp-director: 7.2 - HA overcloud deployment with netiso fails Error: res...
Keywords:
Status: CLOSED DUPLICATE of bug 1285363
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 7.0 (Kilo)
Assignee: chris alfonso
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-11-20 22:23 UTC by Alexander Chuzhoy
Modified: 2016-02-01 02:38 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-25 16:50:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
/var/log dir from one controller. (3.73 MB, application/x-gzip)
2015-11-20 22:26 UTC, Alexander Chuzhoy
no flags Details

Description Alexander Chuzhoy 2015-11-20 22:23:25 UTC
rhel-osp-director: 7.2 - HA overcloud deployment with netiso fails Error: resources.ControllerNodesPostDeployment.resources.ControllerPostPuppet.resources.ControllerPostPuppetRestartDeployment.resources[0]: Deployment to server failed"

Environment:
instack-undercloud-2.1.2-34.el7ost.noarch
openstack-swift-container-2.3.0-2.el7ost.noarch
redhat-access-plugin-openstack-7.0.0-0.el7ost.noarch
openstack-tripleo-0.0.7-0.1.1664e566.el7ost.noarch
openstack-heat-engine-2015.1.2-2.el7ost.noarch
openstack-ceilometer-common-2015.1.2-1.el7ost.noarch
openstack-tripleo-heat-templates-0.8.6-79.el7ost.noarch
openstack-heat-templates-0-0.7.20150605git.el7ost.noarch
openstack-selinux-0.6.43-1.el7ost.noarch
openstack-tempest-kilo-20151020.1.el7ost.noarch
openstack-neutron-ml2-2015.1.2-2.el7ost.noarch
python-django-openstack-auth-1.2.0-5.el7ost.noarch
openstack-swift-proxy-2.3.0-2.el7ost.noarch
openstack-tripleo-puppet-elements-0.0.1-5.el7ost.noarch
openstack-ceilometer-collector-2015.1.2-1.el7ost.noarch
openstack-ironic-common-2015.1.2-2.el7ost.noarch
openstack-ceilometer-api-2015.1.2-1.el7ost.noarch
openstack-ironic-api-2015.1.2-2.el7ost.noarch
openstack-keystone-2015.1.2-2.el7ost.noarch
openstack-tuskar-ui-extras-0.0.4-2.el7ost.noarch
openstack-glance-2015.1.2-1.el7ost.noarch
openstack-heat-api-2015.1.2-2.el7ost.noarch
openstack-ceilometer-central-2015.1.2-1.el7ost.noarch
openstack-puppet-modules-2015.1.8-30.el7ost.noarch
openstack-nova-compute-2015.1.2-4.el7ost.noarch
openstack-nova-conductor-2015.1.2-4.el7ost.noarch
openstack-swift-object-2.3.0-2.el7ost.noarch
openstack-dashboard-2015.1.2-3.el7ost.noarch
openstack-heat-common-2015.1.2-2.el7ost.noarch
openstack-tuskar-0.4.18-5.el7ost.noarch
openstack-neutron-openvswitch-2015.1.2-2.el7ost.noarch
openstack-nova-api-2015.1.2-4.el7ost.noarch
openstack-nova-console-2015.1.2-4.el7ost.noarch
openstack-neutron-common-2015.1.2-2.el7ost.noarch
openstack-neutron-2015.1.2-2.el7ost.noarch
openstack-heat-api-cfn-2015.1.2-2.el7ost.noarch
openstack-ironic-conductor-2015.1.2-2.el7ost.noarch
openstack-swift-2.3.0-2.el7ost.noarch
openstack-nova-novncproxy-2015.1.2-4.el7ost.noarch
openstack-swift-account-2.3.0-2.el7ost.noarch
openstack-tuskar-ui-0.4.0-5.el7ost.noarch
openstack-utils-2014.2-1.el7ost.noarch
openstack-tripleo-common-0.0.1.dev6-4.git49b57eb.el7ost.noarch
openstack-heat-api-cloudwatch-2015.1.2-2.el7ost.noarch
openstack-ceilometer-notification-2015.1.2-1.el7ost.noarch
python-openstackclient-1.0.3-3.el7ost.noarch
openstack-ceilometer-alarm-2015.1.2-1.el7ost.noarch
openstack-swift-plugin-swift3-1.7-3.el7ost.noarch
openstack-dashboard-theme-2015.1.2-3.el7ost.noarch
openstack-ironic-discoverd-1.1.0-8.el7ost.noarch
openstack-nova-scheduler-2015.1.2-4.el7ost.noarch
openstack-nova-cert-2015.1.2-4.el7ost.noarch
openstack-nova-common-2015.1.2-4.el7ost.noarch
openstack-tripleo-image-elements-0.9.6-10.el7ost.noarch

Steps to reproduce:
Attempt to deploy HA overcloud with network isolation.

Result:

Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates
Stack failed with status: Resource CREATE failed: Error: resources.ControllerNodesPostDeployment.resources.ControllerPostPuppet.resources.ControllerPostPuppetRestartDeployment.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 1
ERROR: openstack Heat Stack create failed.


| resource_status_reason | Error: resources.ControllerNodesPostDeployment.resources.ControllerPostPuppet.resources.ControllerPostPuppetRestartDeployment.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 1 |

Seeing this error in os-collect-config:


Nov 20 16:32:15 overcloud-controller-0.localdomain os-collect-config[4484]: cemaker::Stonith/Exec[Disable STONITH]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Tripleo::Loadbalancer/Haproxy::Listen[redis]/Concat::Fragment[redis_listen_block]/File[/var/lib/puppet/concat/_etc_haproxy_haproxy.cfg/fragments/20-redis-00_redis_listen_block]/ensure: defined content as '{md5}cd28fbbf8e50285de05d3378cc28f58e'\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy::Config/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy::Config/Concat[/etc/haproxy/haproxy.cfg]/Exec[concat_/etc/haproxy/haproxy.cfg]: Triggered 'refresh' from 43 events\u001b[0m\n\u001b[mNotice: /Stage[main]/Haproxy::Config/Concat[/etc/haproxy/haproxy.cfg]/File[/etc/haproxy/haproxy.cfg]/content: content changed '{md5}1f337186b0e1ba5ee82760cb437fb810' to '{md5}5e72f14d4e4467d247cecd526bb2586b'\u001b[0m\n\u001b[mNotice: /File[/etc/haproxy/haproxy.cfg]/seluser: seluser changed 'unconfined_u' to 'system_u'\u001b[0m\n\u001b[mNotice: Finished catalog run in 48.16 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}                                                                                                                                                                                     
Nov 20 16:32:15 overcloud-controller-0.localdomain os-collect-config[4484]: [2015-11-20 16:32:15,186] (heat-config) [INFO] Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Nov 20 16:32:15 overcloud-controller-0.localdomain os-collect-config[4484]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)                                                                                                                               
Nov 20 16:36:10 overcloud-controller-0.localdomain os-collect-config[4484]: ]/Pcmk_constraint[colo-ip-192.168.120.10-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Colocation[public_vip-with-haproxy]/Pcmk_constraint[colo-ip-192.168.200.180-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[public_vip-then-haproxy]/Exec[Creating order constraint public_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Colocation[internal_api_vip-with-haproxy]/Pcmk_constraint[colo-ip-192.168.100.11-haproxy-clone]/ensure: created\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[redis_vip-then-haproxy]/Exec[Creating order constraint redis_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Pacemaker::Constraint::Base[storage_vip-then-haproxy]/Exec[Creating order constraint storage_vip-then-haproxy]/returns: executed successfully\u001b[0m\n\u001b[mNotice: Finished catalog run in 129.93 seconds\u001b[0m\n", "deploy_stderr": "\u001b[1;31mError: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n\u001b[1;31mError: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)\u001b[0m\n", "deploy_status_code": 0}
Nov 20 16:36:10 overcloud-controller-0.localdomain os-collect-config[4484]: [2015-11-20 16:36:10,677] (heat-config) [INFO] Error: Could not prefetch mysql_user provider 'mysql': Execution of '/usr/bin/mysql -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
Nov 20 16:36:10 overcloud-controller-0.localdomain os-collect-config[4484]: Error: Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql -NBe show databases' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)   


Expected results:
Successful deployment.

Comment 2 Alexander Chuzhoy 2015-11-20 22:26:50 UTC
Created attachment 1097345 [details]
/var/log dir from one controller.

Comment 3 Jiri Stransky 2015-11-25 16:50:01 UTC
In /var/log/messages of the attached log dir i can see:

Nov 20 17:11:11 localhost os-collect-config: httpd never started after 200 seconds, which is:

https://bugzilla.redhat.com/show_bug.cgi?id=1285363

*** This bug has been marked as a duplicate of bug 1285363 ***


Note You need to log in before you can comment on or make changes to this bug.