Bug 1462806

Summary: [Openstack-containers][OSP12]: Overcloud with containers Deployment failed - we should not attempt to "yum -y install python-pip\\n pip install paunch" during overcloud deployment.
Product: Red Hat OpenStack Reporter: Omri Hochman <ohochman>
Component: openstack-tripleo-heat-templatesAssignee: Steve Baker <sbaker>
Status: CLOSED ERRATA QA Contact: Omri Hochman <ohochman>
Severity: urgent Docs Contact: Andrew Burden <aburden>
Priority: urgent    
Version: 12.0 (Pike)CC: jschluet, m.andre, mburns, mcornea, rhallise, rhel-osp-director-maint, sasha
Target Milestone: Upstream M2Keywords: Triaged
Target Release: 12.0 (Pike)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170611233361.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-12-13 21:32:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1447127    
Bug Blocks:    

Description Omri Hochman 2017-06-19 15:03:04 UTC
[Openstack-containers][OSP12]: Overcloud with containers Deployment failed - we should not attempt to "yum -y install python-pip\\n pip install paunch" during overcloud deployment. 


Environment: 
------------
images from 2017-06-15.2
Description: 
-------------
the paunch should be exist and not be installed during the overcloud deployment, it might have been a workaround that needs to be remove, since paunch already exist. 

Steps:
-------
(1) Attempt to deploy osp12 with containers according to etherpad.corp.redhat.com/testing-osp12-containers

Results: 
---------
Overcloud deployment fails over : 	FAILED! => {\"changed\": true, \"cmd\": \"yum -y install python-pip\\n pip install paunch\", \"delta\": \"0:00:00.486056\", \"end\": \"2017-06-19 10:26:03.015832\", \"failed\": true, \"rc\": 127, \"start\": \"2017-06-19 10:26:02.529776\", \"stderr\": \"There are no enabled repos.\\n Run \\\"yum repolist all\\\" to see the repos you have.\\n To enable Red Hat Subscription Management repositories:\\n



[stack@undercloud-0 ~]$ heat resource-list overcloud -n 5 | grep -v COMPLETE
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
+-------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| resource_name                             | physical_resource_id                                                                                                                                                                 | resource_type                                                                                                                  | resource_status | updated_time         | stack_name                                                                                                                                                      |
+-------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| AllNodesDeploySteps                       | 230813c6-4e4f-4226-bfb6-b5e0e18698f1                                                                                                                                                 | OS::TripleO::PostDeploySteps                                                                                                   | CREATE_FAILED   | 2017-06-19T14:11:40Z | overcloud                                                                                                                                                       |
| ComputeHostPrepDeployment                 | 4a428c53-68ab-4031-8f05-11a639ad3d5d                                                                                                                                                 | OS::Heat::SoftwareDeploymentGroup                                                                                              | CREATE_FAILED   | 2017-06-19T14:25:23Z | overcloud-AllNodesDeploySteps-nybpj3t6c4oe                                                                                                                      |
| ControllerHostPrepDeployment              | 9d135ac5-49f4-4dbe-97c8-284ef40b4661                                                                                                                                                 | OS::Heat::SoftwareDeploymentGroup                                                                                              | CREATE_FAILED   | 2017-06-19T14:25:23Z | overcloud-AllNodesDeploySteps-nybpj3t6c4oe                                                                                                                      |
| 0                                         | a2f0d437-c14c-4730-b22a-acb44caf2226                                                                                                                                                 | OS::Heat::SoftwareDeployment                                                                                                   | CREATE_FAILED   | 2017-06-19T14:25:31Z | overcloud-AllNodesDeploySteps-nybpj3t6c4oe-ControllerHostPrepDeployment-cnpe6m7icel4                                                                            |
| 0                                         | dcb8ee4e-7266-4523-8d90-bef12fc0e8c9                                                                                                                                                 | OS::Heat::SoftwareDeployment                                                                                                   | CREATE_FAILED   | 2017-06-19T14:25:33Z | overcloud-AllNodesDeploySteps-nybpj3t6c4oe-ComputeHostPrepDeployment-kp5qmb7ydurk                                                                               |
+-------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------+


(undercloud) [stack@undercloud-0 ~]$ heat deployment-show dcb8ee4e-7266-4523-8d90-bef12fc0e8c9
WARNING (shell) "heat deployment-show" is deprecated, please use "openstack software deployment show" instead
{
  "status": "FAILED",
  "server_id": "f28c6396-a952-4017-ae8d-10d6b3d52033",
  "config_id": "795734ad-8a4e-4260-80a3-8edaeffefea0",
  "output_values": {
    "deploy_stdout": "\nPLAY [localhost] ***************************************************************\n\nTASK [Gathering Facts] *********************************************************\nok: [localhost]\n\nTASK [create persistent directories] *******************************************\nchanged: [localhost] => (item=/var/log/containers/nova)\nok: [localhost] => (item=/var/lib/nova)\nok: [localhost] => (item=/var/lib/libvirt)\n\nTASK [create libvirt persistent data directories] ******************************\nok: [localhost] => (item=/etc/libvirt/qemu)\nok: [localhost] => (item=/var/lib/libvirt)\nok: [localhost] => (item=/var/log/containers/nova)\n\nTASK [set enable_package_install fact] *****************************************\nok: [localhost]\n\nTASK [install libvirt-daemon] **************************************************\nskipping: [localhost]\n\nTASK [start virtlogd socket] ***************************************************\nskipping: [localhost]\n\nTASK [create persistent logs directory] ****************************************\nchanged: [localhost]\n\nTASK [Create /var/lib/tripleo-config directory] ********************************\nchanged: [localhost]\n\nTASK [Write the puppet step_config manifest] ***********************************\nchanged: [localhost]\n\nTASK [Create /var/lib/docker-puppet] *******************************************\nchanged: [localhost]\n\nTASK [Write docker-puppet-tasks json files] ************************************\nchanged: [localhost]\n\nTASK [Write docker-puppet.py] **************************************************\nchanged: [localhost]\n\nTASK [Write docker-container-startup-configs] **********************************\nchanged: [localhost]\n\nTASK [Write per-step docker-container-startup-configs] *************************\nchanged: [localhost] => (item={'key': u'step_3', 'value': {u'nova_libvirt': {u'image': u'192.168.24.1:8787/rhosp12/openstack-nova-libvirt-docker:2017-06-15.2', u'pid': u'host', u'environment': [u'KOLLA_CONFIG_STRATEGY=COPY_ALWAYS'], u'volumes': [u'/etc/hosts:/etc/hosts:ro', u'/etc/localtime:/etc/localtime:ro', u'/etc/puppet:/etc/puppet:ro', u'/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', u'/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', u'/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', u'/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', u'/dev/log:/dev/log', u'/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro', u'/var/lib/kolla/config_files/nova_libvirt.json:/var/lib/kolla/config_files/config.json:ro', u'/var/lib/config-data/nova_libvirt/etc/libvirt/:/etc/libvirt/:ro', u'/lib/modules:/lib/modules:ro', u'/dev:/dev', u'/run:/run', u'/sys/fs/cgroup:/sys/fs/cgroup', u'/var/lib/nova:/var/lib/nova', u'/var/run/libvirt:/var/run/libvirt', u'/var/lib/libvirt:/var/lib/libvirt', u'/etc/libvirt/qemu:/etc/libvirt/qemu', u'/var/log/libvirt/qemu:/var/log/libvirt/qemu:ro', u'/var/log/containers/nova:/var/log/nova'], u'net': u'host', u'privileged': True, u'restart': u'always'}}})\nchanged: [localhost] => (item={'key': u'step_4', 'value': {u'nova_compute': {u'image': u'192.168.24.1:8787/rhosp12/openstack-nova-compute-docker:2017-06-15.2', u'environment': [u'KOLLA_CONFIG_STRATEGY=COPY_ALWAYS'], u'user': u'nova', u'volumes': [u'/etc/hosts:/etc/hosts:ro', u'/etc/localtime:/etc/localtime:ro', u'/etc/puppet:/etc/puppet:ro', u'/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', u'/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', u'/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', u'/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', u'/dev/log:/dev/log', u'/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro', u'/var/lib/kolla/config_files/nova_compute.json:/var/lib/kolla/config_files/config.json:ro', u'/var/lib/config-data/nova_libvirt/etc/nova/:/etc/nova/:ro', u'/dev:/dev', u'/etc/iscsi:/etc/iscsi', u'/lib/modules:/lib/modules:ro', u'/run:/run', u'/var/lib/nova:/var/lib/nova', u'/var/lib/libvirt:/var/lib/libvirt', u'/var/log/containers/nova:/var/log/nova'], u'net': u'host', u'privileged': True, u'restart': u'always'}, u'ceilometer_agent_compute': {u'image': u'192.168.24.1:8787/rhosp12/openstack-ceilometer-compute-docker:2017-06-15.2', u'environment': [u'KOLLA_CONFIG_STRATEGY=COPY_ALWAYS'], u'volumes': [u'/etc/hosts:/etc/hosts:ro', u'/etc/localtime:/etc/localtime:ro', u'/etc/puppet:/etc/puppet:ro', u'/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', u'/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', u'/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', u'/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', u'/dev/log:/dev/log', u'/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro', u'/var/lib/kolla/config_files/ceilometer_agent_compute.json:/var/lib/kolla/config_files/config.json:ro', u'/var/lib/config-data/ceilometer/etc/ceilometer/:/etc/ceilometer/:ro', u'/var/run/libvirt:/var/run/libvirt:ro'], u'net': u'host', u'privileged': False, u'restart': u'always'}, u'neutron_ovs_agent': {u'image': u'192.168.24.1:8787/rhosp12/openstack-neutron-openvswitch-agent-docker:2017-06-15.2', u'pid': u'host', u'environment': [u'KOLLA_CONFIG_STRATEGY=COPY_ALWAYS'], u'volumes': [u'/etc/hosts:/etc/hosts:ro', u'/etc/localtime:/etc/localtime:ro', u'/etc/puppet:/etc/puppet:ro', u'/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', u'/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', u'/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', u'/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', u'/dev/log:/dev/log', u'/etc/ssh/ssh_known_hosts:/etc/ssh/ssh_known_hosts:ro', u'/var/lib/kolla/config_files/neutron_ovs_agent.json:/var/lib/kolla/config_files/config.json:ro', u'/var/lib/config-data/neutron/etc/neutron/:/etc/neutron/:ro', u'/lib/modules:/lib/modules:ro', u'/run:/run', u'/var/log/containers/neutron:/var/log/neutron'], u'net': u'host', u'privileged': True, u'restart': u'always'}}})\n\nTASK [Create /var/lib/kolla/config_files directory] ****************************\nchanged: [localhost]\n\nTASK [Write kolla config json files] *******************************************\nchanged: [localhost] => (item={'key': u'/var/lib/kolla/config_files/neutron_ovs_agent.json', 'value': {u'command': u'/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini --config-file /etc/neutron/plugins/ml2/ml2_conf.ini', u'permissions': [{u'owner': u'neutron:neutron', u'path': u'/var/log/neutron', u'recurse': True}]}})\nchanged: [localhost] => (item={'key': u'/var/lib/kolla/config_files/nova_libvirt.json', 'value': {u'command': u'/usr/sbin/libvirtd --config /etc/libvirt/libvirtd.conf', u'permissions': [{u'owner': u'nova:nova', u'path': u'/var/log/nova', u'recurse': True}]}})\nchanged: [localhost] => (item={'key': u'/var/lib/kolla/config_files/nova_compute.json', 'value': {u'command': u'/usr/bin/nova-compute --config-file /etc/nova/nova.conf --config-file /etc/nova/rootwrap.conf', u'permissions': [{u'owner': u'nova:nova', u'path': u'/var/log/nova', u'recurse': True}, {u'owner': u'nova:nova', u'path': u'/var/lib/nova', u'recurse': True}]}})\nchanged: [localhost] => (item={'key': u'/var/lib/kolla/config_files/ceilometer_agent_compute.json', 'value': {u'command': u'/usr/bin/ceilometer-polling --polling-namespaces compute'}})\n\nTASK [Install paunch FIXME remove when packaged] *******************************\nfatal: [localhost]: FAILED! => {\"changed\": true, \"cmd\": \"yum -y install python-pip\\n pip install paunch\", \"delta\": \"0:00:00.486056\", \"end\": \"2017-06-19 10:26:03.015832\", \"failed\": true, \"rc\": 127, \"start\": \"2017-06-19 10:26:02.529776\", \"stderr\": \"There are no enabled repos.\\n Run \\\"yum repolist all\\\" to see the repos you have.\\n To enable Red Hat Subscription Management repositories:\\n     subscription-manager repos --enable <repo>\\n To enable custom repositories:\\n     yum-config-manager --enable <repo>\\n/bin/sh: line 1: pip: command not found\", \"stderr_lines\": [\"There are no enabled repos.\", \" Run \\\"yum repolist all\\\" to see the repos you have.\", \" To enable Red Hat Subscription Management repositories:\", \"     subscription-manager repos --enable <repo>\", \" To enable custom repositories:\", \"     yum-config-manager --enable <repo>\", \"/bin/sh: line 1: pip: command not found\"], \"stdout\": \"Loaded plugins: product-id, search-disabled-repos, subscription-manager\\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\", \"stdout_lines\": [\"Loaded plugins: product-id, search-disabled-repos, subscription-manager\", \"This system is not registered with an entitlement server. You can use subscription-manager to register.\"]}\n\tto retry, use: --limit @/var/lib/heat-config/heat-config-ansible/795734ad-8a4e-4260-80a3-8edaeffefea0_playbook.retry\n\nPLAY RECAP *********************************************************************\nlocalhost                  : ok=14   changed=11   unreachable=0    failed=1   \n\n",
    "deploy_stderr": " [WARNING]: Consider using yum module rather than running yum\n",
    "deploy_status_code": 2
  },
  "creation_time": "2017-06-19T14:25:34Z",
  "updated_time": "2017-06-19T14:26:04Z",
  "input_values": {},
  "action": "CREATE",
  "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 2",
  "id": "dcb8ee4e-7266-4523-8d90-bef12fc0e8c9"
}

Comment 2 Red Hat Bugzilla Rules Engine 2017-06-19 15:04:30 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 4 Martin André 2017-06-19 15:24:58 UTC
It appears the overcloud-full image already includes python-paunch rpm. So we just need to import the commit https://github.com/openstack/tripleo-heat-templates/commit/30bf499eae10efc180ce68ebc3891fcab59875d5 downstream.

Comment 5 Alexander Chuzhoy 2017-06-19 18:23:14 UTC
The w/a suggested in comment #4 works. Was able to deploy overcloud.

Comment 7 Omri Hochman 2017-06-21 12:11:07 UTC
unable to reproduce with openstack-tripleo-heat-templates-7.0.0-0.20170611233361.el7ost.noarch

Comment 14 errata-xmlrpc 2017-12-13 21:32:50 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462