Bug 1443663 - rhosp-director: Overcloud upgrade osp10->osp11 fails on a setup upgraded from OSP9: Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'oslo::messaging::default'
Summary: rhosp-director: Overcloud upgrade osp10->osp11 fails on a setup upgraded fro...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
urgent
unspecified
Target Milestone: rc
: 11.0 (Ocata)
Assignee: Sofer Athlan-Guyot
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-19 16:24 UTC by Alexander Chuzhoy
Modified: 2017-05-17 20:23 UTC (History)
9 users (show)

Fixed In Version: openstack-tripleo-heat-templates-6.0.0-7.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-17 20:23:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1684587 0 None None None 2017-04-20 10:43:49 UTC
OpenStack gerrit 458530 0 None MERGED N->O Manual puppet commands have the right modulepath. 2020-08-29 00:24:22 UTC
Red Hat Product Errata RHEA-2017:1245 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC

Description Alexander Chuzhoy 2017-04-19 16:24:04 UTC
rhosp-director:  Overcloud upgrade osp10->osp11 fails on a setup upgraded from OSP9: Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'oslo::messaging::default'


Environment:
openstack-tripleo-heat-templates-6.0.0-5.el7ost.noarch
instack-undercloud-6.0.0-6.el7ost.noarch
openstack-tripleo-heat-templates-compat-5.2.0-7.el7ost.noarch
openstack-puppet-modules-10.0.0-1.el7ost.noarch
python-oslo-messaging-5.17.1-2.el7ost.noarch

Steps to reproduce:
1) Deploy OSP9 with:
 Deployment command: openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --ceph-storage-scale 1   --neutron-network-type vxlan --neutron-tunnel-types vxlan  --ntp-server 10.5.26.10 --timeout 90 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e network-environment.yaml


2) Upgrade the setup to OSP10

3) Attempt to upgrade to OSP11

Result:
The upgrade fails during overcloud upgrade, in step with " -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps.yaml -e init-repo.yaml"

[stack@undercloud72 ~]$ openstack stack failures list overcloud                                                                                                                               overcloud.AllNodesDeploySteps.ControllerUpgrade_Step5.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: d9f82961-e758-4f7e-903f-afa23e59c0ac
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |
    ...
    TASK [Create puppet manifest to set transport_url in nova.conf] ****************
    changed: [localhost]

    TASK [Run puppet apply to set tranport_url in nova.conf] ***********************
    fatal: [localhost]: FAILED! => {"changed": false, "cmd": ["puppet", "apply", "--detailed-exitcodes", "/root/nova-api_upgrade_manifest.pp"], "delta": "0:00:08.325284", "end": "2017-04-19 16:12:47.475946", "failed": true, "failed_when_result": true, "rc": 1, "start": "2017-04-19 16:12:39.150662", "stderr": "exception: connect failed\n\u001b[1;33mWarning: Undefined variable '::deploy_config_name'; \n   (file & line not available)\u001b[0m\n\u001b[1;31mError: Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'oslo::messaging::default' at /root/nova-api_upgrade_manifest.pp:8:4 on node overcloud-controller-0.localdomain\u001b[0m", "stdout": "", "stdout_lines": [], "warnings": []}
        to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/0a1bf20b-93eb-41fc-87c5-57414479d07b_playbook.retry

    PLAY RECAP *********************************************************************
    localhost                  : ok=19   changed=16   unreachable=0    failed=1

    (truncated, view all with --long)
  deploy_stderr: |

(reverse-i-search)`re': cat init-^Cpo.yaml    





[stack@undercloud72 ~]$ heat resource-list -n5 overcloud|grep -v COMPLE
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
+---------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+------------------------------------------------------------------------------------------------------------------------------+
| resource_name                               | physical_resource_id                                                                                                     | resource_type                                                                                                         | resource_status | updated_time         | stack_name                                                                                                                   |
+---------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+------------------------------------------------------------------------------------------------------------------------------+
| AllNodesDeploySteps                         | 8d14e21e-6c96-49dd-ae70-98b4da4afff4                                                                                     | OS::TripleO::PostDeploySteps                                                                                          | CREATE_FAILED   | 2017-04-19T15:36:40Z | overcloud                                                                                                                    |
| ControllerUpgrade_Step5                     | 0dbec2c0-43a8-4a36-996a-9cd588c72549                                                                                     | OS::Heat::SoftwareDeploymentGroup                                                                                     | CREATE_FAILED   | 2017-04-19T15:36:42Z | overcloud-AllNodesDeploySteps-j5zdjt3ghxuu                                                                                   |
| 0                                           | d9f82961-e758-4f7e-903f-afa23e59c0ac                                                                                     | OS::Heat::SoftwareDeployment                                                                                          | CREATE_FAILED   | 2017-04-19T16:11:57Z | overcloud-AllNodesDeploySteps-j5zdjt3ghxuu-ControllerUpgrade_Step5-sw275xztnlvh                                              |
+---------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+------------------------------------------------------------------------------------------------------------------------------+

Comment 1 Marius Cornea 2017-04-19 16:33:04 UTC
Looks like there's no /etc/puppet/modules/oslo/ directory on the node where it failed:

[root@overcloud-controller-0 ~]# ls -l /etc/puppet/modules/oslo/
ls: cannot access /etc/puppet/modules/oslo/: No such file or directory

Comment 3 Sofer Athlan-Guyot 2017-04-20 16:44:20 UTC
In review in stable/ocata.

Comment 4 Sofer Athlan-Guyot 2017-04-21 14:18:37 UTC
Merged in stable/ocata.

Comment 9 errata-xmlrpc 2017-05-17 20:23:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245


Note You need to log in before you can comment on or make changes to this bug.