Bug 1584951
| Summary: | FFU: openstack overcloud ffwd-upgrade prepare fails with: u'message': u'Init Minor update failed' | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Marius Cornea <mcornea> | ||||||||
| Component: | openstack-tripleo-common | Assignee: | mathieu bultel <mbultel> | ||||||||
| Status: | CLOSED ERRATA | QA Contact: | Marius Cornea <mcornea> | ||||||||
| Severity: | urgent | Docs Contact: | |||||||||
| Priority: | urgent | ||||||||||
| Version: | 13.0 (Queens) | CC: | ccamacho, dbecker, jschluet, mbracho, mbultel, mburns, morazi, nbarcet, slinaber | ||||||||
| Target Milestone: | rc | Keywords: | Triaged | ||||||||
| Target Release: | 13.0 (Queens) | ||||||||||
| Hardware: | All | ||||||||||
| OS: | All | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | openstack-tripleo-common-8.6.1-19.el7ost | Doc Type: | If docs needed, set a value | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2018-06-27 13:57:12 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Created attachment 1446498 [details]
ffwd prepare log
Created attachment 1446500 [details]
sosreport
Created attachment 1446501 [details]
stack.tar.gz containing all environment files
Error from /var/log/mistral/executor.log:
ERROR: The Stack (ef48bd3a-8db4-40b3-b22a-c4595df81ab5) could not be found.: HTTPNotFound: ERROR: The Stack (ef48bd3a-8db4-40b3-b22a-c4595df81ab5) could not be found.
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor Traceback (most recent call last):
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/mistral/executors/default_executor.py", line 114, in run_action
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor result = action.run(action_ctx)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/actions/config.py", line 50, in run
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor config_path = config.download_config(self.container, self.config_dir)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/tripleo_common/utils/config.py", line 243, in download_config
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor deployment_stack_id, resolve_outputs=False)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/v1/stacks.py", line 280, in get
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor resp = self.client.get('/stacks/%s' % stack_id, **kwargs)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 286, in get
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor return self.client_request("GET", url, **kwargs)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 279, in client_request
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor resp, body = self.json_request(method, url, **kwargs)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 268, in json_request
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor resp = self._http_request(url, method, **kwargs)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor File "/usr/lib/python2.7/site-packages/heatclient/common/http.py", line 231, in _http_request
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor raise exc.from_response(resp)
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor HTTPNotFound: ERROR: The Stack (ef48bd3a-8db4-40b3-b22a-c4595df81ab5) could not be found.
2018-05-31 22:48:43.923 1475 ERROR mistral.executors.default_executor
I applied https://review.openstack.org/#/c/571765/1 and wasn't able to reproduce the issue reported initially. Hi Maria, can you PM_ack this one, I need to build the package for this blocker. Thank you very much. Mathieu For creating the package we need the blocker flag accepted openstack-tripleo-common-8.6.1-19.el7ost.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2018:2086 |
Description of problem: FFU: openstack overcloud ffwd-upgrade prepare fails with: u'message': u'Init Minor update failed': 2018-06-01 02:47:45Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.ControllerPrePuppet]: DELETE_IN_PROGRESS state changed 2018-06-01 02:47:46Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya-ControllerPrePuppet-6chp5nf3gnro]: DELETE_IN_PROGRESS Stack DELETE started 2018-06-01 02:47:46Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya-ControllerPrePuppet-6chp5nf3gnro.ControllerPrePuppetMaintenanceModeDeployment]: DELETE_IN_PROGRESS state changed 2018-06-01 02:47:46Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.CephStorageConfig]: DELETE_IN_PROGRESS state changed 2018-06-01 02:47:46Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya-CephStorageConfig-6anashbrrrau]: DELETE_IN_PROGRESS Stack DELETE started 2018-06-01 02:47:46Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya-CephStorageConfig-6anashbrrrau.CephStoragePuppetConfigImpl]: DELETE_IN_PROGRESS state changed 2018-06-01 02:47:47Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.ControllerConfig]: DELETE_COMPLETE state changed 2018-06-01 02:47:47Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.ComputeConfig]: DELETE_COMPLETE state changed 2018-06-01 02:47:47Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.CephStorageConfig]: DELETE_COMPLETE state changed 2018-06-01 02:47:51Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya.ControllerPrePuppet]: DELETE_COMPLETE state changed 2018-06-01 02:47:51Z [overcloud-AllNodesDeploySteps-gu5fxv3wdfya]: UPDATE_COMPLETE Stack UPDATE completed successfully 2018-06-01 02:47:52Z [AllNodesDeploySteps]: UPDATE_COMPLETE state changed 2018-06-01 02:47:55Z [overcloud]: UPDATE_COMPLETE Stack UPDATE completed successfully Stack overcloud UPDATE_COMPLETE Started Mistral Workflow tripleo.package_update.v1.get_config. Execution ID: e0327a73-d609-4182-9b81-894aa4759c55 Waiting for messages on queue 'tripleo' with no timeout. {u'execution': {u'created_at': u'2018-06-01 02:47:58', u'id': u'e0327a73-d609-4182-9b81-894aa4759c55', u'input': {u'container': u'overcloud', u'queue_name': u'tripleo'}, u'name': u'tripleo.package_update.v1.get_config', u'params': {u'namespace': u''}, u'spec': {u'input': [u'container', {u'queue_name': u'tripleo'}], u'name': u'get_config', u'tags': [u'tripleo-common-managed'], u'tasks': {u'get_config': {u'action': u'tripleo.config.get_overcloud_config container=<% $.container %>', u'name': u'get_config', u'on-complete': u'send_message', u'publish': {u'message': u'<% task().result %>', u'status': u'SUCCESS'}, u'publish-on-error': {u'message': u'Init Minor update failed', u'status': u'FAILED'}, u'type': u'direct', u'version': u'2.0'}, u'send_message': {u'action': u'zaqar.queue_post', u'input': {u'messages': {u'body': {u'payload': {u'execution': u'<% execution() %>', u'message': u"<% $.get('message', '') %>", u'status': u"<% $.get('status', 'SUCCESS') %>"}, u'type': u'tripleo.package_update.v1.package_update_plan'}}, u'queue_name': u'<% $.queue_name %>'}, u'name': u'send_message', u'on-success': [{u'fail': u'<% $.get(\'status\') = "FAILED" %>'}], u'type': u'direct', u'version': u'2.0'}}, u'version': u'2.0'}, u'updated_at': u'2018-06-01 02:47:58'}, u'message': u'Init Minor update failed', u'status': u'FAILED'} Version-Release number of selected component (if applicable): python-tripleoclient-9.2.1-12.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy OSP10 with 3 controller + 2 compute nodes + 3 ceph OSD nodes 2. Upgrade undercloud to OSP11/12/13 3. Convert deprecated parameters to their new correspondents according to https://doc-stage.usersys.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html-single/fast_forward_upgrades/#preparing_for_deprecated_parameters Parameters converted: OvercloudControlFlavor, NovaComputeExtraConfig, NovaComputeSchedulerHints, NovaComputeIPs 4. Run openstack overcloud ffwd-upgrade prepare: openstack overcloud ffwd-upgrade prepare --templates /usr/share/openstack-tripleo-heat-templates \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-management.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-ip.yaml \ -e ~/openstack_deployment/environments/enable-cpu-pinning.yaml \ -e ~/openstack_deployment/environments/nodes.yaml \ -e ~/openstack_deployment/environments/network-environment.yaml \ -e ~/openstack_deployment/environments/disk-layout.yaml \ -e ~/openstack_deployment/environments/public_vip.yaml \ -e ~/openstack_deployment/environments/enable-tls.yaml \ -e ~/openstack_deployment/environments/inject-trust-anchor.yaml \ -e ~/openstack_deployment/environments/scheduler_hints_env.yaml \ -e ~/openstack_deployment/environments/ips-from-pool-all.yaml \ -e ~/openstack_deployment/environments/neutron-settings.yaml \ -e ~/openstack_deployment/environments/custom_hiera.yaml \ -e $HOME/ffu_repos.yaml \ --container-registry-file /home/stack/virt/docker-images.yaml \ Actual results: Fails. Expected results: Doesn't fail. Additional info: Attaching sosreport and ffwd prepare output.