Bug 1466219 - Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to: "The conditional check 'stat_iscsid_service.stat.exists' failed. The error was: error while evaluating conditional (stat_iscsid_service.stat.exists): 'stat_iscsid_service' is undefined"
Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to: "The conditio...
Status: VERIFIED
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
12.0 (Pike)
x86_64 Linux
urgent Severity high
: beta
: 12.0 (Pike)
Assigned To: Jiri Stransky
Marius Cornea
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-29 05:34 EDT by Artem Hrechanychenko
Modified: 2017-11-08 08:36 EST (History)
11 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170715081739.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1701221 None None None 2017-06-29 05:48 EDT
OpenStack gerrit 478842 None None None 2017-06-29 06:16 EDT

  None (edit)
Description Artem Hrechanychenko 2017-06-29 05:34:48 EDT
Description of problem:
2017-06-29 09:10:47Z [overcloud-AllNodesDeploySteps-4fk2ekopz4jx.ControllerUpgrade_Step2.0]: CREATE_FAILED  Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2017-06-29 09:10:47Z [overcloud-AllNodesDeploySteps-4fk2ekopz4jx.ControllerUpgrade_Step2]: CREATE_FAILED  Resource CREATE failed: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
2017-06-29 09:10:47Z [overcloud-AllNodesDeploySteps-4fk2ekopz4jx.ControllerUpgrade_Step2]: CREATE_FAILED  Error: resources.ControllerUpgrade_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2017-06-29 09:10:47Z [overcloud-AllNodesDeploySteps-4fk2ekopz4jx]: UPDATE_FAILED  Error: resources.ControllerUpgrade_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2017-06-29 09:10:48Z [AllNodesDeploySteps]: UPDATE_FAILED  resources.AllNodesDeploySteps: Error: resources.ControllerUpgrade_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
2017-06-29 09:10:48Z [overcloud]: UPDATE_FAILED  resources.AllNodesDeploySteps: Error: resources.ControllerUpgrade_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 2
 
 Stack overcloud UPDATE_FAILED
 
overcloud.AllNodesDeploySteps.ControllerUpgrade_Step2.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: a93e5e5c-f1c3-4caa-90a6-21913ebde429
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |
    ...
    TASK [Stop and disable horizon service (running under httpd)] ******************
    ok: [localhost]
 
    TASK [Stop and disable iscsid service] *****************************************
    fatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check 'stat_iscsid_service.stat.exists' failed. The error was: error while evaluating conditional (stat_iscsid_service.stat.exists): 'stat_iscsid_service' is undefined"}
        to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/6d340431-7033-4c95-9318-716da99aae0a_playbook.retry
 
    PLAY RECAP *********************************************************************
    localhost                  : ok=18   changed=12   unreachable=0    failed=1
 
    (truncated, view all with --long)
  deploy_stderr: |
 
Heat Stack update failed.

Version-Release number of selected component (if applicable):
OSP12
(undercloud) [stack@undercloud-0 ~]$ rpm -qa *openstack*
openstack-nova-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-keystone-12.0.0-0.20170623233743.9070172.el7ost.noarch
openstack-ironic-inspector-5.1.1-0.20170622234613.05896b5.el7ost.noarch
puppet-openstack_extras-11.2.0-0.20170613085321.0b0ea62.el7ost.noarch
openstack-tripleo-common-containers-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
openstack-neutron-common-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-neutron-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-compute-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-mistral-executor-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-tempest-16.0.1-0.20170623203531.4386df8.el7ost.noarch
openstack-neutron-openvswitch-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-placement-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-api-cfn-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-api-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-api-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
python-openstack-mistral-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-heat-common-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-tripleo-image-elements-7.0.0-0.20170607161959.401d861.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.0-0.20170614005502.9285877.el7ost.noarch
openstack-swift-object-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-neutron-ml2-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-glance-15.0.0-0.20170623215940.8188eca.el7ost.noarch
openstack-nova-scheduler-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-engine-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-engine-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-conductor-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-tripleo-common-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
python-openstackclient-3.11.0-0.20170613232431.c69304e.el7ost.noarch
openstack-ironic-common-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-swift-account-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-selinux-0.8.8-0.20170622195307.74ddc0e.el7ost.noarch
openstack-swift-container-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-nova-conductor-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-tripleo-validations-7.1.1-0.20170621111847.fb7346f.el7ost.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170624014919.el7ost.noarch
puppet-openstacklib-11.2.0-0.20170613150439.93b8e7d.el7ost.noarch
python-openstacksdk-0.9.17-0.20170621195806.7946243.el7ost.noarch
openstack-swift-proxy-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-heat-api-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-zaqar-5.0.0-0.20170623003642.68eac5a.el7ost.noarch
openstack-tripleo-ui-7.1.1-0.20170624213423.cb896fd.el7ost.noarch
openstack-puppet-modules-10.0.0-1.el7ost.noarch
openstack-mistral-common-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-nova-common-16.0.0-0.20170624031428.3863eca.el7ost.noarch


Steps to Reproduce:
1.Install OSP11
infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa --cleanup yes && infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa  --topology-nodes undercloud:1,controller:1,compute:1  -e  override.controller.cpu=8 -e override.controller.memory=16384 -e  override.undercloud.disks.disk1.size=100G && infrared tripleo-undercloud --version 11 --images-task=rpm && infrared tripleo-overcloud -v --introspect yes --tagging yes --post no --deployment-files virt --version 11 --deploy yes 

2.Upgrade undercloud and overcloud to osp11 latest+ rhel7.4
ir tripleo-undercloud -v --update-undercloud yes --mirror qeos --build 7.4-testing --osrelease 7.4 

ir tripleo-overcloud -v --updateto 7.4-testing --deployment-files virt --mirror qeos --osrelease 7.4
#Please note, during yum update stage on overcloud controller run "sudo pcs cluster start and sudo pcs cleanup resources" - w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1464588

3. Upgrade undercloud and overcloud to osp12
http://etherpad.corp.redhat.com/osp12-upgrade
#please note, before updating overcloud apply w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1460421
https://bugzilla.redhat.com/show_bug.cgi?id=1463227

Actual results:
overcloud upgrade failed

Expected results:
overcloud upgrade was succeed 


Additional info:
failures list --long overcloud - http://pastebin.test.redhat.com/498793
Comment 1 Red Hat Bugzilla Rules Engine 2017-06-29 05:34:55 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 2 Red Hat Bugzilla Rules Engine 2017-06-29 06:16:55 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 3 Red Hat Bugzilla Rules Engine 2017-06-29 09:33:42 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 4 Sofer Athlan-Guyot 2017-07-12 07:13:29 EDT
Hi,

we need pm ack on the blocker flag.

Thanks,
Comment 5 Jiri Stransky 2017-07-17 08:32:23 EDT
Merged upstream.

Note You need to log in before you can comment on or make changes to this bug.