Bug 1460421 - Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to : ERROR! no action detected in task
Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed due to : ERROR! no ac...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates (Show other bugs)
12.0 (Pike)
x86_64 Linux
high Severity urgent
: beta
: 12.0 (Pike)
Assigned To: Jiri Stransky
Gurenko Alex
: Triaged
Depends On:
Blocks: 1399762
  Show dependency treegraph
 
Reported: 2017-06-10 05:05 EDT by Artem Hrechanychenko
Modified: 2017-12-13 16:31 EST (History)
12 users (show)

See Also:
Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170706121722.el7ost openstack-tripleo-puppet-elements-7.0.0-0.20170703005458.0684eb4.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-12-13 16:31:55 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1701208 None None None 2017-06-29 05:55 EDT
OpenStack gerrit 478815 None None None 2017-06-29 05:56 EDT
OpenStack gerrit 478830 None None None 2017-06-29 05:58 EDT

  None (edit)
Description Artem Hrechanychenko 2017-06-10 05:05:42 EDT
Description of problem:
017-06-10 08:48:25Z [overcloud-AllNodesDeploySteps-axkfftqzrjwt.ControllerUpgrade_Step0.0]: CREATE_FAILED  Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
2017-06-10 08:48:25Z [overcloud-AllNodesDeploySteps-axkfftqzrjwt.ControllerUpgrade_Step0]: CREATE_FAILED  Resource CREATE failed: Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
2017-06-10 08:48:26Z [overcloud-AllNodesDeploySteps-axkfftqzrjwt.ControllerUpgrade_Step0]: CREATE_FAILED  Error: resources.ControllerUpgrade_Step0.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 4
2017-06-10 08:48:26Z [overcloud-AllNodesDeploySteps-axkfftqzrjwt]: UPDATE_FAILED  Error: resources.ControllerUpgrade_Step0.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 4
2017-06-10 08:48:27Z [AllNodesDeploySteps]: UPDATE_FAILED  resources.AllNodesDeploySteps: Error: resources.ControllerUpgrade_Step0.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 4
2017-06-10 08:48:27Z [overcloud]: UPDATE_FAILED  resources.AllNodesDeploySteps: Error: resources.ControllerUpgrade_Step0.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 4

 Stack overcloud UPDATE_FAILED 

overcloud.AllNodesDeploySteps.ControllerUpgrade_Step0.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: 2a6b37b9-b991-422f-8bce-6c3bc640e8b1
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
  deploy_stdout: |

  deploy_stderr: |
    ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
Heat Stack update failed.
Heat Stack update failed.

(undercloud) [stack@undercloud-0 ~]$ openstack stack failures list --long overcloud
overcloud.AllNodesDeploySteps.ControllerUpgrade_Step0.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: 2a6b37b9-b991-422f-8bce-6c3bc640e8b1
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
  deploy_stdout: |

  deploy_stderr: |
    ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.


Version-Release number of selected component (if applicable):
OSP12

How reproducible:


Steps to Reproduce:
1) Deploy OSP11 Undercloud & Overcloud
2) perform minor update of undercloud node
>sudo rhos-release 11 -r 7.4
>sudo systemctl stop 'openstack-*' 'neutron-*' httpd
>sudo yum update -y instack-undercloud openstack-puppet-modules openstack-tripleo-common python-tripleoclient
>openstack undercloud upgrade
>sudo reboot

3) upgrade osp11 undrcloud to osp12

>sudo rhos-release 12-director
>sudo systemctl stop openstack-*
>sudo systemctl stop neutron-*
>sudo systemctl stop httpd
>sudo yum -y update instack-undercloud openstack-puppet-modules openstack-tripleo-common python-tripleoclient
>openstack undercloud upgrade

#Overcloud upgrade 
source stackrc

export IMAGE_TAG=`bash /usr/bin/puddle-version http://download-node-02.eng.bos.redhat.com/rcm-guest/puddles/OpenStack/12.0-RHEL-7/latest_containers`

 sudo wget -O /home/stack/overcloud_containers.yaml http://file.rdu.redhat.com/~ohochman/containers/overcloud_containers.yaml
    sudo chown stack:stack /home/stack/overcloud_containers.yaml
    sudo sed -i 's:<IMAGE_TAG>:'$IMAGE_TAG':g' /home/stack/overcloud_containers.yaml
    sudo sed -i 's/--insecure-registry 192.168.24.3:8787/--insecure-registry docker-registry.engineering.redhat.com/' /etc/sysconfig/docker
    sudo service docker restart
    source /home/stack/stackrc && openstack overcloud container image upload --verbose --config-file /home/stack/overcloud_containers.yaml 
    
sudo wget -O /usr/share/openstack-tripleo-heat-templates/environments/docker-osp12.yaml http://file.rdu.redhat.com/~ohochman/containers/docker-osp12.yaml
    sudo chown stack:stack /usr/share/openstack-tripleo-heat-templates/environments/docker-osp12.yaml

    sudo sed -i 's:<IMAGE_TAG>:'$IMAGE_TAG':g' /usr/share/openstack-tripleo-heat-templates/environments/docker-osp12.yaml


# use master repos
cat > ~/containers-upgrade-repos.yaml <<EOEF
parameter_defaults:
  UpgradeInitCommand: |
    set -e
    curl -LO http://rhos-release.virt.bos.redhat.com/repos/rhos-release/rhos-release-latest.noarch.rpm
    rpm -ivh rhos-release-latest.noarch.rpm || true
    rhos-release 12-director
    yum clean all
EOEF

export THT=/usr/share/openstack-tripleo-heat-templates

Apply workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1456562
Apply workaround for https://bugzilla.redhat.com/show_bug.cgi?id=1458044 

# upgrade overcloud
openstack overcloud deploy --templates --libvirt-type kvm --ntp-server clock.redhat.com -e /home/stack/virt/network/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/virt/hostnames.yml -e /home/stack/virt/debug.yaml -e /home/stack/virt/nodes_data.yaml -e $THT/environments/low-memory-usage.yaml -e $THT/environments/docker.yaml -e $THT/environments/major-upgrade-composable-steps-docker.yaml -e ~/containers-upgrade-repos.yaml -e $THT/environments/docker-osp12.yaml --log-file overcloud_deployment_26.log




Actual results:
Overcloud upgrade was failed


Expected results:
Overcloud upgraded successfully
Comment 1 Red Hat Bugzilla Rules Engine 2017-06-10 05:05:47 EDT
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Comment 2 Artem Hrechanychenko 2017-06-19 09:49:38 EDT
didn't reproduced in last run
Comment 3 Omri Hochman 2017-06-19 11:55:18 EDT
Please check if reproduce on latest , if reproduces please check with dprince
Comment 4 Sofer Athlan-Guyot 2017-06-26 08:45:19 EDT
Hi,

we're moving this to triaged, as we're for reporter information.
Comment 5 Artem Hrechanychenko 2017-06-29 04:27:32 EDT
reproduced 
overcloud.AllNodesDeploySteps.ControllerUpgrade_Step0.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: 761dde55-fb3f-421e-8a42-a07c08cc3f3b
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 4
  deploy_stdout: |

  deploy_stderr: |
    ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.
Comment 6 Marius Cornea 2017-06-29 04:55:11 EDT
I suspect the issue is caused by missing installing ansible-pacemaker module during upgrade.
Comment 8 Jiri Stransky 2017-06-29 05:58:32 EDT
It should also be present in Pike images after we merge the patches, so we don't have to install this for P->Q again, and we may use it for minor updates too.
Comment 15 errata-xmlrpc 2017-12-13 16:31:55 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462

Note You need to log in before you can comment on or make changes to this bug.