Bug 1466308 - Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed: "no crontab for ceilometer"
Summary: Cannot upgrade OSP11 to OSP12: Overcloud upgrade failed: "no crontab for ceil...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 12.0 (Pike)
Hardware: x86_64
OS: Linux
high
high
Target Milestone: beta
: 12.0 (Pike)
Assignee: Emilien Macchi
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks: 1399762
TreeView+ depends on / blocked
 
Reported: 2017-06-29 12:13 UTC by Artem Hrechanychenko
Modified: 2018-02-05 19:10 UTC (History)
8 users (show)

Fixed In Version: openstack-tripleo-heat-templates-7.0.0-0.20170706121722.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 21:35:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1701250 0 None None None 2017-06-29 12:31:25 UTC
OpenStack gerrit 478909 0 None None None 2017-06-29 12:32:00 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Artem Hrechanychenko 2017-06-29 12:13:39 UTC
Description of problem:

overcloud.AllNodesDeploySteps.ControllerUpgrade_Step2.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: a93e5e5c-f1c3-4caa-90a6-21913ebde429
  status: CREATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |
    
    PLAY [localhost] ***************************************************************
    
    TASK [Gathering Facts] *********************************************************
    ok: [localhost]
    
    TASK [Stop and disable aodh service (running under httpd)] *********************
    changed: [localhost]
    
    TASK [Stop and disable openstack-aodh-evaluator service] ***********************
    changed: [localhost]
    
    TASK [Stop and disable openstack-aodh-listener service] ************************
    changed: [localhost]
    
    TASK [Stop and disable openstack-aodh-notifier service] ************************
    changed: [localhost]
    
    TASK [Stop and disable ceilometer agent central service] ***********************
    changed: [localhost]
    
    TASK [Stop and disable ceilometer agent notification service] ******************
    changed: [localhost]
    
    TASK [Stop and disable cinder_api service] *************************************
    ok: [localhost]
    
    TASK [Stop and disable cinder_scheduler service] *******************************
    changed: [localhost]
    
    TASK [Stop and disable glance_api service] *************************************
    changed: [localhost]
    
    TASK [Stop and disable httpd service] ******************************************
    ok: [localhost]
    
    TASK [Stop and disable openstack-gnocchi-metricd service] **********************
    changed: [localhost]
    
    TASK [Stop and disable openstack-gnocchi-statsd service] ***********************
    changed: [localhost]
    
    TASK [Stop and disable heat_api service] ***************************************
    ok: [localhost]
    
    TASK [Stop and disable heat_api_cfn service] ***********************************
    ok: [localhost]
    
    TASK [Check if heat_api_cloudwatch is deployed] ********************************
    fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["systemctl", "is-enabled", "openstack-heat-api-cloudwatch"], "delta": "0:00:00.004891", "end": "2017-06-29 09:10:45.342845", "failed": true, "rc": 1, "start": "2017-06-29 09:10:45.337954", "stderr": "", "stderr_lines": [], "stdout": "disabled", "stdout_lines": ["disabled"]}
    ...ignoring
    
    TASK [Stop and disable heat_engine service] ************************************
    changed: [localhost]
    
    TASK [Stop and disable horizon service (running under httpd)] ******************
    ok: [localhost]
    
    TASK [Stop and disable iscsid service] *****************************************
    fatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check 'stat_iscsid_service.stat.exists' failed. The error was: error while evaluating conditional (stat_iscsid_service.stat.exists): 'stat_iscsid_service' is undefined"}
    	to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/6d340431-7033-4c95-9318-716da99aae0a_playbook.retry
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=18   changed=12   unreachable=0    failed=1   
    
  deploy_stderr: |

overcloud.AllNodesDeploySteps.ControllerUpgrade_Step1.0:
  resource_type: OS::Heat::SoftwareDeployment
  physical_resource_id: eef76217-214a-466d-ad86-6e979a34de14
  status: UPDATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 2
  deploy_stdout: |
    
    PLAY [localhost] ***************************************************************
    
    TASK [Gathering Facts] *********************************************************
    ok: [localhost]
    
    TASK [Remove ceilometer expirer cron tab on upgrade] ***************************
    fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/usr/bin/crontab -u ceilometer -r", "delta": "0:00:00.006122", "end": "2017-06-29 10:42:28.228248", "failed": true, "rc": 1, "start": "2017-06-29 10:42:28.222126", "stderr": "no crontab for ceilometer", "stderr_lines": ["no crontab for ceilometer"], "stdout": "", "stdout_lines": []}
    	to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/db693d1b-cc4f-4b63-b06e-4d433d97462d_playbook.retry
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=1    changed=0    unreachable=0    failed=1   
    
  deploy_stderr: |

Version-Release number of selected component (if applicable):
OSP12

openstack-nova-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-keystone-12.0.0-0.20170623233743.9070172.el7ost.noarch
openstack-ironic-inspector-5.1.1-0.20170622234613.05896b5.el7ost.noarch
puppet-openstack_extras-11.2.0-0.20170613085321.0b0ea62.el7ost.noarch
openstack-tripleo-common-containers-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
openstack-neutron-common-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-neutron-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-compute-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-mistral-executor-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-tempest-16.0.1-0.20170623203531.4386df8.el7ost.noarch
openstack-neutron-openvswitch-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-nova-placement-api-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-api-cfn-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-api-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-api-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
python-openstack-mistral-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-heat-common-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-tripleo-image-elements-7.0.0-0.20170607161959.401d861.el7ost.noarch
openstack-tripleo-puppet-elements-7.0.0-0.20170614005502.9285877.el7ost.noarch
openstack-swift-object-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-neutron-ml2-11.0.0-0.20170624003801.11acb1d.el7ost.noarch
openstack-glance-15.0.0-0.20170623215940.8188eca.el7ost.noarch
openstack-nova-scheduler-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-heat-engine-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-mistral-engine-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-ironic-conductor-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-tripleo-common-7.1.1-0.20170623115707.4ba7d56.el7ost.noarch
python-openstackclient-3.11.0-0.20170613232431.c69304e.el7ost.noarch
openstack-ironic-common-8.0.1-0.20170621044433.f0e6a07.el7ost.noarch
openstack-swift-account-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-selinux-0.8.8-0.20170622195307.74ddc0e.el7ost.noarch
openstack-swift-container-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-nova-conductor-16.0.0-0.20170624031428.3863eca.el7ost.noarch
openstack-tripleo-validations-7.1.1-0.20170621111847.fb7346f.el7ost.noarch
openstack-tripleo-heat-templates-7.0.0-0.20170624014919.el7ost.noarch
puppet-openstacklib-11.2.0-0.20170613150439.93b8e7d.el7ost.noarch
python-openstacksdk-0.9.17-0.20170621195806.7946243.el7ost.noarch
openstack-swift-proxy-2.14.1-0.20170622024006.2d18ecd.el7ost.noarch
openstack-heat-api-9.0.0-0.20170623110018.173f03a.el7ost.noarch
openstack-zaqar-5.0.0-0.20170623003642.68eac5a.el7ost.noarch
openstack-tripleo-ui-7.1.1-0.20170624213423.cb896fd.el7ost.noarch
openstack-puppet-modules-10.0.0-1.el7ost.noarch
openstack-mistral-common-5.0.0-0.20170623025911.6a1ca0f.el7ost.noarch
openstack-nova-common-16.0.0-0.20170624031428.3863eca.el7ost.noarch

Steps to Reproduce:
Steps to Reproduce:
1.Install OSP11
infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa --cleanup yes && infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa  --topology-nodes undercloud:1,controller:1,compute:1  -e  override.controller.cpu=8 -e override.controller.memory=16384 -e  override.undercloud.disks.disk1.size=100G && infrared tripleo-undercloud --version 11 --images-task=rpm && infrared tripleo-overcloud -v --introspect yes --tagging yes --post no --deployment-files virt --version 11 --deploy yes 

2.Upgrade undercloud and overcloud to osp11 latest+ rhel7.4
ir tripleo-undercloud -v --update-undercloud yes --mirror qeos --build 7.4-testing --osrelease 7.4 

ir tripleo-overcloud -v --updateto 7.4-testing --deployment-files virt --mirror qeos --osrelease 7.4
#Please note, during yum update stage on overcloud controller run "sudo pcs cluster start and sudo pcs cleanup resources" - w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1464588

3. Upgrade undercloud and overcloud to osp12
http://etherpad.corp.redhat.com/osp12-upgrade
#please note, before updating overcloud apply w/a for https://bugzilla.redhat.com/show_bug.cgi?id=1460421
https://bugzilla.redhat.com/show_bug.cgi?id=1463227
https://bugzilla.redhat.com/show_bug.cgi?id=1466219

Actual results:
overcloud upgrade failed

Expected results:
overcloud update was succeed

Additional info:

Comment 1 Red Hat Bugzilla Rules Engine 2017-06-29 12:13:44 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 2 Red Hat Bugzilla Rules Engine 2017-06-29 12:32:06 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 3 Red Hat Bugzilla Rules Engine 2017-06-29 13:33:20 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 4 Sofer Athlan-Guyot 2017-07-12 16:40:34 UTC
Merged upstream, moving to post.

Comment 9 errata-xmlrpc 2017-12-13 21:35:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.