Bug 1389131

Summary: rhel-osp-director: OSP10 Minor update fails: Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_cache]: Failed to call refresh: Could not find command '/usr/share/openstack-dashboard/manage.py'
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: rhosp-directorAssignee: Lukas Bezdicka <lbezdick>
Status: CLOSED CURRENTRELEASE QA Contact: Alexander Chuzhoy <sasha>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 10.0 (Newton)CC: dbecker, jcoufal, jslagle, lbezdick, mburns, morazi, rhel-osp-director-maint
Target Milestone: rcKeywords: Triaged
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-16 16:50:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Alexander Chuzhoy 2016-10-27 00:24:56 UTC
rhel-osp-director:  OSP10 Minor update fails: Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_cache]: Failed to call refresh: Could not find command '/usr/share/openstack-dashboard/manage.py'

Environment:
openstack-puppet-modules-9.3.0-0.20161003154825.8c758d6.el7ost.noarch
instack-undercloud-5.0.0-2.el7ost.noarch
openstack-tripleo-heat-templates-5.0.0-0.8.0rc3.el7ost.noarch


Steps to reproduce:
1. Deploy overcloud.
2. Attempt to minor update

Result:
Update fails:

Debugging with heat:
[stack@undercloud-0 ~]$ heat stack-list                                                                                                   
WARNING (shell) "heat stack-list" is deprecated, please use "openstack stack list" instead                                                
+--------------------------------------+------------+---------------+----------------------+----------------------+                       
| id                                   | stack_name | stack_status  | creation_time        | updated_time         |                       
+--------------------------------------+------------+---------------+----------------------+----------------------+                       
| 168f517a-0e5f-4ef9-a62c-7ae44a304a60 | overcloud  | UPDATE_FAILED | 2016-10-26T21:07:24Z | 2016-10-26T22:26:51Z |                       
+--------------------------------------+------------+---------------+----------------------+----------------------+                       
[stack@undercloud-0 ~]$ heat resource-list -n5 overcloud|grep -v COMPLE
WARNING (shell) "heat resource-list" is deprecated, please use "openstack stack resource list" instead
heat deployment-show +-------------------------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------+                                                                                                                                                                                                                   
| resource_name                             | physical_resource_id                                                            | resource_type                                                                                                         | resource_status | updated_time         | stack_name                                                                                                           |                   
+-------------------------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------+                   
| AllNodesDeploySteps                       | 2236e9a9-d856-4e16-8ae4-3d534e15e9a6                                            | OS::TripleO::PostDeploySteps                                                                                          | UPDATE_FAILED   | 2016-10-26T23:00:43Z | overcloud                                                                                                            |                   
| ControllerDeployment_Step4                | 204bde9d-7161-4f70-b9c7-fb5caeec1fee                                            | OS::Heat::StructuredDeploymentGroup                                                                                   | UPDATE_FAILED   | 2016-10-26T23:08:49Z | overcloud-AllNodesDeploySteps-kblqhdbit6bb                                                                           |                   
| 0                                         | 145253cf-18fd-4d5e-9fe2-2647fa7d7730                                            | OS::Heat::StructuredDeployment                                                                                        | UPDATE_FAILED   | 2016-10-26T23:08:50Z | overcloud-AllNodesDeploySteps-kblqhdbit6bb-ControllerDeployment_Step4-tsl6lr63zefi                                   |                   
| 2                                         | d3294ee7-089b-4d0b-8070-06480015694a                                            | OS::Heat::StructuredDeployment                                                                                        | UPDATE_FAILED   | 2016-10-26T23:08:51Z | overcloud-AllNodesDeploySteps-kblqhdbit6bb-ControllerDeployment_Step4-tsl6lr63zefi                                   |                   
| 1                                         | db10e26b-be6e-4094-9580-921bf382f64b                                            | OS::Heat::StructuredDeployment                                                                                        | UPDATE_FAILED   | 2016-10-26T23:08:52Z | overcloud-AllNodesDeploySteps-kblqhdbit6bb-ControllerDeployment_Step4-tsl6lr63zefi                                   |                   
+-------------------------------------------+---------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------+----------------------+----------------------------------------------------------------------------------------------------------------------+                   




Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated
Warning: Scope(Class[Ceilometer]): Both $metering_secret and $telemetry_secret defined, using $telemetry_secret
Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 166 in file /etc/puppet/modules/gnocchi/manifests/api.pp
Warning: Scope(Class[Gnocchi::Api]): gnocchi:api::keystone_identity_uri is deprecated, use gnocchi::keystone::authtoken::auth_url instead
Warning: Scope(Class[Gnocchi::Api]): gnocchi::api::keystone_auth_uri is deprecated, use gnocchi::keystone::authtoken::auth_uri instead
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Warning: Scope(Haproxy::Config[haproxy]): haproxy: The $merge_options parameter will default to true in the next major release. Please review the documentation regarding the implications.
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Warning: Not collecting exported resources without storeconfigs
Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_cache]: Failed to call refresh: Could not find command '/usr/share/openstack-dashboard/manage.py'
Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_cache]: Could not find command '/usr/share/openstack-dashboard/manage.py'
Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_compress]: Failed to call refresh: Could not find command '/usr/share/openstack-dashboard/manage.py'
Error: /Stage[main]/Horizon/Exec[refresh_horizon_django_compress]: Could not find command '/usr/share/openstack-dashboard/manage.py'
", "deploy_status_code": 6 }, "creation_time": "2016-10-26T21:39:33Z", "updated_time": "2016-10-26T23:14:41Z", "input_values": { "step": 4, "update_identifier": "1477520805" }, "action": "UPDATE", "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 6", "id": "145253cf-18fd-4d5e-9fe2-2647fa7d7730" }


Expected result:
Successful update.

Comment 2 Alexander Chuzhoy 2016-10-28 16:04:18 UTC
Reproduced.

Comment 3 Lukas Bezdicka 2016-10-31 12:07:07 UTC
I think this is because of the workaround we made, checking.

Comment 4 Alexander Chuzhoy 2016-11-08 23:17:07 UTC
Verified:
Environment:
openstack-tripleo-common-5.3.0-3.el7ost.noarch
puppet-tripleo-5.3.0-6.el7ost.noarch
python-tripleoclient-5.3.0-5.el7ost.noarch


The update completes successfully.