Bug 1385852

I'm not seeing this upstream, but it looks like it came from this patch: https://github.com/openstack/python-tripleoclient/commit/c7a1b84e713afd59630261d6f3992a4f798504a1

(In reply to Ryan Hallisey from comment #1)
> I'm not seeing this upstream, but it looks like it came from this patch:
> https://github.com/openstack/python-tripleoclient/commit/
> c7a1b84e713afd59630261d6f3992a4f798504a1

why is that the case?

Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates AttributeError: 'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates     self.container, constants.OVERCLOUD_J2_EXCLUDES)[1]
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates   File "/usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py", line 109, in _process_custo
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates     self._process_custom_roles()
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates   File "/usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py", line 184, in run
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates Traceback (most recent call last):
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates [-] Error occurred while processing custom roles.
Oct 18 17:47:37 undercloud-0.redhat.local ironic-conductor[3771]: 2016-10-18 17:47:37.358 3771 DEBUG oslo_concurrency.processutils [req-29058448-134b-490b-a8aa-761cf04846f0 - - - - -]

dougal, can you have a look at this one?

Here's what I see on the console:

[stack@undercloud-0 ~]$ yes ""| openstack overcloud update stack  -i overcloud
u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'"
starting package update on stack overcloud
{   'parameter_defaults': {   'DeployIdentifier': 1476844649,
                              'StackAction': 'UPDATE',
                              'UpdateIdentifier': 1476844649},
    'resource_registry': {   'resources': {   '*': {   '*': {   'UpdateDeployment': {   'hooks': 'pre-update'}}}}}}
u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'"
'unicode' object does not support item assignment

(In reply to Alexander Chuzhoy from comment #5)
I'm sorry I didn't clean up after debugging, what you see is pretty prints from the code that I forgot to remove.
> Here's what I see on the console:
> 
> [stack@undercloud-0 ~]$ yes ""| openstack overcloud update stack  -i
> overcloud
> u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'"
> starting package update on stack overcloud
> {   'parameter_defaults': {   'DeployIdentifier': 1476844649,
>                               'StackAction': 'UPDATE',
>                               'UpdateIdentifier': 1476844649},
>     'resource_registry': {   'resources': {   '*': {   '*': {  
> 'UpdateDeployment': {   'hooks': 'pre-update'}}}}}}
> u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'"
> 'unicode' object does not support item assignment

Fix is in upstream we just need to get it downstream.

I tried this in a fresh downstream install and got this:

[stack@instack ~]$ yes ""| openstack overcloud update stack  -i overcloud
starting package update on stack overcloud
IN_PROGRESS
WAITING
on_breakpoint: [u'overcloud-controller-0', u'overcloud-novacompute-0']
Breakpoint reached, continue? Regexp or Enter=proceed (will clear aa019de3-53f9-45ab-ad9c-bde9fcd1a9ac), no=cancel update, C-c=quit interactive mode: WAITING
completed: [u'overcloud-novacompute-0']
on_breakpoint: [u'overcloud-controller-0']
Breakpoint reached, continue? Regexp or Enter=proceed (will clear 4ef0ffd4-1cff-442a-abdd-d4505a408c76), no=cancel update, C-c=quit interactive mode: IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
IN_PROGRESS
FAILED
update finished with status FAILED
Stack update failed.

It does seem like I am seeing a different problem, I think this bug describes the issue before it was fixed by thrash. 

I am now digging into the heat failure and attempting to understand what the failure I got means:

[stack@instack ~]$ openstack stack failures list overcloud
overcloud.AllNodesDeploySteps.ControllerDeployment_Step4.0:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: 3a83df99-1163-4e18-8f52-21048112049e
  status: UPDATE_FAILED
  status_reason: |
    Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1
  deploy_stdout: |
    Matching apachectl 'Server version: Apache/2.4.6 (CentOS)
    Server built:   Jul 18 2016 15:30:14'
    Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.
  deploy_stderr: |
    ...
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated
    Warning: Scope(Class[Ceilometer]): Both $metering_secret and $telemetry_secret defined, using $telemetry_secret
    Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 166 in file /etc/puppet/modules/gnocchi/manifests/api.pp
    Warning: Scope(Class[Gnocchi::Api]): gnocchi:api::keystone_identity_uri is deprecated, use gnocchi::keystone::authtoken::auth_url instead
    Warning: Scope(Class[Gnocchi::Api]): gnocchi::api::keystone_auth_uri is deprecated, use gnocchi::keystone::authtoken::auth_uri instead
    Warning: Not collecting exported resources without storeconfigs
    Error: Duplicate declaration: Package[python-memcache] is already declared; cannot redeclare at /etc/puppet/modules/oslo/manifests/cache.pp:159 on node overcloud-controller-0.localdomain
    Error: Duplicate declaration: Package[python-memcache] is already declared; cannot redeclare at /etc/puppet/modules/oslo/manifests/cache.pp:159 on node overcloud-controller-0.localdomain
    (truncated, view all with --long)
overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.0:
  resource_type: OS::Heat::StructuredDeployment
  physical_resource_id: 4d6999d5-525e-44f8-a36a-49d18431e5a5
  status: UPDATE_FAILED
  status_reason: |
    UPDATE aborted
  deploy_stdout: |
    Matching apachectl 'Server version: Apache/2.4.6 (CentOS)
    Server built:   Jul 18 2016 15:30:14'
    Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked.
    Notice: Compiled catalog for overcloud-novacompute-0.localdomain in environment production in 3.70 seconds
    Notice: /Stage[setup]/Tripleo::Packages::Upgrades/Exec[package-upgrade]/returns: executed successfully
    Notice: Finished catalog run in 2.66 seconds
  deploy_stderr: |
    ...
    Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::cpu_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated
    Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::ram_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated
    Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::disk_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated
    Warning: Scope(Class[Nova::Compute]): compute_manager is marked as deprecated in Nova but still needed when Ironic is used. It will be removed once Nova removes it.
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::host'; class ::nova::vncproxy has not been evaluated
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::vncproxy_protocol'; class ::nova::vncproxy has not been evaluated
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::port'; class ::nova::vncproxy has not been evaluated
    Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::vncproxy_path'; class ::nova::vncproxy has not been evaluated
    Warning: Scope(Class[Ceilometer]): Both $metering_secret and $telemetry_secret defined, using $telemetry_secret
    Warning: Scope(Class[Ceilometer::Agent::Compute]): This class is deprecated. Please use ceilometer::agent::polling with compute namespace instead.
    (truncated, view all with --long)

re-adding rhallise as I didn't intend to clear that flag.

Lukas covered the why in comment 3.  Sorry I should've included that after my comment.

Verified:
Environment:
openstack-tripleo-common-5.3.0-3.el7ost.noarch

The reported issue doesn't reproduce.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html