Hide Forgot
rhel-osp-director: Minor update fails right away with "'unicode' object does not support item assignment" Environment: instack-undercloud-5.0.0-0.20161007201832.f044a47.el7ost.noarch openstack-puppet-modules-9.3.0-0.20161003154825.8c758d6.el7ost.noarch openstack-tripleo-heat-templates-5.0.0-0.20161003064637.d636e3a.1.2.el7ost.noarch Steps to reproduce: 1. Deploy overcloud with: openstack overcloud deploy --debug --templates --libvirt-type kvm --ntp-server clock.redhat.com --neutron-network-type vxlan --neutron-tunnel-types vxlan --control-scale 3 --control-flavor controller-d75f3dec-c770-5f88-9d4c-3fea1bf9c484 --compute-scale 1 --compute-flavor compute-b634c10a-570f-59ba-bdbf-0c313d745a10 --ceph-storage-scale 2 --ceph-storage-flavor ceph-cf1f074b-dadb-5eb8-9eb0-55828273fab7 -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e virt/ceph.yaml -e virt/hostnames.yml -e virt/network/network-environment.yaml --log-file overcloud_deployment_48.log 2. Update the undercloud. 3. Attempt to update the overcloud with: yes ""| openstack overcloud update stack -i overcloud Result: starting package update on stack overcloud 'unicode' object does not support item assignment The update doesn't even start.
I'm not seeing this upstream, but it looks like it came from this patch: https://github.com/openstack/python-tripleoclient/commit/c7a1b84e713afd59630261d6f3992a4f798504a1
(In reply to Ryan Hallisey from comment #1) > I'm not seeing this upstream, but it looks like it came from this patch: > https://github.com/openstack/python-tripleoclient/commit/ > c7a1b84e713afd59630261d6f3992a4f798504a1 why is that the case?
Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates AttributeError: 'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES' Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates self.container, constants.OVERCLOUD_J2_EXCLUDES)[1] Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates File "/usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py", line 109, in _process_custo Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates self._process_custom_roles() Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates File "/usr/lib/python2.7/site-packages/tripleo_common/actions/templates.py", line 184, in run Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates Traceback (most recent call last): Oct 18 17:47:37 undercloud-0.redhat.local mistral-server[5275]: 2016-10-18 17:47:37.358 5275 ERROR tripleo_common.actions.templates [-] Error occurred while processing custom roles. Oct 18 17:47:37 undercloud-0.redhat.local ironic-conductor[3771]: 2016-10-18 17:47:37.358 3771 DEBUG oslo_concurrency.processutils [req-29058448-134b-490b-a8aa-761cf04846f0 - - - - -]
dougal, can you have a look at this one?
Here's what I see on the console: [stack@undercloud-0 ~]$ yes ""| openstack overcloud update stack -i overcloud u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'" starting package update on stack overcloud { 'parameter_defaults': { 'DeployIdentifier': 1476844649, 'StackAction': 'UPDATE', 'UpdateIdentifier': 1476844649}, 'resource_registry': { 'resources': { '*': { '*': { 'UpdateDeployment': { 'hooks': 'pre-update'}}}}}} u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'" 'unicode' object does not support item assignment
(In reply to Alexander Chuzhoy from comment #5) I'm sorry I didn't clean up after debugging, what you see is pretty prints from the code that I forgot to remove. > Here's what I see on the console: > > [stack@undercloud-0 ~]$ yes ""| openstack overcloud update stack -i > overcloud > u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'" > starting package update on stack overcloud > { 'parameter_defaults': { 'DeployIdentifier': 1476844649, > 'StackAction': 'UPDATE', > 'UpdateIdentifier': 1476844649}, > 'resource_registry': { 'resources': { '*': { '*': { > 'UpdateDeployment': { 'hooks': 'pre-update'}}}}}} > u"'module' object has no attribute 'OVERCLOUD_J2_EXCLUDES'" > 'unicode' object does not support item assignment
Fix is in upstream we just need to get it downstream.
I tried this in a fresh downstream install and got this: [stack@instack ~]$ yes ""| openstack overcloud update stack -i overcloud starting package update on stack overcloud IN_PROGRESS WAITING on_breakpoint: [u'overcloud-controller-0', u'overcloud-novacompute-0'] Breakpoint reached, continue? Regexp or Enter=proceed (will clear aa019de3-53f9-45ab-ad9c-bde9fcd1a9ac), no=cancel update, C-c=quit interactive mode: WAITING completed: [u'overcloud-novacompute-0'] on_breakpoint: [u'overcloud-controller-0'] Breakpoint reached, continue? Regexp or Enter=proceed (will clear 4ef0ffd4-1cff-442a-abdd-d4505a408c76), no=cancel update, C-c=quit interactive mode: IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS IN_PROGRESS FAILED update finished with status FAILED Stack update failed.
It does seem like I am seeing a different problem, I think this bug describes the issue before it was fixed by thrash. I am now digging into the heat failure and attempting to understand what the failure I got means: [stack@instack ~]$ openstack stack failures list overcloud overcloud.AllNodesDeploySteps.ControllerDeployment_Step4.0: resource_type: OS::Heat::StructuredDeployment physical_resource_id: 3a83df99-1163-4e18-8f52-21048112049e status: UPDATE_FAILED status_reason: | Error: resources[0]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1 deploy_stdout: | Matching apachectl 'Server version: Apache/2.4.6 (CentOS) Server built: Jul 18 2016 15:30:14' Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked. deploy_stderr: | ... Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated Warning: Scope(Class[Ceilometer]): Both $metering_secret and $telemetry_secret defined, using $telemetry_secret Warning: You cannot collect exported resources without storeconfigs being set; the collection will be ignored on line 166 in file /etc/puppet/modules/gnocchi/manifests/api.pp Warning: Scope(Class[Gnocchi::Api]): gnocchi:api::keystone_identity_uri is deprecated, use gnocchi::keystone::authtoken::auth_url instead Warning: Scope(Class[Gnocchi::Api]): gnocchi::api::keystone_auth_uri is deprecated, use gnocchi::keystone::authtoken::auth_uri instead Warning: Not collecting exported resources without storeconfigs Error: Duplicate declaration: Package[python-memcache] is already declared; cannot redeclare at /etc/puppet/modules/oslo/manifests/cache.pp:159 on node overcloud-controller-0.localdomain Error: Duplicate declaration: Package[python-memcache] is already declared; cannot redeclare at /etc/puppet/modules/oslo/manifests/cache.pp:159 on node overcloud-controller-0.localdomain (truncated, view all with --long) overcloud.AllNodesDeploySteps.ComputeDeployment_Step4.0: resource_type: OS::Heat::StructuredDeployment physical_resource_id: 4d6999d5-525e-44f8-a36a-49d18431e5a5 status: UPDATE_FAILED status_reason: | UPDATE aborted deploy_stdout: | Matching apachectl 'Server version: Apache/2.4.6 (CentOS) Server built: Jul 18 2016 15:30:14' Notice: Scope(Class[Tripleo::Firewall::Post]): At this stage, all network traffic is blocked. Notice: Compiled catalog for overcloud-novacompute-0.localdomain in environment production in 3.70 seconds Notice: /Stage[setup]/Tripleo::Packages::Upgrades/Exec[package-upgrade]/returns: executed successfully Notice: Finished catalog run in 2.66 seconds deploy_stderr: | ... Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::cpu_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::ram_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated Warning: Scope(Class[Nova]): Could not look up qualified variable '::nova::scheduler::filter::disk_allocation_ratio'; class ::nova::scheduler::filter has not been evaluated Warning: Scope(Class[Nova::Compute]): compute_manager is marked as deprecated in Nova but still needed when Ironic is used. It will be removed once Nova removes it. Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::host'; class ::nova::vncproxy has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::vncproxy_protocol'; class ::nova::vncproxy has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::port'; class ::nova::vncproxy has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::vncproxy::vncproxy_path'; class ::nova::vncproxy has not been evaluated Warning: Scope(Class[Ceilometer]): Both $metering_secret and $telemetry_secret defined, using $telemetry_secret Warning: Scope(Class[Ceilometer::Agent::Compute]): This class is deprecated. Please use ceilometer::agent::polling with compute namespace instead. (truncated, view all with --long)
re-adding rhallise as I didn't intend to clear that flag.
Lukas covered the why in comment 3. Sorry I should've included that after my comment.
Verified: Environment: openstack-tripleo-common-5.3.0-3.el7ost.noarch The reported issue doesn't reproduce.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html