Created attachment 1265749 [details] Full error message Description of problem: The size of the new templates seems to exceed the allowed maximum. I had an OSP10 deployment on virtual machines with 3 controllers + 2 computes + 1 ceph, with customized roles for keystone and a ServiceApi role. After upgrading the undercloud successfully I continued to run the upgrade command of the overcloud: openstack overcloud deploy --templates -r roles_data.yaml --ntp-server clock.redhat.com -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps.yaml --no-cleanup -e fernet.yaml -e init-repo.yaml The upgrade failed right on the beginning with: "ERROR: Request limit exceeded: JSON body size (1119641 bytes) exceeds maximum allowed size (1048576 bytes)". See attached text file with the full error message. Version-Release number of selected component (if applicable): openstack-tripleo-0.0.8-0.2.4de13b3git.el7ost.noarch openstack-heat-engine-8.0.0-4.el7ost.noarch python-heat-agent-json-file-1.0.0-0.20170224185834.8e6dbb1.el7ost.noarch openstack-heat-common-8.0.0-4.el7ost.noarch openstack-tripleo-heat-templates-6.0.0-0.3.0rc2.el7ost.noarch Additional info: There is a bug for it upstream (from 2015!): https://bugs.launchpad.net/heat/+bug/1499379
Hi Udi, Have you been able to reproduce it or is it 100% reproducible with the deploy command above ?
I've also hit this issue on a different environment/scenario: u'message': u"Failed to run action [action_ex_id=726cebca-9980-4973-a939-0fc0f85e1a0a, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'container': u'overcloud', u'timeout': 240}']\n ERROR: Request limit exceeded: JSON body size (1050160 bytes) exceeds maximum allowed size (1048576 bytes).", To workaround it I had to adjust max_json_body_size in heat.conf (>1048576). I think we might need to adjust the default value to avoid such conditions. I hit this message when running the following deploy command: source ~/stackrc export THT=/usr/share/openstack-tripleo-heat-templates/ openstack overcloud deploy --templates $THT \ -r ~/openstack_deployment/roles/roles_data.yaml \ -e $THT/environments/network-isolation.yaml \ -e $THT/environments/network-management.yaml \ -e $THT/environments/storage-environment.yaml \ -e $THT/environments/manila-cephfsnative-config.yaml \ -e $THT/environments/tls-endpoints-public-ip.yaml \ -e ~/openstack_deployment/environments/nodes.yaml \ -e ~/openstack_deployment/environments/network-environment.yaml \ -e ~/openstack_deployment/environments/disk-layout.yaml \ -e ~/openstack_deployment/environments/public_vip.yaml \ -e ~/openstack_deployment/environments/enable-tls.yaml \ -e ~/openstack_deployment/environments/inject-trust-anchor.yaml \ -e ~/openstack_deployment/environments/neutron-settings.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-composable-steps.yaml \ -e ~/repo.yaml \ --log-file overcloud_deployment.log &> overcloud_install.log Environment files: http://paste.openstack.org/show/604155/
Ack thank you Marius. I'm trying to reproduce it as well and i'll fix it, thank you
Updating the Heat configuration is good workaround for now. But, if we need to send more than 1M of template data to Heat, there is a design issue: it's not meant to handle that much, and it's going to be hard to manage memory properly. We need to identify what's in that payload.
The external patch update resolve this issue for me. Keith
Hi, I've cherry picked upstream to stable/ocata and updated the gerrit reference. Thanks Keith for the pointer.
Verified when upgrading from 10.0.z (2017-04-10) to OSP11 (2017-04-20).
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1245