Bug 1381561
Summary: | 'Expression consumed too much memory' at overcloud deploy | ||
---|---|---|---|
Product: | Red Hat OpenStack | Reporter: | Attila Fazekas <afazekas> |
Component: | puppet-heat | Assignee: | Jon Schlueter <jschluet> |
Status: | CLOSED ERRATA | QA Contact: | Amit Ugol <augol> |
Severity: | unspecified | Docs Contact: | |
Priority: | medium | ||
Version: | 10.0 (Newton) | CC: | amaumene, dmacpher, jcoufal, jjoyce, jschluet, mburns, mcornea, rhel-osp-director-maint, sbaker, shardy, slinaber, srevivo, tvignaud, zbitter |
Target Milestone: | rc | Keywords: | AutomationBlocker, Triaged |
Target Release: | 10.0 (Newton) | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | puppet-heat-9.4.0-1.1.el7ost | Doc Type: | Bug Fix |
Doc Text: |
The OpenStack Platform director exceeded the default memory limits for using OpenStack Orchestration (heat) YAQL expressions. This caused an "Expression consumed too much memory" error during an overcloud deployment and subsequent deployment failure. This fix increases the default memory limits for the director, which results in a error-free overcloud deployment.
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2016-12-14 16:07:29 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Attila Fazekas
2016-10-04 12:46:09 UTC
Our version expected to be new enough to have this change: https://github.com/openstack/instack-undercloud/commit/55ccd0e1e8c66c9f474112358063bc263720d84f (not verifed) (In reply to Attila Fazekas from comment #2) > Our version expected to be new enough to have this change: > https://github.com/openstack/instack-undercloud/commit/ > 55ccd0e1e8c66c9f474112358063bc263720d84f (not verifed) The fix appears to be there but the error still shows up. I hit it with 1 controller + 1 compute: grep heat::yaql_memory_quota /usr/share/instack-undercloud/puppet-stack-config/puppet-stack-config.yaml.template heat::yaql_memory_quota: 100000 instack-undercloud.noarch 5.0.0-0.20160929132429.56e65af.el7ost I gather that y'all have this instack-undercloud change: https://review.openstack.org/377782 but are probably missing the corresponding puppet-heat change: https://review.openstack.org/378639 (In reply to Zane Bitter from comment #4) > I gather that y'all have this instack-undercloud change: > > https://review.openstack.org/377782 > > but are probably missing the corresponding puppet-heat change: > > https://review.openstack.org/378639 yep that's it, working on build for it. Hi, With: puppet-heat-9.4.0-1.1.el7ost.noarch instack-undercloud-5.0.0-0.20160929132429.56e65af.el7ost.noarch Result is still: 2016-10-05 07:36:38Z [overcloud-ControllerServiceChain-36ecvlqvrx3t-ServiceChain-jbopffivvg74]: CREATE_COMPLETE Stack CREATE completed successfully 2016-10-05 07:36:39Z [ServiceChain]: CREATE_COMPLETE state changed 2016-10-05 07:36:39Z [overcloud-ControllerServiceChain-36ecvlqvrx3t]: CREATE_COMPLETE Stack CREATE completed successfully 2016-10-05 07:36:39Z [ControllerServiceChain]: CREATE_COMPLETE state changed 2016-10-05 07:36:49Z [CephStorage]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:49Z [Compute]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:51Z [overcloud-Compute-agk5lybjxlmz]: CREATE_IN_PROGRESS Stack CREATE started 2016-10-05 07:36:51Z [ObjectStorage]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:51Z [1]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:52Z [CephStorage]: CREATE_FAILED CREATE aborted 2016-10-05 07:36:52Z [Compute]: CREATE_FAILED CREATE aborted 2016-10-05 07:36:52Z [ObjectStorage]: CREATE_FAILED CREATE aborted 2016-10-05 07:36:52Z [overcloud]: CREATE_FAILED Resource CREATE failed: Expression consumed too much memory 2016-10-05 07:36:54Z [0]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:54Z [overcloud-Compute-agk5lybjxlmz-1-e6nmqr4svmv7]: CREATE_IN_PROGRESS Stack CREATE started 2016-10-05 07:36:54Z [NodeAdminUserData]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:54Z [UpdateConfig]: CREATE_IN_PROGRESS state changed 2016-10-05 07:36:55Z [NodeUserData]: CREATE_IN_PROGRESS state changed Stack overcloud CREATE_FAILED Heat Stack create failed. Moving back to dev stage then (In reply to Alexandre Maumené from comment #6) > Hi, > > With: > puppet-heat-9.4.0-1.1.el7ost.noarch > instack-undercloud-5.0.0-0.20160929132429.56e65af.el7ost.noarch > > Result is still: > 2016-10-05 07:36:38Z > [overcloud-ControllerServiceChain-36ecvlqvrx3t-ServiceChain-jbopffivvg74]: > CREATE_COMPLETE Stack CREATE completed successfully > 2016-10-05 07:36:39Z [ServiceChain]: CREATE_COMPLETE state changed > 2016-10-05 07:36:39Z [overcloud-ControllerServiceChain-36ecvlqvrx3t]: > CREATE_COMPLETE Stack CREATE completed successfully > 2016-10-05 07:36:39Z [ControllerServiceChain]: CREATE_COMPLETE state changed > 2016-10-05 07:36:49Z [CephStorage]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:49Z [Compute]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:51Z [overcloud-Compute-agk5lybjxlmz]: CREATE_IN_PROGRESS > Stack CREATE started > 2016-10-05 07:36:51Z [ObjectStorage]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:51Z [1]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:52Z [CephStorage]: CREATE_FAILED CREATE aborted > 2016-10-05 07:36:52Z [Compute]: CREATE_FAILED CREATE aborted > 2016-10-05 07:36:52Z [ObjectStorage]: CREATE_FAILED CREATE aborted > 2016-10-05 07:36:52Z [overcloud]: CREATE_FAILED Resource CREATE failed: > Expression consumed too much memory > 2016-10-05 07:36:54Z [0]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:54Z [overcloud-Compute-agk5lybjxlmz-1-e6nmqr4svmv7]: > CREATE_IN_PROGRESS Stack CREATE started > 2016-10-05 07:36:54Z [NodeAdminUserData]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:54Z [UpdateConfig]: CREATE_IN_PROGRESS state changed > 2016-10-05 07:36:55Z [NodeUserData]: CREATE_IN_PROGRESS state changed > > Stack overcloud CREATE_FAILED > > Heat Stack create failed. Do you get the 'Expression consumed too much memory' message in the FAIL step? This BZ was targeting this particular error. If there's another issue we should open a different BZ for it. Ok I think I posted to quickly... I didn't run "openstack undercloud install" to update the parameters in /etc/heat/heat.conf after updating the packages. It should be fixed and I'm testing it atm. I'll update this BZ to confirm. Moving back to MODIFIED then I confirm that I'm not hitting this issue any more after updating the packages. Thanks. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html |