Bug 1414900 - python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ironic/ironic-conductor.log on controller
Summary: python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ir...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-ironic-lib
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: Upstream M3
: 11.0 (Ocata)
Assignee: RHOS Maint
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-19 17:02 UTC by Alexander Chuzhoy
Modified: 2017-05-17 19:41 UTC (History)
10 users (show)

Fixed In Version: python-ironic-lib-2.5.2-0.20170208212103.ace87b6.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-17 19:41:32 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2017:1245 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 23:01:50 UTC
OpenStack gerrit 422738 None None None 2017-01-20 09:58:28 UTC
Launchpad 1657808 None None None 2017-01-20 09:57:46 UTC

Description Alexander Chuzhoy 2017-01-19 17:02:37 UTC
python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ironic/ironic-conductor.log on controller 

Environment:
python-ironic-lib-2.1.2-1.el7ost.noarch
openstack-ironic-api-6.2.2-3.el7ost.noarch
python-ironic-inspector-client-1.9.0-2.el7ost.noarch
openstack-ironic-common-6.2.2-3.el7ost.noarch
openstack-ironic-conductor-6.2.2-3.el7ost.noarch
python-ironicclient-1.7.1-1.el7ost.noarch
puppet-ironic-9.4.1-2.el7ost.noarch
instack-undercloud-5.2.0-1.el7ost.noarch
openstack-tripleo-heat-templates-5.2.0-3.el7ost.noarch

Steps to reproduce:
1. Deployed a setup with controllers having 8G of ram (no swap) with ironic enabled:
openstack overcloud deploy --debug --templates --libvirt-type kvm --ntp-server clock.redhat.com --neutron-network-type vxlan --neutron-tunnel-types vxlan -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml -e virt/ceph.yaml -e virt/hostnames.yml -e virt/network/network-environment.yaml -e ironic.yaml -e flat_networks.yaml

2. Configured baremetal network, registered nodes.
3. Tried to clean:
The cleaning failed.
/var/log/ironic/ironic-conductor.log:2017-01-18 20:23:58.127 161689 ERROR ironic.conductor.manager [req-dd81e663-fc93-41b7-862f-4648c6b9bf43 - - - - -] Failed to prepare node 63a5184f-5327-4f97-a0fd-97709f0d5002 for cleaning: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:23:58.127 161689 ERROR ironic.conductor.manager OSError: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:24:07.084 161689 ERROR ironic.conductor.manager [req-30479e22-d338-434a-a5d7-a539ffc07e95 - - - - -] Failed to prepare node 99d0b665-3343-4754-94a6-a63d66427134 for cleaning: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:24:07.084 161689 ERROR ironic.conductor.manager OSError: [Errno 12] Cannot allocate memory

Comment 1 Alexander Chuzhoy 2017-01-19 17:04:58 UTC
According to the guide[1] it's enough to have 8GB on controllers:
"It is recommended to have at least 12 GiB of RAM on the undercloud and at least 8 GiB of RAM on the controllers. "

[1]http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/baremetal_overcloud.html#bare-metal-instances-in-overcloud

Comment 2 Lucas Alvares Gomes 2017-01-20 10:00:08 UTC
So the issue is suppose to be fixed now. The patch upstream [0] is now merged and it has been backported to Ocata and Newton too [1]. I hope it solves the problem, if not, we will need to investigate more.

[0] https://review.openstack.org/422738
[1] https://review.openstack.org/#/q/topic:bug/1657808

Comment 3 Ramon Acedo 2017-02-24 10:43:44 UTC
Please, note that the minimum amount of RAM we support is 16 GB for the controllers [1]

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/director_installation_and_usage/#sect-Controller_Node_Requirements

Comment 6 Alexander Chuzhoy 2017-03-23 15:58:21 UTC
Verified:
Environment: 
python-ironic-lib-2.5.2-0.20170208212103.ace87b6.el7ost.noarch

The reported issue doesn't reproduce.

Comment 10 errata-xmlrpc 2017-05-17 19:41:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245


Note You need to log in before you can comment on or make changes to this bug.