Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1414900 - python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ironic/ironic-conductor.log on controller
python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ir...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-ironic-lib (Show other bugs)
11.0 (Ocata)
Unspecified Unspecified
unspecified Severity unspecified
: Upstream M3
: 11.0 (Ocata)
Assigned To: RHOS Maint
Alexander Chuzhoy
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-01-19 12:02 EST by Alexander Chuzhoy
Modified: 2017-05-17 15:41 EDT (History)
10 users (show)

See Also:
Fixed In Version: python-ironic-lib-2.5.2-0.20170208212103.ace87b6.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-05-17 15:41:32 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1657808 None None None 2017-01-20 04:57 EST
OpenStack gerrit 422738 None None None 2017-01-20 04:58 EST
Red Hat Product Errata RHEA-2017:1245 normal SHIPPED_LIVE Red Hat OpenStack Platform 11.0 Bug Fix and Enhancement Advisory 2017-05-17 19:01:50 EDT

  None (edit)
Description Alexander Chuzhoy 2017-01-19 12:02:37 EST
python-ironic-lib: ironic in overcloud: Cannot allocate memory in /var/log/ironic/ironic-conductor.log on controller 

Environment:
python-ironic-lib-2.1.2-1.el7ost.noarch
openstack-ironic-api-6.2.2-3.el7ost.noarch
python-ironic-inspector-client-1.9.0-2.el7ost.noarch
openstack-ironic-common-6.2.2-3.el7ost.noarch
openstack-ironic-conductor-6.2.2-3.el7ost.noarch
python-ironicclient-1.7.1-1.el7ost.noarch
puppet-ironic-9.4.1-2.el7ost.noarch
instack-undercloud-5.2.0-1.el7ost.noarch
openstack-tripleo-heat-templates-5.2.0-3.el7ost.noarch

Steps to reproduce:
1. Deployed a setup with controllers having 8G of ram (no swap) with ironic enabled:
openstack overcloud deploy --debug --templates --libvirt-type kvm --ntp-server clock.redhat.com --neutron-network-type vxlan --neutron-tunnel-types vxlan -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml -e virt/ceph.yaml -e virt/hostnames.yml -e virt/network/network-environment.yaml -e ironic.yaml -e flat_networks.yaml

2. Configured baremetal network, registered nodes.
3. Tried to clean:
The cleaning failed.
/var/log/ironic/ironic-conductor.log:2017-01-18 20:23:58.127 161689 ERROR ironic.conductor.manager [req-dd81e663-fc93-41b7-862f-4648c6b9bf43 - - - - -] Failed to prepare node 63a5184f-5327-4f97-a0fd-97709f0d5002 for cleaning: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:23:58.127 161689 ERROR ironic.conductor.manager OSError: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:24:07.084 161689 ERROR ironic.conductor.manager [req-30479e22-d338-434a-a5d7-a539ffc07e95 - - - - -] Failed to prepare node 99d0b665-3343-4754-94a6-a63d66427134 for cleaning: [Errno 12] Cannot allocate memory
/var/log/ironic/ironic-conductor.log:2017-01-18 20:24:07.084 161689 ERROR ironic.conductor.manager OSError: [Errno 12] Cannot allocate memory
Comment 1 Alexander Chuzhoy 2017-01-19 12:04:58 EST
According to the guide[1] it's enough to have 8GB on controllers:
"It is recommended to have at least 12 GiB of RAM on the undercloud and at least 8 GiB of RAM on the controllers. "

[1]http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/baremetal_overcloud.html#bare-metal-instances-in-overcloud
Comment 2 Lucas Alvares Gomes 2017-01-20 05:00:08 EST
So the issue is suppose to be fixed now. The patch upstream [0] is now merged and it has been backported to Ocata and Newton too [1]. I hope it solves the problem, if not, we will need to investigate more.

[0] https://review.openstack.org/422738
[1] https://review.openstack.org/#/q/topic:bug/1657808
Comment 3 Ramon Acedo 2017-02-24 05:43:44 EST
Please, note that the minimum amount of RAM we support is 16 GB for the controllers [1]

[1] https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/10/html-single/director_installation_and_usage/#sect-Controller_Node_Requirements
Comment 6 Alexander Chuzhoy 2017-03-23 11:58:21 EDT
Verified:
Environment: 
python-ironic-lib-2.5.2-0.20170208212103.ace87b6.el7ost.noarch

The reported issue doesn't reproduce.
Comment 10 errata-xmlrpc 2017-05-17 15:41:32 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245

Note You need to log in before you can comment on or make changes to this bug.