Bug 1519054 - Bug in log output in hardware.py "Not enough available memory to schedule instance" prints full memory instead of available memory
Summary: Bug in log output in hardware.py "Not enough available memory to schedule ins...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: z7
: 10.0 (Newton)
Assignee: Sahid Ferdjaoui
QA Contact: Joe H. Rahme
URL:
Whiteboard:
Depends On:
Blocks: 1537045 1537047
TreeView+ depends on / blocked
 
Reported: 2017-11-30 01:41 UTC by Andreas Karis
Modified: 2022-07-09 09:58 UTC (History)
13 users (show)

Fixed In Version: openstack-nova-14.1.0-3.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1537045 (view as bug list)
Environment:
Last Closed: 2018-02-27 16:24:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1735321 0 None None None 2017-11-30 01:46:39 UTC
OpenStack gerrit 524038 0 None MERGED Fixes 'Not enough available memory' log message 2020-06-02 15:19:41 UTC
Red Hat Issue Tracker OSP-16972 0 None None None 2022-07-09 09:58:51 UTC
Red Hat Product Errata RHSA-2018:0369 0 normal SHIPPED_LIVE Moderate: openstack-nova and python-novaclient security, bug fix, and enhancement update 2018-02-27 21:24:56 UTC

Description Andreas Karis 2017-11-30 01:41:21 UTC
Description of problem:
Bug in log output in hardware.py "Not enough available memory to schedule instance" prints full memory instead of available memory

Version-Release number of selected component (if applicable):


Additional info:

When nova fails scheduling, it will print:
~~~
2017-11-29 10:50:16.904 325123 DEBUG nova.virt.hardware [req-b62c53d2-13db-4fac-a125-409b4f046418 8f883df20fce46dbef3ce634610c51be53b87e658359f05b7eba1062ce7e5d8b 5b54e36678a542d899f1ff62268fc25a - - -] Not enough available memory to schedule instance. Oversubscription is not possible with pinned instances. Required: 32768, actual: 65406 _numa_fit_instance_cell_with_pinning /usr/lib/python2.7/site-packages/nova/virt/hardware.py:845
2017-11-29 10:50:16.904 325123 DEBUG oslo_concurrency.lockutils [req-b62c53d2-13db-4fac-a125-409b4f046418 8f883df20fce46dbef3ce634610c51be53b87e658359f05b7eba1062ce7e5d8b 5b54e36678a542d899f1ff62268fc25a - - -] Lock "compute_resources" released by "nova.compute.resource_tracker.instance_claim" :: held 0.021s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
2017-11-29 10:50:16.905 325123 DEBUG nova.compute.manager [req-b62c53d2-13db-4fac-a125-409b4f046418 8f883df20fce46dbef3ce634610c51be53b87e658359f05b7eba1062ce7e5d8b 5b54e36678a542d899f1ff62268fc25a - - -] [instance: 86cc16bd-5f51-402b-aa03-01ab0c3ffaf4] Insufficient compute resources: Requested instance NUMA topology cannot fit the given host NUMA topology. _build_and_run_instance /usr/lib/python2.7/site-packages/nova/compute/manager.py:1934
~~~

This of course looks very confusing (we need 32 GB, we have 64, so why is this failing?).

The problem here is the log output, which is flawed --- /usr/lib/python2.7/site-packages/nova/virt/hardware.py:
~~~
    840     if host_cell.avail_memory < instance_cell.memory:
    841         LOG.debug('Not enough available memory to schedule instance. '
    842                   'Oversubscription is not possible with pinned instances. '
    843                   'Required: %(required)s, actual: %(actual)s',
    844                   {'required': instance_cell.memory,
    845                    'actual': host_cell.memory})
    846         return
~~~

This should be:
~~~
    840     if host_cell.avail_memory < instance_cell.memory:
    841         LOG.debug('Not enough available memory to schedule instance. '
    842                   'Oversubscription is not possible with pinned instances. '
    843                   'Required: %(required)s, actual: %(actual)s',
    844                   {'required': instance_cell.memory,
    845                    'actual': host_cell.avail_memory})
    846         return
~~~

Or even better:
~~~
    840     if host_cell.avail_memory < instance_cell.memory:
    841         LOG.debug('Not enough available memory to schedule instance. '
    842                   'Oversubscription is not possible with pinned instances. '
    843                   'Required: %(required)s, actual: %(actual)s, total: %(total)s',
    844                   {'required': instance_cell.memory,
    845                    'actual': host_cell.avail_memory,
    846                    'total': host_cell.memory})
    847         return
~~~

Comment 1 Sahid Ferdjaoui 2017-11-30 08:44:05 UTC
Thanks for the fix upstream. While merged I will take care of doing the backport.

Comment 10 errata-xmlrpc 2018-02-27 16:24:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:0369


Note You need to log in before you can comment on or make changes to this bug.