Bug 1469073 - [RFE] Implement CPU weighter filter
[RFE] Implement CPU weighter filter
Status: POST
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
10.0 (Newton)
Unspecified Unspecified
high Severity high
: Upstream M2
: 14.0 (Rocky)
Assigned To: Stephen Finucane
Joe H. Rahme
: FutureFeature, Triaged
: 1246461 (view as bug list)
Depends On:
Blocks: 1476900 1565089 1571286 1573209 1573210 1576882 1576885 1576887 1576888 1582406 1582409 1582412 1582413
  Show dependency treegraph
Reported: 2017-07-10 06:56 EDT by Pablo Iranzo Gómez
Modified: 2018-05-25 03:10 EDT (History)
18 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1565089 1576882 (view as bug list)
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3109111 None None None 2017-07-10 07:19 EDT
OpenStack gerrit 379525 None None None 2017-07-10 06:56 EDT

  None (edit)
Description Pablo Iranzo Gómez 2017-07-10 06:56:38 EDT
Description of problem:

Currently, hosts are not filtered based on used VCPU's on the system (upstream was proposed and abandoned at https://review.openstack.org/#/c/379525/ )

And systems can get over subscription

Version-Release number of selected component (if applicable):

How reproducible:

Initial status:
compute-0: host (source) migrating instances (4 instances, 2 vCPU each)
compute-1: 2 free vCPUs
compute-2: 2 free vCPUs
compute-3: 4 free vCPUs

- 1 VM (2 vCPU) to compute-1 (could hold 1 vm, got 0)  ← OK:   should got 1 VM's got 1
- 0 VM to compute-2          (could hold 1 vm, got 1)  ← WRONG should got 1 VM's not 0
- 3 VM (6 vCPU) to compute-3 (could hold 2 vm, got 3)  ← WRONG should got 2 VM's not 3

Actual results:

Hosts get oversubscribed

Expected results:

No oversubscription of VCPU's

Additional info:
Comment 3 Aviv Guetta 2017-10-29 05:01:31 EDT
https://review.openstack.org/#/c/379525/ was abandoned.
Reason (from Launchpad - sfinucan@redhat.com):
The real concern I see with that change is that we're working on using the placement API for providing the best destinations related to the existing resources (CPU, memory, disk). While it's fine to merge that one, it would mean that it's not waiting for the scheduler using the placement API, which could be a bit sad. The real issue is that it's really easy to add a custom weighter out-of-tree without needing to add it in the Nova tree, so that's not really needed, see?
I did this fairly quickly while working on the PCI-NUMA filter here. The reason for doing this is that it seemed like a filter that had a clear use case and one I really expected to see there already. If you don't think it's as useful as I think it is, then I'm fine to abandon this :)
Comment 4 Aviv Guetta 2017-10-31 03:27:07 EDT
Following the abandon of upstream bug (see comment #3), Can engineering provide more info about how to achieve a better match of placement destinations in OSP 10?
Comment 7 Stephen Finucane 2017-11-21 09:59:21 EST
Apologies for the delay - I have been on PTO.

I have discussed this upstream and it seems this is something that will *not* be covered by placement. As such, I have reopened the review. However, this kind of feature requires a blueprint and the deadline for this during the current cycle closed some time ago. We do not carry features that are not available upstream, so this is something that would not be available until OSP 14 at the earliest. In addition, we do not tend to backport features (compared to bugfixes), though this might (might) qualify as an exception. I will ask.
Comment 8 Aviv Guetta 2017-11-28 05:32:37 EST
Hi Stefan,
Any update?

Comment 11 Stephen Finucane 2017-11-30 05:09:55 EST
@Aviv. Unfortunately this is considered a feature so it would require an approved blueprint to continue work on it. The window for this during the Queens cycle (what will become OSP 17) has already closed, so this will be delayed until Rocky (OSP 18). Fortunately this is something that would be easily backported, at least from a code perspective, and once merged upstream we could assess the viability of doing so.

I'll also note that weighers and filters are an area that the nova team explicitly allow custom code to run. It's possible that the customer could use the upstream filter in their own code. I don't know if this is supported from a Red Hat perspective though, so you'll need to ask around for info on this.
Comment 12 Aviv Guetta 2017-12-07 03:25:10 EST
Hi Stephen, 
I think you missed the OSP release numbers, Queens will be OSP 13, Rocky 14.

Using upstream code by the customer is optional, but unsupported. We're urging them not to do that and we'll try to avoid it asap.

Is a one-of backport for OSP 10 is feasible somehow? 

Comment 13 Stephen Finucane 2017-12-08 08:48:34 EST
Oops - I was using the upstream nova release numbers.

A one-off backport might be possible and is something I need to discuss. However, just to set expectations for there to be a backport, there must first be a feature to backport. As noted above, this can't happen until OSP 14.
Comment 14 Aviv Guetta 2018-03-08 04:31:51 EST
Hi Stephen,
Can you suggest (if any) a workaround for OSP 10?

Comment 15 Stephen Finucane 2018-03-08 13:18:03 EST
I think the weigher is the only practical approach, unfortunately. Given that the Rocky window is open upstream, I will see if we get this approved for the upcoming release.
Comment 17 Sylvain Bauza 2018-04-09 08:31:08 EDT
*** Bug 1246461 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.