Bug 1678708 - Soft anti-affinity not working as expected [NEEDINFO]
Summary: Soft anti-affinity not working as expected
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.2.6
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ovirt-4.3.4
: 4.3.0
Assignee: Andrej Krejcir
QA Contact: Polina
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-19 12:56 UTC by Gajanan
Modified: 2020-08-03 15:28 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-06 14:10:24 UTC
oVirt Team: Virt
Target Upstream Version:
akrejcir: needinfo? (gchakkar)
lsvaty: testing_plan_complete-


Attachments (Terms of Use)
soft affinity rule (34.15 KB, image/jpeg)
2019-02-19 12:56 UTC, Gajanan
no flags Details
hosts screenshot (83.46 KB, image/jpeg)
2019-02-19 12:58 UTC, Gajanan
no flags Details
vm start (87.64 KB, image/jpeg)
2019-02-19 12:58 UTC, Gajanan
no flags Details
soft affinity rule with host affinity positive (27.61 KB, image/png)
2019-02-20 14:36 UTC, Gajanan
no flags Details
VM start on wrong hosts when host affinity positive (62.80 KB, image/png)
2019-02-20 14:37 UTC, Gajanan
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3923051 None None None 2019-02-19 15:12:39 UTC

Description Gajanan 2019-02-19 12:56:38 UTC
Created attachment 1536308 [details]
soft affinity rule

Description of problem: 

Customer reported issue as below:

I have 3 servers in an soft anti-affinity group. This still leads to two of those VMs being started on the same server.  
Expectation:  as long as there are hardware hypervisors with plenty of free resources the anti-affinity would work

------------------------------------------


I tried to reproduce it at my end, the soft affinity indeed does not work as expected. 

See the attached screenshots for the settings and the observations in my test environment. 

Observations: 

I have three hosts in my environment, and two vms. 

I configured soft affinity where: 

~~~~~~~~
Vm affinity rule > Negative
Host affinity rule > Negative 

Virtual Machines   :   Hosts 

test2                   RHV420H2       <<<< test2 should start on RHV420H2
testz                   root            <<< testz should start on root 
~~~~~~~~~~

But vms does not start as expected, sometimes it does start as above and sometimes it does not.

Comment 1 Gajanan 2019-02-19 12:58:24 UTC
Created attachment 1536321 [details]
hosts screenshot

Comment 2 Gajanan 2019-02-19 12:58:46 UTC
Created attachment 1536322 [details]
vm start

Comment 3 Gajanan 2019-02-19 13:02:26 UTC
Customers Environment is: rhvm-4.2.6.4-0.1.el7ev.noarch 

My test environment is: rhvm-4.2.7.5-0.1.el7ev.noarch

Comment 4 Ryan Barry 2019-02-20 00:26:03 UTC
Closing.

The description here is unclear, and it's almost certainly a duplicate of either https://bugzilla.redhat.com/show_bug.cgi?id=1651747 or https://bugzilla.redhat.com/show_bug.cgi?id=1594810

Soft affinity is a best effort, and hard requirement should use hard affinity. In any case, there will be scheduler improvements in 4.3 which may mitigate this, but it's always going to be "best effort" support

Comment 5 Gajanan 2019-02-20 11:09:12 UTC
Hello Ryan,

Let me know what details you need here.

Comment 6 Ryan Barry 2019-02-20 11:44:35 UTC
There are really 2:

1) why not hard affinity?

2) how does this differ from https://bugzilla.redhat.com/show_bug.cgi?id=1651747 ?

In general, bugs like "Foo does not work" or "improve Foo" don't give us a clear idea of what to work on. Your initial comment contains the expected result, but it's quite broad, and still appears to be a duplicate. Why not attach the case to the other bug?

Comment 7 Andrej Krejcir 2019-02-20 13:32:14 UTC
It looks like there are two issues here:

1. In the screenshot of the affinity group, the host affinity is set to 'Negative', which means that the VMs should NOT run on any of the selected hosts.

2. The default importance of VM soft affinity is low. When starting a VM, the soft affinity may be overridden by other weight modules, for example CPU load or free memory.
The importance can be increased by creating a custom scheduling policy and increasing the factor for "VmAffinityGroups" weight module.


This does not look like a bug. Increasing the factor of the weight module should make the scheduler prefer soft affinity over CPU load or memory load.

Comment 8 Gajanan 2019-02-20 14:36:09 UTC
Hello Andrej, 

1) I have tested again with host affinity  set to 'Positive', it still does not work. Attaching screenshot of it to the bug. 

2) There is no load on the hosts here. So still that should be under consideration ?

Comment 9 Gajanan 2019-02-20 14:36:57 UTC
Created attachment 1536712 [details]
soft affinity rule with host affinity positive

Comment 10 Gajanan 2019-02-20 14:37:37 UTC
Created attachment 1536714 [details]
VM start on wrong hosts when  host affinity positive

Comment 16 Michal Skrivanek 2019-03-11 12:51:02 UTC
comment #7 is still relevant then. Please include logs (in debug, ideally) when you see it's not working as expected.

Comment 17 Gajanan 2019-03-11 17:00:58 UTC
Hello Michal,

Which all logs are required here ?

Comment 18 Ryan Barry 2019-03-11 17:02:13 UTC
Engine logs, hopefully in debug

Comment 19 Gajanan 2019-03-14 17:03:32 UTC
Hello Ryan, 

I am unable to find a way to get logs in debug for RHV 4.x version

I found one but its for RHEV 3.x https://access.redhat.com/solutions/435333

Comment 20 Ryan Barry 2019-03-14 22:26:30 UTC
These can now be found in /usr/share/ovirt-engine/services/ovirt-engine

ovirt-engine-logging.properties.in is the easiest way to do it


Note You need to log in before you can comment on or make changes to this bug.