Bug 1583968

Summary: Hosted Engine VM is selected for balancing even though the BalanceVM command is not enabled for HE
Product: [oVirt] ovirt-engine Reporter: Polina <pagranat>
Component: Backend.CoreAssignee: Andrej Krejcir <akrejcir>
Status: CLOSED CURRENTRELEASE QA Contact: Polina <pagranat>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2.2CC: bperkins, bugs, dfediuck, emarcus, pagranat, rbarry
Target Milestone: ovirt-4.3.0Keywords: Automation, Reopened
Target Release: 4.3.0Flags: rule-engine: ovirt-4.3+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: ovirt-engine-4.3.0_alpha Doc Type: Bug Fix
Doc Text:
The self-hosted engine VM was selected for balancing although the BalanceVM command was not enabled for the self-hosted engine. In this release, balancing is no longer blocked.
Story Points: ---
Clone Of:
: 1639650 (view as bug list) Environment:
Last Closed: 2019-02-13 07:44:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1639650    
Attachments:
Description Flags
logs and screenshot
none
engine log and screenshot
none
engine with debug none

Description Polina 2018-05-30 05:57:31 UTC
Created attachment 1445655 [details]
logs and screenshot

Description of problem: when host0 is configured CPU and memory overutilized, and the host1 with memory underutilized, the VMs from host0 must be balanced to host1. Instead, the Hosted Engine VM is attempted to be balanced and fails. Regular VM remains on the overutilized host.

Version-Release number of selected component (if applicable): rhv-release-4.2.4-1-001.noarch

How reproducible:100%

Steps to Reproduce:
1. Configure 
    Host_0 CPU over utilized and memory overutilized
    Host_1 CPU normal utilized and memory underutilized

Expected results: VM from Host_0 (golden_env_mixed_virtio_1_0 - see the system overview) must migrate on the Host_1

Actual results: tries to balance HE VM. Balancing VM HostedEngine messages in webUI Tasks tab (please see the screenshot in attached logs).

Additional info: system overview:
{
    "Default": {
        "hosts": {}, 
        "id": "8c3043a8-8fad-4ba9-9d6a-989edf8f603f", 
        "policy": {
            "none": {
                "balances": {
                    "None": {
                        "id": "38440000-8cf0-14bd-c43e-10b96e4ef00a"
                    }
                }, 
                "filters": {
                    "CPU": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f720cddc"
                    }, 
                    "CPU-Level": {
                        "id": "438b052c-90ab-40e8-9be0-a22560202ea6"
                    }, 
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "Compatibility-Version": {
                        "id": "3e4a7d54-9e7f-11e5-8994-feff819cdc9f"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "Emulated-Machine": {
                        "id": "58894b5b-d55d-4f85-8f82-5bf217e640b0"
                    }, 
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "HostedEngineSpares": {
                        "id": "53bff075-8306-446f-a53b-9c872a29d197"
                    }, 
                    "HugePages": {
                        "id": "23e07b34-66dd-4735-bc45-bcddded02c05"
                    }, 
                    "Label": {
                        "id": "27846536-f653-11e5-9ce9-5e5517507c66"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "Swap": {
                        "id": "43b9e654-337f-4b0d-a896-cbb22ce5a7fc"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "b4ed2332-a7ac-4d5f-9596-99a439cb2812", 
                "weights": {
                    "HA": {
                        "id": "98e92667-6161-41fb-b3fa-34f820ccbc4b"
                    }, 
                    "OptimalForCpuEvenDistribution": {
                        "id": "7db4ab05-81ab-42e8-868a-aee2df483edb"
                    }, 
                    "OptimalForHaReservation": {
                        "id": "7f262d70-6cac-11e3-981f-0800200c9a66"
                    }, 
                    "OptimalForMemoryEvenDistribution": {
                        "id": "4134247a-9c58-4b9a-8593-530bb9e37c59"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db567"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "427aed70-dae3-48ba-8fe9-a902a9d563c8"
                    }
                }
            }
       }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "2", 
            "HighUtilization": "80"
        }
    }, 
    "golden_env_mixed_1": {
        "hosts": {
            "host_mixed_1": {
                "id": "4f8b8633-3bc4-4bfd-a970-4991a44e7d8b", 
                "max_scheduling_memory": "11423MB", 
                "status": "up", 
                "vms": {
                    "golden_env_mixed_virtio_1_0": {
                        "guaranteed_memory": "1024MB", 
                        "id": "b20aef7b-ce01-481b-9016-af6bf2d4ff00", 
                        "memory": "1024MB", 
                        "status": "up"
                    }, 
                    "vm_overutilized_0": {
                        "guaranteed_memory": "18853MB", 
                        "id": "a94dc1d9-f0ed-4348-b590-0baa762def93", 
                        "memory": "18853MB", 
                        "status": "up"
                    }
                }
            }, 
            "host_mixed_2": {
                "id": "eb96e250-9635-4921-8b71-524fc12a6772", 
                "max_scheduling_memory": "30392MB", 
                "status": "up", 
                "vms": {
                    "golden_env_mixed_virtio_1_1": {
                        "guaranteed_memory": "1024MB", 
                        "id": "59161df1-a5d1-4078-8ed0-35813bf7645b", 
                        "memory": "1024MB", 
                        "status": "up"
                    }
                }
            }, 
            "host_mixed_3": {
                "id": "4a5b6eb3-046d-4aea-88c1-a53e0d962c6c", 
                "max_scheduling_memory": "12412MB", 
                "status": "up", 
                "vms": {
                    "HostedEngine": {
                        "guaranteed_memory": "1024MB", 
                        "id": "9db72273-1cf5-4ba6-b95e-33ffe47fea6e", 
                        "memory": "16384MB", 
                        "status": "up"
                    }, 
                    "golden_env_mixed_virtio_3_0": {
                        "guaranteed_memory": "15581MB", 
                        "id": "23d66fd5-cb45-49f9-8c93-c1952070c4db", 
                        "memory": "15581MB", 
                        "status": "up"
                    }, 
                    "vm_overutilized_2": {
                        "guaranteed_memory": "18853MB", 
                        "id": "a865d9b2-a054-4aa7-a1b4-77bab982b9d1", 
                        "memory": "18853MB", 
                        "status": "up"
                    }
                }
            }
        }, 
        "id": "5a7bb8fc-61f1-11e8-af04-00163e7be006", 
        "policy": {
            "custom_power_saving_cpu": {
                "balances": {
                    "OptimalForPowerSaving": {
                        "id": "736999d0-1023-46a4-9a75-1316ed50e151"
                    }
                }, 
                "filters": {
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "fbc29a3c-91af-45bd-b810-38856e5afcd3", 
                "weights": {
                    "OptimalForCpuPowerSaving": {
                        "id": "736999d0-1023-46a4-9a75-1316ed50e15b"
                    }, 
                    "OptimalForMemoryPowerSaving": {
                        "id": "9dfe6086-646d-43b8-8eef-4d94de8472c8"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }
                }
            }
        }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "1", 
            "HighUtilization": "75", 
            "LowUtilization": "35", 
            "MaxFreeMemoryForOverUtilized": "13733", 
            "MinFreeMemoryForUnderUtilized": "17829"
        }
    }, 
    "golden_env_mixed_2": {
        "hosts": {}, 
        "id": "341efeb6-de37-4044-b64c-ec7396ea09fd", 
        "policy": {
            "none": {
                "balances": {
                    "None": {
                        "id": "38440000-8cf0-14bd-c43e-10b96e4ef00a"
                    }
                }, 
                "filters": {
                    "CPU": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f720cddc"
                    }, 
                    "CPU-Level": {
                        "id": "438b052c-90ab-40e8-9be0-a22560202ea6"
                    }, 
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "Compatibility-Version": {
                        "id": "3e4a7d54-9e7f-11e5-8994-feff819cdc9f"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "Emulated-Machine": {
                        "id": "58894b5b-d55d-4f85-8f82-5bf217e640b0"
                    }, 
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "HostedEngineSpares": {
                        "id": "53bff075-8306-446f-a53b-9c872a29d197"
                    }, 
                    "HugePages": {
                        "id": "23e07b34-66dd-4735-bc45-bcddded02c05"
                    }, 
                    "Label": {
                        "id": "27846536-f653-11e5-9ce9-5e5517507c66"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "Swap": {
                        "id": "43b9e654-337f-4b0d-a896-cbb22ce5a7fc"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "b4ed2332-a7ac-4d5f-9596-99a439cb2812", 
                "weights": {
                    "HA": {
                        "id": "98e92667-6161-41fb-b3fa-34f820ccbc4b"
                    }, 
                    "OptimalForCpuEvenDistribution": {
                        "id": "7db4ab05-81ab-42e8-868a-aee2df483edb"
                    }, 
                    "OptimalForHaReservation": {
                        "id": "7f262d70-6cac-11e3-981f-0800200c9a66"
                    }, 
                    "OptimalForMemoryEvenDistribution": {
                        "id": "4134247a-9c58-4b9a-8593-530bb9e37c59"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db567"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "427aed70-dae3-48ba-8fe9-a902a9d563c8"
                    }
                }
            }
        }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "2", 
            "HighUtilization": "80"
        }
    }
}

Comment 1 Doron Fediuck 2018-05-31 10:00:19 UTC
Thanks for the report.
Please note that you created your own policy instead of using the built-in one:

        "policy": {
            "custom_power_saving_cpu": {
                "balances": {
                    "OptimalForPowerSaving": {

Your custom policy is missing the HA filter which avoids this behavior.
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 


To summarize- if you want to test our policies, please use our policies and do not create custom ones.

Comment 2 Polina 2018-06-07 15:07:11 UTC
To exclude the effect of the customization, I repeated the test, configuring the cluster with original 'power_saving' policy instead of custom.

engine still tries to balance HE VM - "Balancing VM HostedEngine".

Please see the system overview.

{
    "Default": {
        "hosts": {}, 
        "id": "42349bf1-cbfb-4e23-b524-c59b88bfb04c", 
        "policy": {
            "none": {
                "balances": {
                    "None": {
                        "id": "38440000-8cf0-14bd-c43e-10b96e4ef00a"
                    }
                }, 
                "filters": {
                    "CPU": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f720cddc"
                    }, 
                    "CPU-Level": {
                        "id": "438b052c-90ab-40e8-9be0-a22560202ea6"
                    }, 
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "Compatibility-Version": {
                        "id": "3e4a7d54-9e7f-11e5-8994-feff819cdc9f"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "Emulated-Machine": {
                        "id": "58894b5b-d55d-4f85-8f82-5bf217e640b0"
                    }, 
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "HostedEngineSpares": {
                        "id": "53bff075-8306-446f-a53b-9c872a29d197"
                    }, 
                    "HugePages": {
                        "id": "23e07b34-66dd-4735-bc45-bcddded02c05"
                    }, 
                    "Label": {
                        "id": "27846536-f653-11e5-9ce9-5e5517507c66"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "Swap": {
                        "id": "43b9e654-337f-4b0d-a896-cbb22ce5a7fc"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "b4ed2332-a7ac-4d5f-9596-99a439cb2812", 
                "weights": {
                    "HA": {
                        "id": "98e92667-6161-41fb-b3fa-34f820ccbc4b"
                    }, 
                    "OptimalForCpuEvenDistribution": {
                        "id": "7db4ab05-81ab-42e8-868a-aee2df483edb"
                    }, 
                    "OptimalForHaReservation": {
                        "id": "7f262d70-6cac-11e3-981f-0800200c9a66"
                    }, 
                    "OptimalForMemoryEvenDistribution": {
                        "id": "4134247a-9c58-4b9a-8593-530bb9e37c59"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db567"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "427aed70-dae3-48ba-8fe9-a902a9d563c8"
                    }
                }
            }
        }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "2", 
            "HighUtilization": "80"
        }
    }, 
    "golden_env_mixed_1": {
        "hosts": {
            "host_mixed_1": {
                "id": "2ca499d4-f9bd-4100-8b7b-7d4c50cb4463", 
                "max_scheduling_memory": "19260MB", 
                "status": "up", 
                "vms": {
                    "golden_env_mixed_virtio_1_0": {
                        "guaranteed_memory": "1024MB", 
                        "id": "17194a48-bfef-4877-b6ae-aa3630658798", 
                        "memory": "1024MB", 
                        "status": "up"
                    }, 
                    "golden_env_mixed_virtio_3_1": {
                        "guaranteed_memory": "8319MB", 
                        "id": "adaa17b9-68b2-4204-b1ba-dae2722dd700", 
                        "memory": "8319MB", 
                        "status": "up"
                    }, 
                    "vm_normalutilized_0": {
                        "guaranteed_memory": "18657MB", 
                        "id": "6b274874-4d1c-4073-9e68-752e3d2418b8", 
                        "memory": "18657MB", 
                        "status": "up"
                    }
                }
            }, 
            "host_mixed_2": {
                "id": "bf84842f-2b7d-4101-8a07-bb1f2b96cc46", 
                "max_scheduling_memory": "38097MB", 
                "status": "up", 
                "vms": {
                    "golden_env_mixed_virtio_1_1": {
                        "guaranteed_memory": "1024MB", 
                        "id": "37a28f46-0b6b-4303-9306-380085ed54e2", 
                        "memory": "1024MB", 
                        "status": "up"
                    }, 
                    "golden_env_mixed_virtio_3_0": {
                        "guaranteed_memory": "8319MB", 
                        "id": "9bcf42f3-459c-4875-8f81-bfefdcdb3a36", 
                        "memory": "8319MB", 
                        "status": "up"
                    }
                }
            }, 
            "host_mixed_3": {
                "id": "9043c1b4-ef60-4bdb-bdff-bd56e03f4adc", 
                "max_scheduling_memory": "39362MB", 
                "status": "up", 
                "vms": {
                    "HostedEngine": {
                        "guaranteed_memory": "8192MB", 
                        "id": "3ae6ec39-681a-4eae-85e8-64721c90a44c", 
                        "memory": "8192MB", 
                        "status": "up"
                    }
                }
            }
        }, 
        "id": "f8f8b41a-5361-11e8-9e69-001a4a162166", 
        "policy": {
            "power_saving": {
                "balances": {
                    "OptimalForPowerSaving": {
                        "id": "736999d0-1023-46a4-9a75-1316ed50e151"
                    }
                }, 
                "filters": {
                    "CPU": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f720cddc"
                    }, 
                    "CPU-Level": {
                        "id": "438b052c-90ab-40e8-9be0-a22560202ea6"
                    }, 
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "Compatibility-Version": {
                        "id": "3e4a7d54-9e7f-11e5-8994-feff819cdc9f"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "Emulated-Machine": {
                        "id": "58894b5b-d55d-4f85-8f82-5bf217e640b0"
                    }, 
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "HostedEngineSpares": {
                        "id": "53bff075-8306-446f-a53b-9c872a29d197"
                    }, 
                    "HugePages": {
                        "id": "23e07b34-66dd-4735-bc45-bcddded02c05"
                    }, 
                    "Label": {
                        "id": "27846536-f653-11e5-9ce9-5e5517507c66"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "Swap": {
                        "id": "43b9e654-337f-4b0d-a896-cbb22ce5a7fc"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "5a2b0939-7d46-4b73-a469-e9c2c7fc6a53", 
                "weights": {
                    "HA": {
                        "id": "98e92667-6161-41fb-b3fa-34f820ccbc4b"
                    }, 
                    "OptimalForCpuPowerSaving": {
                        "id": "736999d0-1023-46a4-9a75-1316ed50e15b"
                    }, 
                    "OptimalForHaReservation": {
                        "id": "7f262d70-6cac-11e3-981f-0800200c9a66"
                    }, 
                    "OptimalForMemoryPowerSaving": {
                        "id": "9dfe6086-646d-43b8-8eef-4d94de8472c8"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db567"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "427aed70-dae3-48ba-8fe9-a902a9d563c8"
                    }
                }
            }
        }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "1", 
            "HeSparesCount": "0", 
            "HighUtilization": "75", 
            "LowUtilization": "35", 
            "MaxFreeMemoryForOverUtilized": "17633", 
            "MinFreeMemoryForUnderUtilized": "21729"
        }
    }, 
    "golden_env_mixed_2": {
        "hosts": {}, 
        "id": "9b066b08-c13a-45a8-940c-79cf23fefc1c", 
        "policy": {
            "none": {
                "balances": {
                    "None": {
                        "id": "38440000-8cf0-14bd-c43e-10b96e4ef00a"
                    }
                }, 
                "filters": {
                    "CPU": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f720cddc"
                    }, 
                    "CPU-Level": {
                        "id": "438b052c-90ab-40e8-9be0-a22560202ea6"
                    }, 
                    "CPUOverloaded": {
                        "id": "98842bc5-4094-4b83-8224-7b50f86a94c9"
                    }, 
                    "Compatibility-Version": {
                        "id": "3e4a7d54-9e7f-11e5-8994-feff819cdc9f"
                    }, 
                    "CpuPinning": {
                        "id": "6d636bf6-a35c-4f9d-b68d-0731f731cddc"
                    }, 
                    "Emulated-Machine": {
                        "id": "58894b5b-d55d-4f85-8f82-5bf217e640b0"
                    }, 
                    "HA": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08dffd"
                    }, 
                    "HostDevice": {
                        "id": "728a21f1-f97e-4d32-bc3e-b3cc49756abb"
                    }, 
                    "HostedEngineSpares": {
                        "id": "53bff075-8306-446f-a53b-9c872a29d197"
                    }, 
                    "HugePages": {
                        "id": "23e07b34-66dd-4735-bc45-bcddded02c05"
                    }, 
                    "Label": {
                        "id": "27846536-f653-11e5-9ce9-5e5517507c66"
                    }, 
                    "Memory": {
                        "id": "c9ddbb34-0e1d-4061-a8d7-b0893fa80932"
                    }, 
                    "Migration": {
                        "id": "e659c871-0bf1-4ccc-b748-f28f5d08ddda"
                    }, 
                    "Network": {
                        "id": "72163d1c-9468-4480-99d9-0888664eb143"
                    }, 
                    "PinToHost": {
                        "id": "12262ab6-9690-4bc3-a2b3-35573b172d54"
                    }, 
                    "Swap": {
                        "id": "43b9e654-337f-4b0d-a896-cbb22ce5a7fc"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db566"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "e69808a9-8a41-40f1-94ba-dd5d385d82d8"
                    }
                }, 
                "id": "b4ed2332-a7ac-4d5f-9596-99a439cb2812", 
                "weights": {
                    "HA": {
                        "id": "98e92667-6161-41fb-b3fa-34f820ccbc4b"
                    }, 
                    "OptimalForCpuEvenDistribution": {
                        "id": "7db4ab05-81ab-42e8-868a-aee2df483edb"
                    }, 
                    "OptimalForHaReservation": {
                        "id": "7f262d70-6cac-11e3-981f-0800200c9a66"
                    }, 
                    "OptimalForMemoryEvenDistribution": {
                        "id": "4134247a-9c58-4b9a-8593-530bb9e37c59"
                    }, 
                    "PreferredHosts": {
                        "id": "591cdb81-ba67-45b4-9642-e28f61a97d57"
                    }, 
                    "VmAffinityGroups": {
                        "id": "84e6ddee-ab0d-42dd-82f0-c297779db567"
                    }, 
                    "VmToHostsAffinityGroups": {
                        "id": "427aed70-dae3-48ba-8fe9-a902a9d563c8"
                    }
                }
            }
        }, 
        "policy_params": {
            "CpuOverCommitDurationMinutes": "2", 
            "HighUtilization": "80"
        }
    }
}

Comment 3 Polina 2018-06-07 15:22:54 UTC
Created attachment 1448753 [details]
engine log and screenshot

Comment 4 Martin Sivák 2018-06-13 12:22:03 UTC
But balancing the Hosted Engine VM is perfectly fine. It is just a VM. What exactly do you mean by "fails"? Is there a reason the engine VM can't move?

Comment 5 Polina 2018-06-17 14:57:46 UTC
The environment has VM on hist1 with 1024 MB memory size which is expected to be migrated. Though the balancer chooses the Hosted Engine VM (which is NON_MANAGED_VM) and fails in balancing with error in Tasks.

I've run the test with full debug (see debug configuration below).

In engine.log you can see an error (please see the attachment) 
2018-06-17 17:32:54,244+03 WARN  [org.ovirt.engine.core.bll.BalanceVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [dab8e20] Validation of action 'BalanceVm' failed for user SYSTEM. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,ACTION_TYPE_FAILED_CANNOT_RUN_ACTION_ON_NON_MANAGED_VM



debug configuration-
      <logger category="org.ovirt.engine.core.bll">
        <level name="DEBUG"/>
      </logger>
      <logger category="org.ovirt.engine.core.bll.scheduling">
        <level name="DEBUG"/>
      </logger>

Comment 6 Polina 2018-06-17 14:58:53 UTC
Created attachment 1452391 [details]
engine with debug

Comment 7 Martin Sivák 2018-06-22 11:23:06 UTC
The issue is here: org.ovirt.engine.core.common.validation.VmActionByVmOriginTypeValidator - BalanceVmCommand is not allowed for hosted engine.


So the question is whether we should allow balancing of hosted engine. Notice for example that migration of the hosted engine VM is already supported and there is not much difference between those two. It would also remove another difference between hosted engine and regular VMs.

Comment 8 Yaniv Lavi 2018-06-24 14:33:22 UTC
(In reply to Martin Sivák from comment #7)
> The issue is here:
> org.ovirt.engine.core.common.validation.VmActionByVmOriginTypeValidator -
> BalanceVmCommand is not allowed for hosted engine.
> 
> 
> So the question is whether we should allow balancing of hosted engine.
> Notice for example that migration of the hosted engine VM is already
> supported and there is not much difference between those two. It would also
> remove another difference between hosted engine and regular VMs.

Let's remove this limitation, it seems unnecessary at this point.

Comment 9 Polina 2018-11-29 12:10:40 UTC
The verification is done in ovirt-engine-4.3.0-0.2.master.20181121071050.gita8fcd23.el7.noarch.

The Hosted Engine VM migration could be initiated by the Balancer now. 
It is reported in engine log , like

2018-11-29 11:30:48,266+02 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [2e67b425] EVENT_ID: VM_MIGRATION_START_SYSTEM_INITIATED(67), Migration initiated by system (VM: HostedEngine, Source: host_mixed_3, Destination: host_mixed_1, Reason: Load balancing).
...
2018-11-29 11:34:56,699+02 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] VM '60b6d114-83ec-4061-9e9f-0df6456012cc'(HostedEngine) moved from 'MigratingFrom' --> 'Down'
2018-11-29 11:34:56,701+02 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-12) [] Handing over VM '60b6d114-83ec-4061-9e9f-0df6456012cc'(HostedEngine) to Host 'b0838dcd-9fc1-40b8-9802-a6e49cc8755d'. Setting VM to status 'MigratingTo'
2018-11-29 11:34:57,279+02 INFO  [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-5) [] VM '60b6d114-83ec-4061-9e9f-0df6456012cc'(HostedEngine) moved from 'MigratingTo' --> 'Up'

Comment 10 Sandro Bonazzola 2019-02-13 07:44:56 UTC
This bugzilla is included in oVirt 4.3.0 release, published on February 4th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.