Description of problem: Even though high-performance VMs do not support live migration, they need to be highly-available - and that requires pinning to more than one host. Currently, because of the vNUMA, it'll fail (via the API) with: 2018-04-20 04:39:34,812-04 WARN [org.ovirt.engine.core.bll.numa.vm.AddVmNumaNodesCommand] (default task-4) [b31ad1f6-a519-4d80-986c-4f62bf3531e3] Validation of action 'AddVmNumaNodes' failed for user admin@internal-authz. Reasons: ACTION_TYPE_FAILED_VM_PINNED_TO_MULTIPLE_HOSTS 2018-04-20 04:39:34,813-04 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-4) [b31ad1f6-a519-4d80-986c-4f62bf3531e3] method: runAction, params: [AddVmNumaNodes, VmNumaNodeOperationParameters:{commandId='300721ea-f192-4192-b8ea-8d3a6b4f715d', user='null', commandType='Unknown', vmId='518a4a08-8813-49f4-9f01-dde2e2836d68'}], timeElapsed: 29ms 2018-04-20 04:39:34,818-04 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-4) [] Operation Failed: [Cannot ${action} ${type}. VM must be pinned to a single host.] We must allow it, at least via the API (and perhaps warn on the UI?) Version-Release number of selected component (if applicable): 4.2.3
I am targeting it for consideration in 4.3 and we can implement something very limited (allow removing the pinning restriction) under the assumption that all hosts have the same numa topology for 4.2 if the PMs agree to it.
(In reply to Martin Sivák from comment #1) > I am targeting it for consideration in 4.3 and we can implement something > very limited (allow removing the pinning restriction) under the assumption > that all hosts have the same numa topology for 4.2 if the PMs agree to it. Martin (S.) - thanks, I think it makes sense. Perhaps in 4.3 we can give a warning in the UI or something ('Make sure all your pinned hosts have correct NUMA topology...' or something). Also note that we want to allow live-migration of high-perf. VMs in 4.3 (and potentially backport this to 4.2.z, if feasible). Martin (T.) - please ACK.
We can easily stop requiring the Migration disabled option from our side. But the Virt team needs to make sure it will actually work.
(In reply to Yaniv Kaul from comment #2) > (In reply to Martin Sivák from comment #1) > > I am targeting it for consideration in 4.3 and we can implement something > > very limited (allow removing the pinning restriction) under the assumption > > that all hosts have the same numa topology for 4.2 if the PMs agree to it. > > Martin (S.) - thanks, I think it makes sense. Perhaps in 4.3 we can give a > warning in the UI or something ('Make sure all your pinned hosts have > correct NUMA topology...' or something). Also note that we want to allow > live-migration of high-perf. VMs in 4.3 (and potentially backport this to > 4.2.z, if feasible). > > Martin (T.) - please ACK. I totally agree to this approach. Acked.
Verification on ovirt-engine-4.3.0-0.4.master.20190106162157.gitd96a412.el7.noarch. Tests for verification: 1. Pin VM to multiple hosts supporting numa, configure HP VM with 12 CPU, configure 4 vnuma nodes and pin two vnuma per numa node , configure manually cpu pinning topology in Resource Allocation tab 0#0_1#1. Migrate via UI and Rest for two options: 'manual migrated only' and 'manual and automatic'. Result : success. 2. Repeat the test1 for Desktop/Server VM. 3. Add High Availability configuration to VM with a lease. Kill the qemu process on the host. Result: The VM is restarted. 4. Repeat the test3. with HP/Desktop.
This bugzilla is included in oVirt 4.3.0 release, published on February 4th 2019. Since the problem described in this bug report should be resolved in oVirt 4.3.0 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.
Sharon, please review this doc_text: High-performance virtual machines require pinning to multiple hosts to be highly-available. Previously virtual machines with NUMA pinning enabled could not be configured to run on more than one host. Now virtual machines with NUMA pinning enabled can be configured to run on one or more hosts. To configure NUMA pinning in the Administration Portal: 1. Select Compute > Pools. 2. Select a pool and click Edit. Alternatively, click New to create a new pool. A dialog opens to edit or define the pool. 3. On the Host tab, under "Start running on" select "Specific Host(s)". 4. On the Resource Allocation tab, under CPU Allocation > CPU Pinning topology, enter the desired topology. All hosts should have the same topology so the virtual machine can run on every host. The Manager verifies the configuration.
Polina, can you please review the doc_text? See the doc_text field or comment 7.
Hi Steven, It is a little bit confusing - the first paragraph says about NUMA pinning configuration "To configure NUMA pinning in the Administration Portal:" Then 1.-4. talks about CPU Pinning topology while creating or editing the Pool and not Numa topology. Besides, the Numa topology could not be configured while the Pool editing in the Administration Portal. It is configured within 'VM Edit window'/'Host tab'/'Configure Numa section'.
Hi Polina, For now I'm getting rid of the "how to" info. I see the NUMA section you mention, but I don't know how to use it.
(In reply to Steve Goodman from comment #10) > For now I'm getting rid of the "how to" info. I see the NUMA section you > mention, but I don't know how to use it. Hi Steven, looks good to me except that I would add to the sentence " Now virtual machines with NUMA pinning enabled can be configured to run on one or more hosts" something like the following: "by allowing to set the "Start running on" select "Specific Host(s)" field to include more than one host"