Bug 1411716 - [nova] [SR-IOV] [NUMA] failed to create sriov VM
Summary: [nova] [SR-IOV] [NUMA] failed to create sriov VM
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 9.0 (Mitaka)
Hardware: All
OS: Linux
high
urgent
Target Milestone: async
: 9.0 (Mitaka)
Assignee: Stephen Finucane
QA Contact: Joe H. Rahme
URL:
Whiteboard:
Depends On:
Blocks: 1417617
TreeView+ depends on / blocked
 
Reported: 2017-01-10 11:28 UTC by VIKRANT
Modified: 2020-04-15 15:04 UTC (History)
15 users (show)

Fixed In Version: openstack-nova-13.1.2-13.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1417617 (view as bug list)
Environment:
Last Closed: 2017-03-08 17:46:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 419479 0 None None None 2017-01-12 16:09:44 UTC
Red Hat Knowledge Base (Solution) 2533751 0 None None None 2017-01-10 11:36:33 UTC
Red Hat Knowledge Base (Solution) 2859281 0 None None None 2017-01-17 03:42:21 UTC
Red Hat Product Errata RHBA-2017:0467 0 normal SHIPPED_LIVE openstack-nova bug fix advisory 2017-03-08 22:44:46 UTC

Description VIKRANT 2017-01-10 11:28:22 UTC
Description of problem:

Cu is failed to create sriov  VM when using following properties in flavor.

~~~
{"hw:wrs:pci_numa_affinity": "prefer", "hw:cpu_model": "SandyBridge", "hw:numa_nodes": "1", "aggregate_instance_extra_spces:vEPC_HA": "true", "hw:cpu_policy": "dedicated"} 
~~

here is the command used to spawn the instance. 

~~~
nova boot --flavor bc5eadbe-d7c3-4e35-8770-7e176d6b84d3   --block-device source=volume,id=e9f982a3-8321-4462-a735-0ffa59f7d99f,dest=volume,size=6,bootindex=0   --nic port-id=5d6db89a-10fd-40ea-86d9-ec17a26ad5a4 test
~~~

Version-Release number of selected component (if applicable):
RHEL OSP 9

How reproducible:
Everytime for Cu. 

Steps to Reproduce:
1. Try to spawn the instance.
2. It's getting failed with "no valid host found" error message. 
3.

Actual results:
Instance launch operation is getting failed. 

Expected results:
It should get completed successfully. 

Additional info:

Call trace seen in nova show output.

~~~
|
| fault                                | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 392, in build_instances |
|                                      |     context, request_spec, filter_properties)                                                                                                                                                              |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 436, in _schedule_instances                                                                                                    |
|                                      |     hosts = self.scheduler_client.select_destinations(context, spec_obj)                                                                                                                                   |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/utils.py\", line 372, in wrapped                                                                                                                  |
|                                      |     return func(*args, **kwargs)                                                                                                                                                                           |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 51, in select_destinations                                                                                             |
|                                      |     return self.queryclient.select_destinations(context, spec_obj)                                                                                                                                         |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 37, in __run_method                                                                                                    |
|                                      |     return getattr(self.instance, __name)(*args, **kwargs)                                                                                                                                                 |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py\", line 32, in select_destinations                                                                                                |
|                                      |     return self.scheduler_rpcapi.select_destinations(context, spec_obj)                                                                                                                                    |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py\", line 121, in select_destinations                                                                                                     |
|                                      |     return cctxt.call(ctxt, 'select_destinations', **msg_args)                                                                                                                                             |
|                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py\", line 158, in call                                                                                                                |
|                                      |     retry=self.retry)                                                                                                                                                                                      |
|                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py\", line 90, in _send                                                                                                                 |
|                                      |     timeout=timeout, retry=retry)                                                                                                                                                                          |
|                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 470, in send                                                                                                       |
|                                      |     retry=retry)                                                                                                                                                                                           |
|                                      |   File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 461, in _send                                                                                                      |
|                                      |     raise result                                                                                                                                                                                           |
|                                      | ", "created": 
~~~


Output of nova flavor.

~~~
[root@slot5 ~]# nova flavor-show bc5eadbe-d7c3-4e35-8770-7e176d6b84d3 
+----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property                   | Value                                                                                                                                                                       |
+----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                                                                                                                                                       |
| OS-FLV-EXT-DATA:ephemeral  | 0                                                                                                                                                                           |
| disk                       | 0                                                                                                                                                                           |
| extra_specs                | {"hw:wrs:pci_numa_affinity": "prefer", "hw:cpu_model": "SandyBridge", "hw:numa_nodes": "1", "aggregate_instance_extra_spces:vEPC_HA": "true", "hw:cpu_policy": "dedicated"} |
| id                         | bc5eadbe-d7c3-4e35-8770-7e176d6b84d3                                                                                                                                        |
| name                       | vnfm_flavor_5584e475-23b4-434a-b7d2-c9b72a54e398                                                                                                                            |
| os-flavor-access:is_public | True                                                                                                                                                                        |
| ram                        | 32768                                                                                                                                                                       |
| rxtx_factor                | 1.0                                                                                                                                                                         |
| swap                       |                                                                                                                                                                             |
| vcpus                      | 8                                                                                                                                                                           |
+----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
~~~

Comment 17 Stephen Finucane 2017-01-11 15:43:36 UTC
I have built a test package [1] that adds a couple of additional logs along with a potential fix for the issue. Could you use this build and attempt to reproduce the issue?

[1] https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=12337457

Comment 50 errata-xmlrpc 2017-03-08 17:46:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0467.html


Note You need to log in before you can comment on or make changes to this bug.