Bug 1087529 - Configure neutron correctly to be able to notify nova about port changes
Summary: Configure neutron correctly to be able to notify nova about port changes
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: RDO
Classification: Community
Component: openstack-packstack
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Martin Magr
QA Contact: yeylon@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-14 14:23 UTC by Attila Fazekas
Modified: 2016-04-18 06:47 UTC (History)
5 users (show)

Fixed In Version: openstack-packstack-2014.1.1-0.9.dev1055
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-03-30 23:10:26 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
OpenStack gerrit 87009 None None None Never

Description Attila Fazekas 2014-04-14 14:23:09 UTC
Description of problem:
neutron needs to have nova credentials configured in order to be able to notify nova about port changes.

$  nova boot server --image cirros-0.3.2-x86_64-uec --flavor 42
After a long-long wait the instance status become ERROR.
 
$ nova show fb1a1ee2-1f72-4488-ac63-fce0d874fa6b
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property                             | Value                                                                                                                                                                          |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                                                                                                                                         |
| OS-EXT-AZ:availability_zone          | nova                                                                                                                                                                           |
| OS-EXT-SRV-ATTR:host                 | afazekas-f20-1.novalocal                                                                                                                                                       |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | afazekas-f20-1.novalocal                                                                                                                                                       |
| OS-EXT-SRV-ATTR:instance_name        | instance-0000000e                                                                                                                                                              |
| OS-EXT-STS:power_state               | 0                                                                                                                                                                              |
| OS-EXT-STS:task_state                | -                                                                                                                                                                              |
| OS-EXT-STS:vm_state                  | error                                                                                                                                                                          |
| OS-SRV-USG:launched_at               | -                                                                                                                                                                              |
| OS-SRV-USG:terminated_at             | -                                                                                                                                                                              |
| accessIPv4                           |                                                                                                                                                                                |
| accessIPv6                           |                                                                                                                                                                                |
| config_drive                         |                                                                                                                                                                                |
| created                              | 2014-04-14T13:01:58Z                                                                                                                                                           |
| fault                                | {"message": "Virtual Interface creation failed", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 296, in decorated_function |
|                                      |     return function(self, context, *args, **kwargs)                                                                                                                            |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2073, in run_instance                                                                                |
|                                      |     do_run_instance()                                                                                                                                                          |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py\", line 249, in inner                                                                             |
|                                      |     return f(*args, **kwargs)                                                                                                                                                  |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 2072, in do_run_instance                                                                             |
|                                      |     legacy_bdm_in_spec)                                                                                                                                                        |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1205, in _run_instance                                                                               |
|                                      |     notify(\"error\", fault=e)  # notify that build failed                                                                                                                     |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\", line 68, in __exit__                                                                            |
|                                      |     six.reraise(self.type_, self.value, self.tb)                                                                                                                               |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1189, in _run_instance                                                                               |
|                                      |     instance, image_meta, legacy_bdm_in_spec)                                                                                                                                  |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1333, in _build_instance                                                                             |
|                                      |     network_info.wait(do_raise=False)                                                                                                                                          |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\", line 68, in __exit__                                                                            |
|                                      |     six.reraise(self.type_, self.value, self.tb)                                                                                                                               |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1309, in _build_instance                                                                             |
|                                      |     set_access_ip=set_access_ip)                                                                                                                                               |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 399, in decorated_function                                                                           |
|                                      |     return function(self, context, *args, **kwargs)                                                                                                                            |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1721, in _spawn                                                                                      |
|                                      |     LOG.exception(_('Instance failed to spawn'), instance=instance)                                                                                                            |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\", line 68, in __exit__                                                                            |
|                                      |     six.reraise(self.type_, self.value, self.tb)                                                                                                                               |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/compute/manager.py\", line 1718, in _spawn                                                                                      |
|                                      |     block_device_info)                                                                                                                                                         |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py\", line 2251, in spawn                                                                                   |
|                                      |     block_device_info)                                                                                                                                                         |
|                                      |   File \"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py\", line 3654, in _create_domain_and_network                                                              |
|                                      |     raise exception.VirtualInterfaceCreateException()                                                                                                                          |
|                                      | ", "created": "2014-04-14T13:07:03Z"}                                                                                                                                          |
| flavor                               | m1.nano (42)                                                                                                                                                                   |
| hostId                               | 3c039cc461f5194ee4ed37e927222d3bcef71efb92d93806f38c746a                                                                                                                       |
| id                                   | fb1a1ee2-1f72-4488-ac63-fce0d874fa6b                                                                                                                                           |
| image                                | cirros-0.3.2-x86_64-uec (c02aa525-f048-4666-a3d5-172513dccf04)                                                                                                                 |
| key_name                             | -                                                                                                                                                                              |
| metadata                             | {}                                                                                                                                                                             |
| name                                 | server                                                                                                                                                                         |
| os-extended-volumes:volumes_attached | []                                                                                                                                                                             |
| public network                       | 172.24.4.27                                                                                                                                                                    |
| security_groups                      | default                                                                                                                                                                        |
| status                               | ERROR                                                                                                                                                                          |
| tenant_id                            | a54d656e2bfe4c50a326199d8562e1c7                                                                                                                                               |
| updated                              | 2014-04-14T13:07:03Z                                                                                                                                                           |
| user_id                              | e3bfff9d2da14c1abe7b0ff3cc46afd1                                                                                                                                               |
+--------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


several related errors from the /var/log/neutron/server.log:
2014-04-14 13:09:41.744 13303 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'6dfcede5-126c-40de-a6ac-e856af424441', 'name': 'network-vif-plugged', 'server_uuid': u'5de6b8e8-561c-40fb-a97e-205ad9147fca'}]
2014-04-14 13:14:41.876 13303 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'6dfcede5-126c-40de-a6ac-e856af424441', 'name': 'network-vif-unplugged', 'server_uuid': u'5de6b8e8-561c-40fb-a97e-205ad9147fca'}]
2014-04-14 13:16:16.139 13303 ERROR neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api [req-3ee3aa65-6cea-4bfe-a64c-e59c1e79f0a4 None] Will not send event port_create_end for network 4b120092-9c3a-4075-81fa-c4e95a1832d5: no agent available. Payload: {'port': {'status': 'DOWN', 'binding:host_id': u'afazekas-f20-1.novalocal', 'name': '', 'allowed_address_pairs': [], 'admin_state_up': True, 'network_id': u'4b120092-9c3a-4075-81fa-c4e95a1832d5', 'tenant_id': u'a227c1b3ffb8486386bbd76d2228b273', 'binding:vif_details': {'port_filter': True, 'ovs_hybrid_plug': True}, 'binding:vif_type': 'ovs', 'device_owner': u'compute:None', 'mac_address': 'fa:16:3e:f2:fb:d8', 'fixed_ips': [{'subnet_id': u'0e1dc8c6-0fc7-45de-98e1-acb953b56cda', 'ip_address': u'10.0.0.2'}], 'id': '1e3e2044-8b65-4770-aa05-7c499c19e210', 'security_groups': [u'78713c00-fa92-46e9-8d95-91151e65e195'], 'device_id': u'6cc7a5a8-0e7c-4380-b8f4-a334be5099be'}}
2014-04-14 13:16:22.076 13303 ERROR neutron.notifiers.nova [req-9b6ac828-6971-4772-bb6f-999e382444b3 None] Failed to notify nova on events: [{'status': 'completed', 'tag': u'1e3e2044-8b65-4770-aa05-7c499c19e210', 'name': 'network-vif-plugged', 'server_uuid': u'6cc7a5a8-0e7c-4380-b8f4-a334be5099be'}]


Version-Release number of selected component (if applicable):
openstack-packstack-puppet-2014.1.1-0.8.dev1045.fc21.noarch
openstack-packstack-2014.1.1-0.8.dev1045.fc21.noarch




Additional info:
Example working config /etc/neutron/neutron.conf part:
[DEFAULT]
nova_admin_auth_url = http://192.168.1.16:35357/v2.0
nova_admin_tenant_id = c45fc27844e24e40b96a5acec10a0a7b
nova_admin_password = secret
nova_admin_username = nova
nova_url = http://192.168.1.16:8774/v2
notify_nova_on_port_data_changes = True
notify_nova_on_port_status_change = True

Example not working config part:
# ======== neutron nova interactions ==========
# Send notification to nova when port status is active.
# notify_nova_on_port_status_changes = True

# Send notifications to nova when port data (fixed_ips/floatingips) change
# so nova can update it's cache.
# notify_nova_on_port_data_changes = True

# URL for connection to nova (Only supports one nova region currently).
# nova_url = http://127.0.0.1:8774

# Name of nova region to use. Useful if keystone manages more than one region
# nova_region_name =

# Username for connection to nova in admin context
# nova_admin_username =

# The uuid of the admin nova tenant
# nova_admin_tenant_id =

# Password for connection to nova in admin context.
# nova_admin_password =

# Authorization URL for connection to nova in admin context.
# nova_admin_auth_url =

# Number of seconds between sending events to nova if there are any events to send
# send_events_interval = 2

# ======== end of neutron nova interactions ==========


Note You need to log in before you can comment on or make changes to this bug.