1195067 – nova boot vm with '--nic net-id=xxxx, v4-fixed-ip=xxx' failed

Bug 1195067 - nova boot vm with '--nic net-id=xxxx, v4-fixed-ip=xxx' failed

Summary: nova boot vm with '--nic net-id=xxxx, v4-fixed-ip=xxx' failed

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-nova
Sub Component:
Version:	6.0 (Juno)
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	z3
Target Release:	6.0 (Juno)
Assignee:	Artom Lifshitz
QA Contact:	nlevinki
Docs Contact:
URL:
Whiteboard:
Duplicates (2):	1195066 1207531 (view as bug list)
Depends On:	1210455
Blocks:	1155653
TreeView+	depends on / blocked

Reported:	2015-02-22 17:39 UTC by John Fulton
Modified:	2019-09-09 16:16 UTC (History)
CC List:	16 users (show)
Fixed In Version:	openstack-nova-2014.2.3-3.el7ost
Doc Type:	Bug Fix
Doc Text:	The v4-fixed-ip option to the 'nova boot' command previously resulted in an instance booting to the ERROR state. This was caused by the IP address being represented as an IPAddress object instead of a string. With this update, the IP address is now represented as a string to allow instances to boot with the v4-fixed-ip option.
Clone Of:
Environment:
Last Closed:	2015-05-05 13:30:11 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Launchpad	1408529	None	None	None	Never
OpenStack gerrit	149905	None	None	None	Never
OpenStack gerrit	158435	None	None	None	Never
Red Hat Knowledge Base (Solution)	1315963	None	None	None	Never
Red Hat Product Errata	RHBA-2015:0931	normal	SHIPPED_LIVE	openstack-nova bug fix advisory	2015-05-05 17:29:39 UTC

Description John Fulton 2015-02-22 17:39:27 UTC

I. DESCRIPTION OF PROBLEM:

Customer reported: "‘v4-fixed-ip’ option on the nova command when deploying does not work with Juno"

My testing confirms this is OSP6.  

* The nova boot command with the v4-fixed-ip option looks fine when executed [1] but results in an instance in status error [2]. 
* The same command on the same OSP6 system with v4-fixed-ip removed results in a working instance. 
* The same command with v4-fixed-ip present on OSP5 results in a working instance. 

This appears to be tied to the following upstream bug (also sited by customer).  

 https://bugs.launchpad.net/nova/+bug/1408529
 https://review.openstack.org/#/c/149905/

As per the above report the following workaround works on OSP6: 

$ neutron port-create --fixed-ip subnet_id=SUBNET_ID,ip_address=IP_ADDRESS NET_ID
$ nova boot --image IMAGE --flavor FLAVOR --nic port-id=PORT_ID VM_NAME

I have updated our knowledge base article for OSP6 to suggest the above workaround in the meantime:  

 https://access.redhat.com/solutions/1315963

The full nova-compute log is below [4]. Truncating it we see:   

File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 65, in sanitizer
    return six.text_type(obj, 'utf8')
TypeError: coercing to Unicode: need string or buffer, IPAddress found

As per the change log: 
"""
If ip address is provided when running nova boot, nova compute
will invoke neutron client to create a port. However, the ip
address parameter is an IPAddress object so neutron client will
fail to send the request to neutron server. Transform IPAddress
object to string to address this issue.
"""

II. VERSION-RELEASE NUMBER OF SELECTED COMPONENT (IF APPLICABLE):
* python-novaclient-2.20.0-1.el7ost.noarch

III. HOW REPRODUCIBLE: 
* deterministic

IV. STEPS TO REPRODUCE:
* Follow https://access.redhat.com/solutions/1315963 on OSP6 (works on OSP5)

V. ACTUAL RESULTS:
* The nova boot command with the v4-fixed-ip option looks fine when executed [1] but results in an instance in status error [2]. 

VI. EXPECTED RESULTS:
* The nova boot command with the v4-fixed-ip option results in a working instance as in OSP5

VII. ADDITIONAL INFO:
* Footnotes sited above

[1] Booting an instance using the v4-fixed-ip option: 

[root@osp6p ~(keystone_admin)]# nova boot --nic net-id=acd9ab2b-6d8c-40df-86f9-578a59c6ab54,v4-fixed-ip=10.0.1.42 --key-name osp5-x230-keys --image rhel7 rhel7-test --flavor m1.small 
+--------------------------------------+----------------------------------------------+
| Property                             | Value                                        |
+--------------------------------------+----------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                       |
| OS-EXT-AZ:availability_zone          | nova                                         |
| OS-EXT-SRV-ATTR:host                 | -                                            |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                            |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000007                            |
| OS-EXT-STS:power_state               | 0                                            |
| OS-EXT-STS:task_state                | scheduling                                   |
| OS-EXT-STS:vm_state                  | building                                     |
| OS-SRV-USG:launched_at               | -                                            |
| OS-SRV-USG:terminated_at             | -                                            |
| accessIPv4                           |                                              |
| accessIPv6                           |                                              |
| adminPass                            | g6PT4pFzqkPY                                 |
| config_drive                         |                                              |
| created                              | 2015-02-21T22:41:20Z                         |
| flavor                               | m1.small (2)                                 |
| hostId                               |                                              |
| id                                   | 125c74eb-43bd-4095-a5ea-51b0aece1faa         |
| image                                | rhel7 (a6749630-da40-483a-a6dc-bba558ea2d6f) |
| key_name                             | osp5-x230-keys                               |
| metadata                             | {}                                           |
| name                                 | rhel7-test                                   |
| os-extended-volumes:volumes_attached | []                                           |
| progress                             | 0                                            |
| security_groups                      | default                                      |
| status                               | BUILD                                        |
| tenant_id                            | c9a2ccf6af8e499b8d8095917bd490ca             |
| updated                              | 2015-02-21T22:41:21Z                         |
| user_id                              | 7a67b107268a48849114be8ceb38a731             |
+--------------------------------------+----------------------------------------------+
[root@osp6p ~(keystone_admin)]# 

[2] Status of instance after using the v4-fixed-ip option

[root@osp6p ~(keystone_admin)]# nova list
+--------------------------------------+------------+--------+------------+-------------+----------+
| ID                                   | Name       | Status | Task State | Power State | Networks |
+--------------------------------------+------------+--------+------------+-------------+----------+
| 125c74eb-43bd-4095-a5ea-51b0aece1faa | rhel7-test | ERROR  | -          | NOSTATE     |          |
+--------------------------------------+------------+--------+------------+-------------+----------+
[root@osp6p ~(keystone_admin)]# 

[3] Workaround

[root@osp6p ~(keystone_admin)]# neutron port-create --fixed-ip subnet_id=8a1dce4f-be08-461d-93fc-06398d417c5d,ip_address=10.0.1.42 acd9ab2b-6d8c-40df-86f9-578a59c6ab54 
Created a new port:
+-----------------------+----------------------------------------------------------------------------------+
| Field                 | Value                                                                            |
+-----------------------+----------------------------------------------------------------------------------+
| admin_state_up        | True                                                                             |
| allowed_address_pairs |                                                                                  |
| binding:host_id       |                                                                                  |
| binding:profile       | {}                                                                               |
| binding:vif_details   | {}                                                                               |
| binding:vif_type      | unbound                                                                          |
| binding:vnic_type     | normal                                                                           |
| device_id             |                                                                                  |
| device_owner          |                                                                                  |
| fixed_ips             | {"subnet_id": "8a1dce4f-be08-461d-93fc-06398d417c5d", "ip_address": "10.0.1.42"} |
| id                    | 5b6b29a0-adc9-4d93-96f1-7259b4ac2499                                             |
| mac_address           | fa:16:3e:e8:d3:95                                                                |
| name                  |                                                                                  |
| network_id            | acd9ab2b-6d8c-40df-86f9-578a59c6ab54                                             |
| security_groups       | c88e8ccf-b44c-49fc-9591-6d8fbc33a086                                             |
| status                | DOWN                                                                             |
| tenant_id             | c9a2ccf6af8e499b8d8095917bd490ca                                                 |
+-----------------------+----------------------------------------------------------------------------------+
[root@osp6p ~(keystone_admin)]# 

[root@osp6p ~(keystone_admin)]# nova boot --nic port-id=5b6b29a0-adc9-4d93-96f1-7259b4ac2499 --key-name osp5-xhel7-test --flavor m1.small 
+--------------------------------------+----------------------------------------------+
| Property                             | Value                                        |
+--------------------------------------+----------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                       |
| OS-EXT-AZ:availability_zone          | nova                                         |
| OS-EXT-SRV-ATTR:host                 | -                                            |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                            |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000008                            |
| OS-EXT-STS:power_state               | 0                                            |
| OS-EXT-STS:task_state                | scheduling                                   |
| OS-EXT-STS:vm_state                  | building                                     |
| OS-SRV-USG:launched_at               | -                                            |
| OS-SRV-USG:terminated_at             | -                                            |
| accessIPv4                           |                                              |
| accessIPv6                           |                                              |
| adminPass                            | 7ASmdfVpjQtM                                 |
| config_drive                         |                                              |
| created                              | 2015-02-21T23:22:37Z                         |
| flavor                               | m1.small (2)                                 |
| hostId                               |                                              |
| id                                   | c65d3a3e-ab39-42dd-91c8-553c01c519d0         |
| image                                | rhel7 (a6749630-da40-483a-a6dc-bba558ea2d6f) |
| key_name                             | osp5-x230-keys                               |
| metadata                             | {}                                           |
| name                                 | rhel7-test                                   |
| os-extended-volumes:volumes_attached | []                                           |
| progress                             | 0                                            |
| security_groups                      | default                                      |
| status                               | BUILD                                        |
| tenant_id                            | c9a2ccf6af8e499b8d8095917bd490ca             |
| updated                              | 2015-02-21T23:22:37Z                         |
| user_id                              | 7a67b107268a48849114be8ceb38a731             |
+--------------------------------------+----------------------------------------------+
[root@osp6p ~(keystone_admin)]# 

[4] nova-compute.log

2015-02-21 17:41:22.707 3779 ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager Traceback (most recent call last):
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1643, in _allocate_network_async
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     dhcp_options=dhcp_options)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 443, in allocate_for_instance
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     self._delete_ports(neutron, instance, created_port_ids)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 423, in allocate_for_instance
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     security_group_ids, available_macs, dhcp_opts)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 214, in _create_port
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     port_id = port_client.create_port(port_req_body)['port']['id']
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     ret = obj(*args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 98, in with_params
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     ret = self.function(instance, *args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 322, in create_port
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return self.post(self.ports_path, body=body)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     ret = obj(*args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1325, in post
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     headers=headers, params=params)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     ret = obj(*args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1236, in do_request
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     body = self.serialize(body)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/__init__.py", line 84, in wrapper
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     ret = obj(*args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 1266, in serialize
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     self.get_attr_metadata()).serialize(data, self.content_type())
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 390, in serialize
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return self._get_serialize_handler(content_type).serialize(data)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 54, in serialize
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return self.dispatch(data, action=action)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 44, in dispatch
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return action_method(*args, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 66, in default
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return jsonutils.dumps(data, default=sanitizer)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/openstack/common/jsonutils.py", line 168, in dumps
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return json.dumps(value, default=default, **kwargs)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     sort_keys=sort_keys, **kw).encode(obj)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     chunks = self.iterencode(o, _one_shot=True)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return _iterencode(o, 0)
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager   File "/usr/lib/python2.7/site-packages/neutronclient/common/serializer.py", line 65, in sanitizer
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager     return six.text_type(obj, 'utf8')
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager TypeError: coercing to Unicode: need string or buffer, IPAddress found
2015-02-21 17:41:22.707 3779 TRACE nova.compute.manager

Comment 5 Artom Lifshitz 2015-02-23 20:09:35 UTC

*** Bug 1195066 has been marked as a duplicate of this bug. ***

Comment 6 Eoghan Glynn 2015-04-04 20:49:50 UTC

*** Bug 1207531 has been marked as a duplicate of this bug. ***

Comment 9 nlevinki 2015-04-26 13:43:16 UTC

nova boot --nic net-id=bef402af-51b0-4434-aad6-c8f71fde5644,v4-fixed-ip=172.24.4.228 --image cirros --flavor 1 test
+--------------------------------------+-----------------------------------------------+
| Property                             | Value                                         |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                        |
| OS-EXT-AZ:availability_zone          |                                               |
| OS-EXT-SRV-ATTR:host                 | -                                             |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                             |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000004                             |
| OS-EXT-STS:power_state               | 0                                             |
| OS-EXT-STS:task_state                | scheduling                                    |
| OS-EXT-STS:vm_state                  | building                                      |
| OS-SRV-USG:launched_at               | -                                             |
| OS-SRV-USG:terminated_at             | -                                             |
| accessIPv4                           |                                               |
| accessIPv6                           |                                               |
| adminPass                            | RvAZpqyu5AMh                                  |
| config_drive                         |                                               |
| created                              | 2015-04-26T13:42:12Z                          |
| flavor                               | m1.tiny (1)                                   |
| hostId                               |                                               |
| id                                   | 83d67ac6-a5ed-416c-bc94-d919fabe5e8a          |
| image                                | cirros (fc4450fa-82bd-4d56-8087-340103dfe5af) |
| key_name                             | -                                             |
| metadata                             | {}                                            |
| name                                 | test                                          |
| os-extended-volumes:volumes_attached | []                                            |
| progress                             | 0                                             |
| security_groups                      | default                                       |
| status                               | BUILD                                         |
| tenant_id                            | 8d08c9e81401414291a04d978fa355c8              |
| updated                              | 2015-04-26T13:42:12Z                          |
| user_id                              | 04fe018bd8bb428c8edb732ae3fdd704              |
+--------------------------------------+-----------------------------------------------+
[root@dhcp-2-211 ~(keystone_admin)]# nova list
+--------------------------------------+------+--------+------------+-------------+---------------------+
| ID                                   | Name | Status | Task State | Power State | Networks            |
+--------------------------------------+------+--------+------------+-------------+---------------------+
| 83d67ac6-a5ed-416c-bc94-d919fabe5e8a | test | ACTIVE | -          | Running     | public=172.24.4.228 |
+--------------------------------------+------+--------+------------+-------------+---------------------+

Comment 11 errata-xmlrpc 2015-05-05 13:30:11 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0931.html

Note You need to log in before you can comment on or make changes to this bug.