python-novaclient: Allow modifying 'fixed_ips' in nova quota-update to avoid 'exception.FixedIpLimitExceeded'. Environment: ------------ python-nova-2012.2.3-7.el6ost.noarch python-novaclient-2.10.0-7.el6ost.noarch openstack-nova-common-2012.2.3-7.el6ost.noarch openstack-nova-scheduler-2012.2.3-7.el6ost.noarch openstack-dashboard-2012.2.3-8.el6ost.noarch openstack-nova-api-2012.2.3-7.el6ost.noarch Description: ------------- Currently when attempting to boot more than 10 instances, the action fails on exception.FixedIpLimitExceeded, this quota value has been added as part of the fix for #919648. the value 'fixed_ips' can be changed by running API, but it should be supported and documented as part of python-novaclient. the API support such an action: ------------------------------- curl http://10.35.160.11:8774/v2/2b409314aff74e2aa014209afa07aa3c/os-quota-sets/2b409314aff74e2aa014209afa07aa3c -X PUT -H "X-Auth-Project-Id: admin" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: 65ee60da861f4810afa24b3aadf55b0a" -d '{"quota_set": {"fixed_ips": 100, "tenant_id": "2b409314aff74e2aa014209afa07aa3c"}}' The exception compute.log: -------------------------- 2013-03-18 17:23:38 ERROR nova.compute.manager [req-73d62854-9377-44fc-a58a-17f6e36e34ee 00c37f45630747b28d60645d73ed1988 2b409314aff74e2aa014209afa07aa3c] [instance: b67 2b630-5dc0-47f8-9fbf-8dc3e248f03f] Instance failed network setup 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] Traceback (most recent call last): 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", l ine 721, in _allocate_network 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] requested_networks=requested_networks) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 44, in wrapper 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] res = f(self, context, *args, **kwargs) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 249, in allocate_for_instance 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 'args': args}) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/_ _init__.py", line 108, in call 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] return _get_impl().call(cfg.CONF, context, topic, msg, timeout) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/i mpl_qpid.py", line 554, in call 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] rpc_amqp.get_connection_pool(conf, Connection)) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 369, in call 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] rv = list(rv) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 337, in __iter__ 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] raise result 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] FixedIpLimitExceeded_Remote: Maximum number of fixed ips exceeded 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] Traceback (most recent call last): 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/d ispatcher.py", line 145, in dispatch 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] return getattr(proxyobj, method)(ctxt, **kwargs) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 266, in wrapped 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] return func(self, context, *args, **kwargs) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 329, in allocate_for_instance 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] **kwargs) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 266, in wrapped 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] return func(self, context, *args, **kwargs) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 1028, in allocate_for_instance 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] requested_networks=requested_networks) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 213, in _allocate_fixed_ips 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] vpn=vpn, address=address) 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 1305, in allocate_fixed_ip 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] raise exception.FixedIpLimitExceeded() 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] FixedIpLimitExceeded: Maximum number of fixed ips exceeded 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 31361 TRACE nova.compute.manager [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] 2013-03-18 17:23:38 INFO nova.compute.resource_tracker [req-73d62854-9377-44fc-a58a-17f6e36e34ee 00c37f45630747b28d60645d73ed1988 2b409314aff74e2aa014209afa07aa3c] Aborting claim: [Claim b672b630-5dc0-47f8-9fbf-8dc3e248f03f: 512 MB memory, 0 GB disk, 1 VCPUS] 2013-03-18 17:23:39 ERROR nova.compute.manager [req-73d62854-9377-44fc-a58a-17f6e36e34ee 00c37f45630747b28d60645d73ed1988 2b409314aff74e2aa014209afa07aa3c] [instance: b672b630-5dc0-47f8-9fbf-8dc3e248f03f] Build error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 498, in _run_instance\n requested_networks)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 721, in _allocate_network\n requested_networks=requested_networks)\n', ' File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 44, in wrapper\n res = f(self, context, *args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/nova/network/api.py", line 249, in allocate_for_instance\n \'args\': args})\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/__init__.py", line 108, in call\n return _get_impl().call(cfg.CONF, context, topic, msg, timeout)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/impl_qpid.py", line 554, in call\n rpc_amqp.get_connection_pool(conf, Connection))\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 369, in call\n rv = list(rv)\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 337, in __iter__\n raise result\n', 'FixedIpLimitExceeded_Remote: Maximum number of fixed ips exceeded\nTraceback (most recent call last):\n\n File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py", line 276, in _process_data\n rval = self.proxy.dispatch(ctxt, version, method, **args)\n\n File "/usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py", line 145, in dispatch\n return getattr(proxyobj, method)(ctxt, **kwargs)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 266, in wrapped\n return func(self, context, *args, **kwargs)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 329, in allocate_for_instance\n **kwargs)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 266, in wrapped\n return func(self, context, *args, **kwargs)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 1028, in allocate_for_instance\n requested_networks=requested_networks)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 213, in _allocate_fixed_ips\n vpn=vpn, address=address)\n\n File "/usr/lib/python2.6/site-packages/nova/network/manager.py", line 1305, in allocate_fixed_ip\n raise exception.FixedIpLimitExceeded()\n\nFixedIpLimitExceeded: Maximum number of fixed ips exceeded\n\n']
upstream fix is available at: https://review.openstack.org/#/c/24553/
We need to test we can view/modify/ fixed-ip via CLI/Horizon.
From ClI: ---------- * nova help quota-update: --fixed-ips <fixed-ips> New value for the "fixed-ips" quota * nova quota-show --tenant 1fccb1ae34e8423eba4940a07ca87c0a +-----------------------------+---------+ | Property | Value | +-----------------------------+---------+ | metadata_items | 128 | | injected_file_content_bytes | 10240 | | injected_files | 5 | | gigabytes | 5120000 | | ram | 5120000 | | floating_ips | 10 | | key_pairs | 100 | | instances | 100 | | security_group_rules | 20 | | volumes | 10 | | cores | 200 | | fixed_ips | 10 | | injected_file_path_bytes | 255 | | security_groups | 10 | +-----------------------------+---------+ * Before nova quota-update - when attempted to boot more than 10 instances (fixed-ips default = 10) --> Instances that been created over the quota limit raised exception.FixedIpLimitExceeded then those instances switched to status 'Error'. Note: ----- this 'FixedIpLimitExceeded' exception can be seen in compute.log and not like other quota error messages, that display message : "ERROR: Quota exceeded for instances: Requested 1, but already used 36 of 10 instances" * After nova quota-update --fixed-ips - Instances boot successfully.
From Horizon: ------------- * In Default Quotas page <controller>/dashboard/syspanel/quotas/ --> the value 'Fixed Ips' can be seen. * In projects --> modify quota --> /dashboard/syspanel/projects/ --> It seems that the value for fixed_ips is missing.
(In reply to comment #7) > From Horizon: > ------------- > > * In Default Quotas page <controller>/dashboard/syspanel/quotas/ --> the > value 'Fixed Ips' can be seen. > > * In projects --> modify quota --> /dashboard/syspanel/projects/ --> It > seems that the value for fixed_ips is missing. Opened #927182.
Verified with: python-novaclient-2.10.0-8.el6ost.noarch. * After nova quota-update --fixed-ips - Instances boot successfully.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2013-0706.html