Description of problem: As described in [1], if we disable port-security on a network, then nova refuses to boot any more instances (500 no valid hosts), probably because the 'default' security groups was applied at boot time (see [2] for the launchpad comments and proposed patches) It may be that we're missing the patches of [2] in OSP8 Version-Release number of selected component (if applicable): OSP 8.0 How reproducible: Follow [1], section "Disable port security per network", it always fail (always 500 error), as expected by the blog author. Steps to Reproduce: 1. neutron net-create $NETNAME --shared --router:external --provider:physical_network datacentre --provider:network_type vlan --port-security-enabled=False 2. neutron subnet-create --name subn-$NETNAME --enable_dhcp=False $NETNAME 192.168.1.0/24 3. openstack server create --image Fedora24LiveCD --flavor m1.small --nic net-id=$ISP LaptopVM NOTE: if I do the neutron net-create without adding the port-security-enabled, everything works fine. Be aware that I'm not using Neutron DHCP on purpose, this is not the problem (it's a NFV demo so we're booting networking servers) Actual results: [admin@vcpe_lower_layer#overcloud: ~]$ openstack server create --image Fedora24LiveCD --flavor m1.small --nic net-id=$ISP LaptopVM +--------------------------------------+-------------------------------------------------------+ | Field | Value | +--------------------------------------+-------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-SRV-ATTR:host | None | | OS-EXT-SRV-ATTR:hypervisor_hostname | None | | OS-EXT-SRV-ATTR:instance_name | instance-00000129 | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | building | | OS-SRV-USG:launched_at | None | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | | | adminPass | tcBP6GfnowAj | | config_drive | | | created | 2016-09-29T00:06:43Z | | flavor | m1.small (2) | | hostId | | | id | 452aecc7-a829-4ae6-9723-5e6fee6d69d7 | | image | Fedora24LiveCD (bddf5882-3fa2-44ef-b9fc-c57eed9209ba) | | key_name | None | | name | LaptopVM | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | project_id | 86fbb467a88d4a388536be4e280c79f2 | | properties | | | security_groups | [{u'name': u'default'}] | | status | BUILD | | updated | 2016-09-29T00:06:44Z | | user_id | de1f9e2ff72a48268e493c1e48cafe7c | +--------------------------------------+-------------------------------------------------------+ [admin@vcpe_lower_layer#overcloud: ~]$ nova show 452aecc7-a829-4ae6-9723-5e6fee6d69d7 +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-SRV-ATTR:host | - | | OS-EXT-SRV-ATTR:hostname | laptopvm | | OS-EXT-SRV-ATTR:hypervisor_hostname | - | | OS-EXT-SRV-ATTR:instance_name | instance-00000129 | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-oigiyiw4 | | OS-EXT-SRV-ATTR:root_device_name | /dev/hda | | OS-EXT-SRV-ATTR:user_data | - | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | - | | OS-EXT-STS:vm_state | error | | OS-SRV-USG:launched_at | - | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | config_drive | | | created | 2016-09-29T00:06:43Z | | fault | {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": " File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 739, in build_instances | | | request_spec, filter_properties) | | | File \"/usr/lib/python2.7/site-packages/nova/scheduler/utils.py\", line 343, in wrapped | | | return func(*args, **kwargs) | | | File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 52, in select_destinations | | | context, request_spec, filter_properties) | | | File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/__init__.py\", line 37, in __run_method | | | return getattr(self.instance, __name)(*args, **kwargs) | | | File \"/usr/lib/python2.7/site-packages/nova/scheduler/client/query.py\", line 34, in select_destinations | | | context, request_spec, filter_properties) | | | File \"/usr/lib/python2.7/site-packages/nova/scheduler/rpcapi.py\", line 120, in select_destinations | | | request_spec=request_spec, filter_properties=filter_properties) | | | File \"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/client.py\", line 158, in call | | | retry=self.retry) | | | File \"/usr/lib/python2.7/site-packages/oslo_messaging/transport.py\", line 90, in _send | | | timeout=timeout, retry=retry) | | | File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 431, in send | | | retry=retry) | | | File \"/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py\", line 422, in _send | | | raise result | | | ", "created": "2016-09-29T00:07:26Z"} | | flavor | m1.small (2) | | hostId | | | id | 452aecc7-a829-4ae6-9723-5e6fee6d69d7 | | image | Fedora24LiveCD (bddf5882-3fa2-44ef-b9fc-c57eed9209ba) | | key_name | - | | locked | False | | metadata | {} | | name | LaptopVM | | os-extended-volumes:volumes_attached | [] | | status | ERROR | | tenant_id | 86fbb467a88d4a388536be4e280c79f2 | | updated | 2016-09-29T00:07:26Z | | user_id | de1f9e2ff72a48268e493c1e48cafe7c | +--------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Expected results: Nova should be able to boot (and find a valid host) a VM without port-security. See [2] for a detailed report of abandoned bugs and backports. It may be that we're missing the patches of [2] in OSP8 Additional info: [1] http://kimizhang.com/neutron-ml2-port-security/ [2] https://bugs.launchpad.net/nova/+bug/1175464
[root@controller-0 nova]# grep 452aecc7-a829-4ae6-9723-5e6fee6d69d7 * nova-conductor.log:2016-09-29 00:07:26.723 14237 ERROR nova.scheduler.utils [req-8ad11077-f7ac-4c98-a2c2-8bd3f2d1191e de1f9e2ff72a48268e493c1e48cafe7c 86fbb467a88d4a388536be4e280c79f2 - - -] [instance: 452aecc7-a829-4ae6-9723-5e6fee6d69d7] Error from last host: compute-1.localdomain (node compute-1.localdomain): [u'Traceback (most recent call last):\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in _do_build_and_run_instance\n filter_properties)\n', u' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2058, in _build_and_run_instance\n instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 452aecc7-a829-4ae6-9723-5e6fee6d69d7 was re-scheduled: Network requires port_security_enabled and subnet associated in order to apply security groups.\n'] nova-conductor.log:2016-09-29 00:07:26.748 14237 WARNING nova.scheduler.utils [req-8ad11077-f7ac-4c98-a2c2-8bd3f2d1191e de1f9e2ff72a48268e493c1e48cafe7c 86fbb467a88d4a388536be4e280c79f2 - - -] [instance: 452aecc7-a829-4ae6-9723-5e6fee6d69d7] Setting instance to ERROR state.
Hello Marcos, Can you indicate which version of the RHOS8 packages are used here ? Thanks, s.
Created attachment 1205947 [details] RPM list of the undercloud (rpm -qa)
Created attachment 1205948 [details] RPM list of the controller (rpm -qa)
Created attachment 1205949 [details] RPM list of the compute (rpm -qa)
These are the overcloud images ironic-python-agent-8.0-20160415.1.tar overcloud-full-8.0-20160415.1.tar RPM versions attached as TXT
Guys I've reviewed the 8.0.1 release notes (I am running 8.0) and I've found the ERRATA that fixes this issue https://access.redhat.com/errata/product/191/ver=8/rhel---7/x86_64/RHBA-2016:1226 https://bugzilla.redhat.com/show_bug.cgi?id=1331418 It seems I'll have to upgrade after all (we have manually changed settings in OSP post-install to integrate with Tacker) You can close this bug
*** This bug has been marked as a duplicate of bug 1331418 ***
Hum... So you are running Nova 12.0.2-5 and the change has been introduced for Nova 12.0.3-2 (bug 1331418) Can you try to update packages of your environment ?