Version: Grizzly on rhel6.4 with openstack-nova-network-2013.1.1-4.el6ost.noarch (puddle 2013-06-05.2) Two machines environment - one of them is compute node only, the second one is services + compute node. CONFIG_NOVA_NETWORK_HOST=10.35.160.29 CONFIG_NOVA_NETWORK_PUBIF=eth3 CONFIG_NOVA_NETWORK_PRIVIF=eth3 CONFIG_NOVA_NETWORK_FIXEDRANGE=172.16.0.0/24 CONFIG_NOVA_NETWORK_FLOATRANGE=10.35.164.0/24 CONFIG_NOVA_NETWORK_DEFAULTFLOATINGPOOL=nova CONFIG_NOVA_NETWORK_AUTOASSIGNFLOATINGIP=y How Reproducible: 100% (2/2) Description: I've changed the following nova.conf configuration and restarted all running nova services: openstack-config --set /etc/nova/nova.conf DEFAULT multi_host True openstack-config --set /etc/nova/nova.conf DEFAULT send_arp_for_ha True I've created 2 instances and managed to ping to their floating ips from an outside-world machine. I've added a multi_host network and created a new instance with nic on the multi_host network, I've noticed that all instances have not responded to ping anymore and lost their dhcp, the instance on the multi-host network also didn't have dhcp. I've added ip and default gw to all instances manually and I've managed to ping them again from an outside-world machine. I've tried to clear all iptables rules on both machines and request dhcp using dhclient - it didn't work, they didn't get an ip. # nova list +--------------------------------------+---------------------+--------+-------------------------------------+ | ID | Name | Status | Networks | +--------------------------------------+---------------------+--------+-------------------------------------+ | 2676ee0a-d409-4556-8ca4-3ba36efa3c75 | instance_multi_host | ACTIVE | multi_host=172.16.1.2, 10.35.164.3 | | 62379fb2-681a-4672-87fe-848caaf5fa7d | rhel64_1 | ACTIVE | novanetwork=172.16.0.2, 10.35.164.1 | | 4f93082e-4f10-4bff-a729-ec0f643dd1fb | rhel64_2 | ACTIVE | novanetwork=172.16.0.3, 10.35.164.2 | +--------------------------------------+---------------------+--------+-------------------------------------+ # nova network-list +--------------------------------------+-------------+---------------+ | ID | Label | Cidr | +--------------------------------------+-------------+---------------+ | 0c27e574-d690-40a3-a132-41baaf04e13d | novanetwork | 172.16.0.0/24 | | 490d9c8d-68e4-4d29-8a76-74e5df1d22bc | multi_host | 172.16.1.0/24 | +--------------------------------------+-------------+---------------+ # nova network-show 490d9c8d-68e4-4d29-8a76-74e5df1d22bc +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | bridge | br100 | | vpn_public_port | None | | dhcp_start | 172.16.1.2 | | bridge_interface | eth3 | | updated_at | None | | id | 490d9c8d-68e4-4d29-8a76-74e5df1d22bc | | cidr_v6 | None | | deleted_at | None | | gateway | 172.16.1.1 | | rxtx_base | None | | label | multi_host | | priority | None | | project_id | None | | vpn_private_address | None | | deleted | 0 | | vlan | None | | broadcast | 172.16.1.255 | | netmask | 255.255.255.0 | | injected | False | | cidr | 172.16.1.0/24 | | vpn_public_address | None | | multi_host | True | | dns2 | None | | created_at | 2013-06-10T10:39:45.000000 | | host | None | | gateway_v6 | None | | netmask_v6 | None | | dns1 | 8.8.4.4 | +---------------------+--------------------------------------+ # nova network-show 0c27e574-d690-40a3-a132-41baaf04e13d +---------------------+--------------------------------------+ | Property | Value | +---------------------+--------------------------------------+ | bridge | br100 | | vpn_public_port | None | | dhcp_start | 172.16.0.2 | | bridge_interface | eth3 | | updated_at | 2013-06-10T10:13:48.000000 | | id | 0c27e574-d690-40a3-a132-41baaf04e13d | | cidr_v6 | None | | deleted_at | None | | gateway | 172.16.0.1 | | rxtx_base | None | | label | novanetwork | | priority | None | | project_id | None | | vpn_private_address | None | | deleted | 0 | | vlan | None | | broadcast | 172.16.0.255 | | netmask | 255.255.255.0 | | injected | False | | cidr | 172.16.0.0/24 | | vpn_public_address | None | | multi_host | False | | dns2 | None | | created_at | 2013-06-09T22:07:23.000000 | | host | puma10.scl.lab.tlv.redhat.com | | gateway_v6 | None | | netmask_v6 | None | | dns1 | 8.8.4.4 | +---------------------+--------------------------------------+ # nova boot --flavor 2 --image rhel64 --key-name oskey --nic net-id=490d9c8d-68e4-4d29-8a76-74e5df1d22bc instance_multi_host +-------------------------------------+--------------------------------------+ | Property | Value | +-------------------------------------+--------------------------------------+ | OS-EXT-STS:task_state | scheduling | | image | rhel64 | | OS-EXT-STS:vm_state | building | | OS-EXT-SRV-ATTR:instance_name | instance-00000003 | | flavor | m1.small | | id | 2676ee0a-d409-4556-8ca4-3ba36efa3c75 | | security_groups | [{u'name': u'default'}] | | user_id | e34e389219eb4435871af5b4d8c83803 | | OS-DCF:diskConfig | MANUAL | | accessIPv4 | | | accessIPv6 | | | progress | 0 | | OS-EXT-STS:power_state | 0 | | OS-EXT-AZ:availability_zone | nova | | config_drive | | | status | BUILD | | updated | 2013-06-10T10:41:18Z | | hostId | | | OS-EXT-SRV-ATTR:host | None | | key_name | oskey | | OS-EXT-SRV-ATTR:hypervisor_hostname | None | | name | instance_multi_host | | adminPass | 2qSmQtTz98cz | | tenant_id | baf098c2ce7f4d328658b9f9c0baf0a7 | | created | 2013-06-10T10:41:18Z | | metadata | {} | +-------------------------------------+--------------------------------------+ # ip a | grep -B 2 172 9: br100: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN link/ether 00:9c:02:b0:bf:94 brd ff:ff:ff:ff:ff:ff inet 172.16.1.3/24 brd 172.16.1.255 scope global br100 inet 172.16.0.1/24 brd 172.16.0.255 scope global br100 # ps -elf | grep dnsmasq 5 S nobody 14228 1 0 80 0 - 3221 poll_s 01:03 ? 00:00:01 /usr/sbin/dnsmasq --strict-order --local=// --domain-needed --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --bind-interfaces --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override --dhcp-hostsfile=/var/lib/libvirt/dnsmasq/default.hostsfile --addn-hosts=/var/lib/libvirt/dnsmasq/default.addnhosts 5 S nobody 17199 1 0 80 0 - 3221 poll_s 13:13 ? 00:00:00 /usr/sbin/dnsmasq --strict-order --bind-interfaces --conf-file= --pid-file=/var/lib/nova/networks/nova-br100.pid --listen-address=172.16.0.1 --except-interface=lo --dhcp-range=set:novanetwork,172.16.0.2,static,255.255.255.0,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro --domain=novalocal 1 S root 17200 17199 0 80 0 - 3221 pipe_w 13:13 ? 00:00:00 /usr/sbin/dnsmasq --strict-order --bind-interfaces --conf-file= --pid-file=/var/lib/nova/networks/nova-br100.pid --listen-address=172.16.0.1 --except-interface=lo --dhcp-range=set:novanetwork,172.16.0.2,static,255.255.255.0,120s --dhcp-lease-max=256 --dhcp-hostsfile=/var/lib/nova/networks/nova-br100.conf --dhcp-script=/usr/bin/nova-dhcpbridge --leasefile-ro --domain=novalocal 0 S root 21782 17036 0 80 0 - 25812 pipe_w 14:38 pts/0 00:00:00 grep dnsmasq # cat /var/lib/libvirt/dnsmasq/default.* # cat /var/lib/nova/networks/nova-br100.conf fa:16:3e:39:72:fd,instance-multi-host.novalocal,172.16.1.2 *** Note that the issue might be connected to another issue that I've experienced when adding a multi-host network, it's described in the following bug: rhbz#972668 - [nova-network] multi host network has created with a default gateway different from the one appears in "nova network-show"
Switching to multi-host on the fly is not supported. You need to setup multi-host from the start so that all the correct settings are in the DB.