Bug 1438082
| Summary: | cloud-init fails to configure network in mixed ipv4/ipv6 environment | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Lars Kellogg-Stedman <lars> | |
| Component: | cloud-init | Assignee: | Ryan McCabe <rmccabe> | |
| Status: | CLOSED ERRATA | QA Contact: | Vratislav Hutsky <vhutsky> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 7.4 | CC: | akaris, dmaley, fdinitto, huzhao, lars, lnatapov, mbracho, mvermaes, pgervase, rmccabe, ushkalim, yacao | |
| Target Milestone: | rc | Keywords: | Triaged | |
| Target Release: | --- | Flags: | lars:
needinfo-
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | cloud-init-0.7.9-8.el7 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1462022 (view as bug list) | Environment: | ||
| Last Closed: | 2017-08-01 23:23:42 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1462022 | |||
|
Description
Lars Kellogg-Stedman
2017-03-31 20:38:50 UTC
Information from the original reporter:
I see the following in the logs:
# grep ERROR /var/log/messages
ERROR : [/etc/sysconfig/network-scripts/ifup-aliases] error in ifcfg-eth0:1: didn't specify device or ipaddr
ERROR : [/etc/sysconfig/network-scripts/ifup-aliases] error in ifcfg-eth0:1: didn't specify device or ipaddr
ERROR : [/etc/sysconfig/network-scripts/ifup-aliases] error in ifcfg-eth0:1: didn't specify device or spader
The interfaces look like this:
$ /sbin/ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::f816:3eff:feb2:4bd4 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:b2:4b:d4 txqueuelen 1000 (Ethernet)
RX packets 88 bytes 17669 (17.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 81 bytes 13735 (13.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.252.197.143 netmask 255.255.255.224 broadcast 10.252.197.159
ether fa:16:3e:b2:4b:d4 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
If I "ifup” the eth0:1 interface, the eth0 interface is configured with the IPv6 address (not the subinterface).
# ifup eth0:1
# /sbin/ifconfig -a
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 2620:160:d2ff:1c07::b prefixlen 64 scopeid 0x0<global>
inet6 fe80::f816:3eff:feb2:4bd4 prefixlen 64 scopeid 0x20<link>
ether fa:16:3e:b2:4b:d4 txqueuelen 1000 (Ethernet)
RX packets 760 bytes 133925 (130.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 576 bytes 114201 (111.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
eth0:0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.252.197.143 netmask 255.255.255.224 broadcast 10.252.197.159
ether fa:16:3e:b2:4b:d4 txqueuelen 1000 (Ethernet)
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Here are the configuration files:
# cat ifcfg-eth0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=none
DEVICE=eth0
HWADDR=fa:16:3e:b2:4b:d4
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
# cat ifcfg-eth0:0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth0:0
GATEWAY=10.252.197.129
HWADDR=fa:16:3e:b2:4b:d4
IPADDR=10.252.197.143
NETMASK=255.255.255.224
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
# cat ifcfg-eth0:1
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth0:1
HWADDR=fa:16:3e:b2:4b:d4
IPV6ADDR=2620:160:d2ff:1c07::b
IPV6INIT=yes
IPV6_DEFAULTGW=2620:160:d2ff:1c07::1
NETMASK=64
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
removing https://bugs.launchpad.net/cloud-init/+bug/1677846 from this bug. Let's focus here on https://bugs.launchpad.net/cloud-init/+bug/1679817 I created https://bugzilla.redhat.com/1438082 to work on the ESXi config drive issue with DVS I created https://bugzilla.redhat.com/show_bug.cgi?id=1442783 to work on the ESXi config drive issue with DVS Hi,
++++++++++++++++++++++++++++++++++++++++++++++++++
Long story short:
messages-20170409:Apr 3 18:15:05 projectb-fwam-rhel72-cloud-init-0 /etc/sysconfig/network-scripts/ifup-aliases: error in ifcfg-eth0:1: didn't specify device or ipaddr is coming from the absence of IPv4 addresses.
The error message comes from here:
/etc/sysconfig/network-scripts/ifup-aliases
~~~
(...)
if [ -z "$DEVICE" -o -z "$IPADDR" ]; then
net_log $"error in $FILE: didn't specify device or ipaddr"
return 1
fi
(...)
~~~
Meaning that if DEVICE or IPADDR is not set, then this error message will be logged, and the script returns.
Note that in /etc/sysconfig/network-scripts/ifup-aliases, it calls
~~~
(...)
[[ "$IPV6INIT" != [nN0]* ]] && /etc/sysconfig/network-scripts/ifup-ipv6 ${DEVICE}
(...)
~~~
However, due to the fact that the script returns function new_interface (returns, not exits!) with the above error message ('didn't specify device or ipaddr'), it will never get here.
++++++++++++++++++++++++++++++++++++++++++++++++++
Looking into this in more detail:
The aliases script ifup-aliases is being called in
~~~
ifup-post: /etc/sysconfig/network-scripts/ifup-aliases ${DEVICE} ${CONFIG}
~~~
Now, looking at /etc/sysconfig/network-scripts/network-functions
~~~
(...)
[ -z "$REALDEVICE" ] && REALDEVICE=${DEVICE%%:*}
[ -z "$SYSCTLDEVICE" ] && SYSCTLDEVICE=${REALDEVICE/.//}
if [ "${DEVICE}" != "${REALDEVICE}" ]; then
ISALIAS=yes
else
ISALIAS=no
fi
(...)
~~~
Meaning that DEVICE != REALDEVICE (eth0:0 != eth0), so this is ISALIAS = yes
Now, looking at /etc/sysconfig/network-scripts/network-functions
~~~
(...)
if [ "$ISALIAS" = no ] ; then
/etc/sysconfig/network-scripts/ifup-aliases ${DEVICE} ${CONFIG}
fi
(...)
~~~
Meaning that if this is not an alias, then call ifup-aliases <INTERFACE> <CONFIG>, e.g. ifup-aliases eth0 <CONFIG>
ifup-aliases then processes any alias file for the ${DEVICE}
~~~
(...)
for FILE in ifcfg-${parent_device}:* ; do
is_ignored_file "$FILE" && continue
ini_env;
. ./$FILE;
[ -z "$DEVICE" ] && DEVICE=${FILE##ifcfg-}
[ "$ONPARENT" != "no" -a "$ONPARENT" != "NO" ] && new_interface;
unset DEVICE
done
(...)
~~~
new_interface is called with the alias name as DEVICE, e.g. eth0:0 and returns (fails) before it can call ifup-ipv6
~~~
function new_interface ()
{
(...)
if [ -z "$DEVICE" -o -z "$IPADDR" ]; then
net_log $"error in $FILE: didn't specify device or ipaddr"
return 1
fi
(...)
[[ "$IPV6INIT" != [nN0]* ]] && /etc/sysconfig/network-scripts/ifup-ipv6 ${DEVICE}
}
(...)
~~~
+++++++++++++++++++++++++++++++++++++++
Looking at IPv6, IPv6 is only configured by
~~~
[root@undercloud-4 network-scripts]# egrep '\-6' *
ifdown-routes: proto="-6"
ifup-eth: if /sbin/dhclient -6 -1 ${DHCPV6C_OPTIONS} ${DHCLIENTCONF} -lf ${LEASEFILE} -pf /var/run/dhclient6-${DEVICE}.pid -H ${DHCP_HOSTNAME:-${HOSTNAME%%.*}} ${DEVICE} ; then
ifup-ib: if /sbin/dhclient -6 -1 ${DHCPV6C_OPTIONS} ${DHCLIENTCONF} -lf ${LEASEFILE} -pf /var/run/dhclient6-${DEVICE}.pid -H ${DHCP_HOSTNAME:-${HOSTNAME%%.*}} ${DEVICE} ; then
ifup-ipv6: /sbin/ip -6 route add $line
ifup-ipv6: /sbin/ip -6 route del ::/96 dev tun6to4
ifup-ipv6: /sbin/ip -6 addr del "::$ipv4addrlocal/128" dev tun6to4
ifup-routes: proto="-6"
ifup-tunnel: proto=-6
network-functions-ipv6: local returntxt="$(/sbin/ip -6 route add $networkipv6 via $gatewayipv6 metric 1 2>&1)"
network-functions-ipv6: local returntxt="$(/sbin/ip -6 route add $networkipv6 dev $device metric 1 2>&1)"
network-functions-ipv6: local returntxt="$(/sbin/ip -6 route add $networkipv6 via $gatewayipv6 dev $device metric 1 2>&1)"
network-functions-ipv6: /sbin/ip -6 addr add $address dev $device
network-functions-ipv6: /sbin/ip -6 route flush dev $device scope global >/dev/null 2>&1
network-functions-ipv6: /sbin/ip -6 route flush dev $device scope site >/dev/null 2>&1
network-functions-ipv6: /sbin/ip -6 addr flush dev $device scope global >/dev/null 2>&1
network-functions-ipv6: /sbin/ip -6 addr flush dev $device scope site >/dev/null 2>&1
network-functions-ipv6: /sbin/ip -6 addr show dev $dev scope global permanent | awk '/\<inet6\>/ && $2 ~ /^2002:/ { print $2 }' | while read addr; do
network-functions-ipv6: /sbin/ip -6 addr del ${addr} dev ${dev}
network-functions-ipv6: /sbin/ip -6 route show dev $device | LC_ALL=C grep "^2002:" | while read ipv6net dummy; do
network-functions-ipv6: /sbin/ip -6 route del $ipv6net dev $device
network-functions-ipv6: /sbin/ip -6 route | LC_ALL=C grep "^unreachable 2002:.*/48 dev lo" | while read token net rest; do
network-functions-ipv6: while [ ${countdown} -gt 0 -a -n "$(ip -6 addr show dev ${device} scope global tentative)" ]; do
~~~
Also note that IPV6ADDR and IPV6ADDR_SECONDARIES is only processed by:
~~~
[root@undercloud-4 network-scripts]# grep IPV6ADDR *
ifup-ippp: if [[ "$IPV6INIT" != [nN0]* && ! -z "$IPV6ADDR" ]]; then
ifup-ipv6:# IPV6ADDR=<IPv6 address>[/<prefix length>]: specify primary static IPv6 address
ifup-ipv6:# IPV6ADDR_SECONDARIES="<IPv6 address>[/<prefix length>] ..." (optional)
ifup-ipv6:if [ -n "$IPV6ADDR" ]; then
ifup-ipv6: ipv6_add_addr_on_device $DEVICE $IPV6ADDR || exit 1
ifup-ipv6:if [ -n "$IPV6ADDR_SECONDARIES" ]; then
ifup-ipv6: for ipv6addr in $IPV6ADDR_SECONDARIES; do
ifup-isdn: if [[ "$IPV6INIT" != [nN0]* && ! -z "$IPV6ADDR" ]]; then
ifup-sit:# IPV6ADDR=<IPv6 address>[/<prefix length>]: (optional) local IPv6 address of a numbered tunnel
ifup-sit:# IPV6ADDR_SECONDARIES="<IPv6 address>[/<prefix length>] ..." (optional) additional local IPv6 addresses
ifup-sit:if [ -n "$IPV6ADDR" ]; then
ifup-sit: ipv6_add_addr_on_device $DEVICE $IPV6ADDR
ifup-sit:if [ -n "$IPV6ADDR_SECONDARIES" ]; then
ifup-sit: for ipv6addr in $IPV6ADDR_SECONDARIES; do
~~~
The primary IPV6ADDR is added in ifup-ipv6 via
~~~
# Setup IPv6 address on specified interface
if [ -n "$IPV6ADDR" ]; then
ipv6_add_addr_on_device $DEVICE $IPV6ADDR || exit 1
fi
~~~
The secondaries are added in ifup-ipv6 via
~~~
(...)
# Setup additional IPv6 addresses from list, if given
if [ -n "$IPV6ADDR_SECONDARIES" ]; then
for ipv6addr in $IPV6ADDR_SECONDARIES; do
ipv6_add_addr_on_device $DEVICE $ipv6addr
done
fi
(...)
~~~
Regards,
Andreas
I talked to one of our network support engineers, and he immediately mentioned the use of IPV6ADDR_SECONDARIES as well. According to him, sysconfig has had IPADDRn and IPV6ADDR_SECONDARIES for ages, so we can put the interfaces all on ethX - RHEL 5 was about the time alias interfaces became the old way. I am currently working on a reproducer for this:
===============================================
I upgraded cloud-init to the latest version that we have in the repositories (0.7.9-3) and it works like a charm now ...
i) Upgrade cloud-init in RHEL image:
~~~
sudo yumdownloader cloud-init pyserial python-jinja2 python-babel python-markupsafe pytz
for i in *.rpm;do virt-customize -a rhel.qcow2 --upload $i:/root/$i ; done
virt-customize -a rhel.qcow2 -v --run-command 'yum -y localinstall /root/*.rpm'
glance image-create --name rhel2 --file rhel.qcow2 --container-format bare --disk-format qcow2 --progress
~~~
ii) Configure config drive for network injection:
On the compute node, modify /etc/nova/nova.conf and then restart openstack-nova-compute.service
~~~
[DEFAULT]
injected_network_template = /usr/lib/python2.7/site-packages/nova/virt/interfaces.template
flat_injected=true
force_config_drive = true
config_drive_cdrom = True
debug=true
[libvirt]
inject_partition=-1
~~~
iii) Spawning instance with the new image:
~~~
NETID=88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 ; nova boot --nic net-id=$NETID --image rhel2 --flavor m1.small --key-name id_rsa --availability-zone nova:overcloud-compute-1.localdomain rhel2 ; sleep 10 ; nova floating-ip-associate rhel2 10.0.0.114
~~~
iv) Then, after spawning an instance with this image, verify network configuration and cloud-init log:
~~~
[root@rhel2 ~]# mount /dev/sr0 /mnt
mount: /dev/sr0 is write-protected, mounting read-only
[root@rhel2 ~]# cat /mnt/
ec2/ openstack/
[root@rhel2 ~]# cat /mnt/openstack/
2012-08-10/ 2013-04-04/ 2013-10-17/ 2015-10-15/ 2016-06-30/ 2016-10-06/ content/ latest/
[root@rhel2 ~]# cat /mnt/openstack/content/0000
# Injected by Nova on instance boot
#
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
hwaddress ether fa:16:3e:23:a4:17
address 192.168.0.2
netmask 255.255.255.0
broadcast 192.168.0.255
gateway 192.168.0.1
[root@rhel2 ~]# cat /mnt/openstack/latest/
meta_data.json network_data.json vendor_data2.json vendor_data.json
[root@rhel2 ~]# cat /mnt/openstack/latest/network_data.json
{"services": [], "networks": [{"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1", "type": "ipv4", "netmask": "255.255.255.0", "link": "tapeaab7dd8-8c", "routes": [{"netmask": "0.0.0.0", "network": "0.0.0.0", "gateway": "192.168.0.1"}], "ip_address": "192.168.0.2", "id": "network0"}], "links": [{"ethernet_mac_address": "fa:16:3e:23:a4:17", "mtu": 1446, "type": "ovs", "id": "tapeaab7dd8-8c", "vif_id": "eaab7dd8-8c10-4b8d-acab-0e032a6e4c18"}]}[root@rhel2 ~]# grep network_data.json /var/log/cloud-init* -R
/var/log/cloud-init.log:2017-04-19 01:36:17,798 - util.py[DEBUG]: Reading from /tmp/tmp5c01EG/openstack/2015-10-15/network_data.json (quiet=False)
/var/log/cloud-init.log:2017-04-19 01:36:17,798 - util.py[DEBUG]: Read 447 bytes from /tmp/tmp5c01EG/openstack/2015-10-15/network_data.json
[root@rhel2 ~]# less /var/log/cloud-init.log
[root@rhel2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth0
GATEWAY=192.168.0.1
HWADDR=fa:16:3e:23:a4:17
IPADDR=192.168.0.2
MTU=1446
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 ~]# ls /etc/sysconfig/network-scripts/ifcfg-*
/etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-lo
[root@rhel2 ~]# rpm -qa | grep cloud-init
cloud-init-0.7.9-3.el7.x86_64
~~~
======================================
Next step, I tried:
~~~
/etc/nova/nova.conf
[os_vif_linux_bridge]
use_ipv6 = true
~~~
I am still not getting the IPv6 data into network_data.json
~~~
[root@rhel2 network-scripts]# cat /mnt/openstack/latest/network_data.json
{"services": [], "networks": [{"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1", "type": "ipv4", "netmask": "255.255.255.0", "link": "tap4384c1bb-c2", "routes": [{"netmask": "0.0.0.0", "network": "0.0.0.0", "gateway": "192.168.0.1"}], "ip_address": "192.168.0.10", "id": "network0"}], "links": [{"ethernet_mac_address": "fa:16:3e:49:15:fe", "mtu": 1446, "type": "ovs", "id": "tap4384c1bb-c2", "vif_id": "4384c1bb-c29a-40dc-bd0b-576180de64d1"}]}[root@rhel2 network-scripts]#
~~~
=============================================
currently checking with the customer what he used to inject ipv6 data into network_data.json.
Turns out that there is a setting on the compute in /etc/nova/nova.conf for DEFAULT and os_vif_linux_bridge, and that I missed that the setting in DEFAULT was being overwritten by another use_ipv6=false later in the config ...:
[root@overcloud-compute-1 ~]# grep use_ipv6 /etc/nova/nova.conf -B1
config_drive_cdrom = True
use_ipv6 = true
--
[os_vif_linux_bridge]
use_ipv6 = true
Also meaning that the second setting (of which I though that it was important) is irrelevant.
Summary for this:
====================================================
I upgraded cloud-init to the latest version that we have in the repositories (0.7.9-3) and it works like a charm now ...
i) Upgrade cloud-init in RHEL image:
~~~
sudo yumdownloader cloud-init pyserial python-jinja2 python-babel python-markupsafe pytz
for i in *.rpm;do virt-customize -a rhel.qcow2 --upload $i:/root/$i ; done
virt-customize -a rhel.qcow2 -v --run-command 'yum -y localinstall /root/*.rpm'
glance image-create --name rhel2 --file rhel.qcow2 --container-format bare --disk-format qcow2 --progress
~~~
ii) Configure config drive for network injection:
On the compute node, modify /etc/nova/nova.conf and then restart openstack-nova-compute.service
~~~
[DEFAULT]
injected_network_template = /usr/lib/python2.7/site-packages/nova/virt/interfaces.template
flat_injected=true
force_config_drive = true
config_drive_cdrom = True
debug=true
use_ipv6=true
[libvirt]
inject_partition=-1
~~~
iii) Configure 3 networks with 5 subnets:
~~~
[stack@undercloud-1 ~]$ neutron net-list
+--------------------------------------+-----------+--------------------------------------------------------+
| id | name | subnets |
+--------------------------------------+-----------+--------------------------------------------------------+
| 7cf160c2-4423-4044-bdfb-23887da5c6e0 | provider1 | b373bceb-6770-4e6b-b53c-f63b979b67be 2000:10::/64 |
| | | 0da11b66-dbcf-445f-ae46-8347b0ebcbf1 10.0.0.0/24 |
| 88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 | private | f0306d30-b908-41db-bb99-75a77ce61ca1 192.168.0.0/24 |
| | | 63cc5607-a084-4f00-938f-be47f8f18362 2000:192:168::/64 |
| ae53dd16-27d9-40e0-b0b5-36e1140a7e0b | ipv6 | b84ff03d-2568-4099-aaef-f42c5c0be180 2000:123::/64 |
+--------------------------------------+-----------+--------------------------------------------------------+
[stack@undercloud-1 ~]$ neutron net-show 7cf160c2-4423-4044-bdfb-23887da5c6e0 ; neutron net-show 88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 ; neutron net-show ae53dd16-27d9-40e0-b0b5-36e1140a7e0b ; neutron subnet-show b373bceb-6770-4e6b-b53c-f63b979b67be ; neutron subnet-show 0da11b66-dbcf-445f-ae46-8347b0ebcbf1 ; neutron subnet-show f0306d30-b908-41db-bb99-75a77ce61ca1 ; neutron subnet-show 63cc5607-a084-4f00-938f-be47f8f18362 ; neutron subnet-show b84ff03d-2568-4099-aaef-f42c5c0be180
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2017-04-09T03:54:41Z |
| description | |
| id | 7cf160c2-4423-4044-bdfb-23887da5c6e0 |
| ipv4_address_scope | |
| ipv6_address_scope | |
| is_default | False |
| mtu | 1496 |
| name | provider1 |
| port_security_enabled | True |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| provider:network_type | vlan |
| provider:physical_network | datacentre |
| provider:segmentation_id | 905 |
| qos_policy_id | |
| revision_number | 8 |
| router:external | True |
| shared | True |
| status | ACTIVE |
| subnets | b373bceb-6770-4e6b-b53c-f63b979b67be |
| | 0da11b66-dbcf-445f-ae46-8347b0ebcbf1 |
| tags | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:45:23Z |
+---------------------------+--------------------------------------+
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2017-04-09T03:54:40Z |
| description | |
| id | 88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 |
| ipv4_address_scope | |
| ipv6_address_scope | |
| mtu | 1446 |
| name | private |
| port_security_enabled | True |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 44 |
| qos_policy_id | |
| revision_number | 27 |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | 63cc5607-a084-4f00-938f-be47f8f18362 |
| | f0306d30-b908-41db-bb99-75a77ce61ca1 |
| tags | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:28:56Z |
+---------------------------+--------------------------------------+
+---------------------------+--------------------------------------+
| Field | Value |
+---------------------------+--------------------------------------+
| admin_state_up | True |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2017-04-20T02:56:13Z |
| description | |
| id | ae53dd16-27d9-40e0-b0b5-36e1140a7e0b |
| ipv4_address_scope | |
| ipv6_address_scope | |
| mtu | 1446 |
| name | ipv6 |
| port_security_enabled | True |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| provider:network_type | vxlan |
| provider:physical_network | |
| provider:segmentation_id | 21 |
| qos_policy_id | |
| revision_number | 5 |
| router:external | False |
| shared | False |
| status | ACTIVE |
| subnets | b84ff03d-2568-4099-aaef-f42c5c0be180 |
| tags | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:56:36Z |
+---------------------------+--------------------------------------+
+-------------------+------------------------------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------------------------------+
| allocation_pools | {"start": "2000:10::251", "end": "2000:10::ffff:ffff:ffff:ffff"} |
| | {"start": "2000:10::1", "end": "2000:10::24f"} |
| cidr | 2000:10::/64 |
| created_at | 2017-04-20T02:29:01Z |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 2000:10::250 |
| host_routes | |
| id | b373bceb-6770-4e6b-b53c-f63b979b67be |
| ip_version | 6 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | provider1-ipv6-subnet |
| network_id | 7cf160c2-4423-4044-bdfb-23887da5c6e0 |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:29:01Z |
+-------------------+------------------------------------------------------------------+
+-------------------+----------------------------------------------+
| Field | Value |
+-------------------+----------------------------------------------+
| allocation_pools | {"start": "10.0.0.100", "end": "10.0.0.150"} |
| cidr | 10.0.0.0/24 |
| created_at | 2017-04-09T03:54:41Z |
| description | |
| dns_nameservers | 8.8.8.8 |
| enable_dhcp | False |
| gateway_ip | 10.0.0.250 |
| host_routes | |
| id | 0da11b66-dbcf-445f-ae46-8347b0ebcbf1 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | provider1-subnet |
| network_id | 7cf160c2-4423-4044-bdfb-23887da5c6e0 |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| revision_number | 3 |
| service_types | |
| subnetpool_id | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:45:23Z |
+-------------------+----------------------------------------------+
+-------------------+--------------------------------------------------+
| Field | Value |
+-------------------+--------------------------------------------------+
| allocation_pools | {"start": "192.168.0.2", "end": "192.168.0.254"} |
| cidr | 192.168.0.0/24 |
| created_at | 2017-04-09T03:54:45Z |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 192.168.0.1 |
| host_routes | |
| id | f0306d30-b908-41db-bb99-75a77ce61ca1 |
| ip_version | 4 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | private-subnet |
| network_id | 88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| revision_number | 3 |
| service_types | |
| subnetpool_id | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-19T01:57:58Z |
+-------------------+--------------------------------------------------+
+-------------------+---------------------------------------------------------------------------+
| Field | Value |
+-------------------+---------------------------------------------------------------------------+
| allocation_pools | {"start": "2000:192:168::2", "end": "2000:192:168:0:ffff:ffff:ffff:ffff"} |
| cidr | 2000:192:168::/64 |
| created_at | 2017-04-20T02:24:02Z |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 2000:192:168::1 |
| host_routes | |
| id | 63cc5607-a084-4f00-938f-be47f8f18362 |
| ip_version | 6 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | private-ipv6-subnet |
| network_id | 88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:24:02Z |
+-------------------+---------------------------------------------------------------------------+
+-------------------+------------------------------------------------------------------+
| Field | Value |
+-------------------+------------------------------------------------------------------+
| allocation_pools | {"start": "2000:123::1", "end": "2000:123::ffff:ffff:ffff:ffff"} |
| cidr | 2000:123::/64 |
| created_at | 2017-04-20T02:56:36Z |
| description | |
| dns_nameservers | |
| enable_dhcp | False |
| gateway_ip | 2000:10::250 |
| host_routes | |
| id | b84ff03d-2568-4099-aaef-f42c5c0be180 |
| ip_version | 6 |
| ipv6_address_mode | |
| ipv6_ra_mode | |
| name | ipv6-subnet |
| network_id | ae53dd16-27d9-40e0-b0b5-36e1140a7e0b |
| project_id | 49ebd4ddc3b242c1909939157ae77c55 |
| revision_number | 2 |
| service_types | |
| subnetpool_id | |
| tenant_id | 49ebd4ddc3b242c1909939157ae77c55 |
| updated_at | 2017-04-20T02:56:36Z |
+-------------------+------------------------------------------------------------------+
~~~
iv) Spawn an instance on all 3 networks
~~~
nova boot --nic net-id=88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1 --nic net-id=7cf160c2-4423-4044-bdfb-23887da5c6e0 --nic net-id=ae53dd16-27d9-40e0-b0b5-36e1140a7e0b --config-drive=True --image rhel2 --flavor m1.small --key-name id_rsa --availability-zone nova:overcloud-compute-1.localdomain rhel2
~~~
v) Verify instance configuration and config-drive
~~~
[root@rhel2 ~]# cat /mnt/openstack/latest/network_data.json | python -m json.tool
{
"links": [
{
"ethernet_mac_address": "fa:16:3e:61:a3:f3",
"id": "tapda32bb2c-06",
"mtu": 1446,
"type": "ovs",
"vif_id": "da32bb2c-0624-4553-9fbd-aab434440324"
},
{
"ethernet_mac_address": "fa:16:3e:d7:f5:e6",
"id": "tap7f77b17a-3c",
"mtu": 1496,
"type": "ovs",
"vif_id": "7f77b17a-3c1e-46ca-a6e4-9265515e3b16"
},
{
"ethernet_mac_address": "fa:16:3e:7a:38:5e",
"id": "tap1a385c51-f6",
"mtu": 1446,
"type": "ovs",
"vif_id": "1a385c51-f67d-4da8-b8c8-059abf0d6b03"
}
],
"networks": [
{
"id": "network0",
"ip_address": "192.168.0.8",
"link": "tapda32bb2c-06",
"netmask": "255.255.255.0",
"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1",
"routes": [
{
"gateway": "192.168.0.1",
"netmask": "0.0.0.0",
"network": "0.0.0.0"
}
],
"type": "ipv4"
},
{
"id": "network1",
"ip_address": "2000:192:168::6",
"link": "tapda32bb2c-06",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1",
"routes": [
{
"gateway": "2000:192:168::1",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
},
{
"id": "network2",
"ip_address": "10.0.0.103",
"link": "tap7f77b17a-3c",
"netmask": "255.255.255.0",
"network_id": "7cf160c2-4423-4044-bdfb-23887da5c6e0",
"routes": [
{
"gateway": "10.0.0.250",
"netmask": "0.0.0.0",
"network": "0.0.0.0"
}
],
"type": "ipv4"
},
{
"id": "network3",
"ip_address": "2000:10::b",
"link": "tap7f77b17a-3c",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "7cf160c2-4423-4044-bdfb-23887da5c6e0",
"routes": [
{
"gateway": "2000:10::250",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
},
{
"id": "network4",
"ip_address": "2000:123::7",
"link": "tap1a385c51-f6",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "ae53dd16-27d9-40e0-b0b5-36e1140a7e0b",
"routes": [
{
"gateway": "2000:10::250",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
}
],
"services": [
{
"address": "8.8.8.8",
"type": "dns"
}
]
}
~~~
~~~
[root@rhel2 ~]# cat /mnt/openstack/latest/network_data.json | python -m json.tool
{
"links": [
{
"ethernet_mac_address": "fa:16:3e:61:a3:f3",
"id": "tapda32bb2c-06",
"mtu": 1446,
"type": "ovs",
"vif_id": "da32bb2c-0624-4553-9fbd-aab434440324"
},
{
"ethernet_mac_address": "fa:16:3e:d7:f5:e6",
"id": "tap7f77b17a-3c",
"mtu": 1496,
"type": "ovs",
"vif_id": "7f77b17a-3c1e-46ca-a6e4-9265515e3b16"
},
{
"ethernet_mac_address": "fa:16:3e:7a:38:5e",
"id": "tap1a385c51-f6",
"mtu": 1446,
"type": "ovs",
"vif_id": "1a385c51-f67d-4da8-b8c8-059abf0d6b03"
}
],
"networks": [
{
"id": "network0",
"ip_address": "192.168.0.8",
"link": "tapda32bb2c-06",
"netmask": "255.255.255.0",
"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1",
"routes": [
{
"gateway": "192.168.0.1",
"netmask": "0.0.0.0",
"network": "0.0.0.0"
}
],
"type": "ipv4"
},
{
"id": "network1",
"ip_address": "2000:192:168::6",
"link": "tapda32bb2c-06",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "88bdc97d-9c3e-4f0d-80c5-00df3f7dc5d1",
"routes": [
{
"gateway": "2000:192:168::1",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
},
{
"id": "network2",
"ip_address": "10.0.0.103",
"link": "tap7f77b17a-3c",
"netmask": "255.255.255.0",
"network_id": "7cf160c2-4423-4044-bdfb-23887da5c6e0",
"routes": [
{
"gateway": "10.0.0.250",
"netmask": "0.0.0.0",
"network": "0.0.0.0"
}
],
"type": "ipv4"
},
{
"id": "network3",
"ip_address": "2000:10::b",
"link": "tap7f77b17a-3c",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "7cf160c2-4423-4044-bdfb-23887da5c6e0",
"routes": [
{
"gateway": "2000:10::250",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
},
{
"id": "network4",
"ip_address": "2000:123::7",
"link": "tap1a385c51-f6",
"netmask": "ffff:ffff:ffff:ffff::",
"network_id": "ae53dd16-27d9-40e0-b0b5-36e1140a7e0b",
"routes": [
{
"gateway": "2000:10::250",
"netmask": "::",
"network": "::"
}
],
"type": "ipv6"
}
],
"services": [
{
"address": "8.8.8.8",
"type": "dns"
}
]
}
[root@rhel2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:61:a3:f3 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.8/24 brd 192.168.0.255 scope global eth0:0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe61:a3f3/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1496 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:d7:f5:e6 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.103/24 brd 10.0.0.255 scope global eth1:0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fed7:f5e6/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 qdisc pfifo_fast state UP qlen 1000
link/ether fa:16:3e:7a:38:5e brd ff:ff:ff:ff:ff:ff
inet6 2000:123::7/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe7a:385e/64 scope link
valid_lft forever preferred_lft forever
[root@rhel2 ~]# ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 state UP qlen 1000
inet6 fe80::f816:3eff:fe61:a3f3/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1496 state UP qlen 1000
inet6 fe80::f816:3eff:fed7:f5e6/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1446 state UP qlen 1000
inet6 2000:123::7/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe7a:385e/64 scope link
valid_lft forever preferred_lft forever
[root@rhel2 ~]# ls /etc/sysconfig/network-scripts/ifcfg-*
/etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1:0
/etc/sysconfig/network-scripts/ifcfg-eth0:0 /etc/sysconfig/network-scripts/ifcfg-eth1:1
/etc/sysconfig/network-scripts/ifcfg-eth0:1 /etc/sysconfig/network-scripts/ifcfg-eth2
/etc/sysconfig/network-scripts/ifcfg-eth1 /etc/sysconfig/network-scripts/ifcfg-lo
~~~
~~~
[root@rhel2 network-scripts]# cat ifcfg-eth0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=none
DEVICE=eth0
HWADDR=fa:16:3e:61:a3:f3
MTU=1446
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 network-scripts]# cat ifcfg-eth0:0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth0:0
GATEWAY=192.168.0.1
HWADDR=fa:16:3e:61:a3:f3
IPADDR=192.168.0.8
MTU=1446
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 network-scripts]# cat ifcfg-eth0:1
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth0:1
HWADDR=fa:16:3e:61:a3:f3
IPV6ADDR=2000:192:168::6
IPV6INIT=yes
IPV6_DEFAULTGW=2000:192:168::1
MTU=1446
NETMASK=64
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 network-scripts]# cat ifcfg-eth1:0
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth1:0
GATEWAY=10.0.0.250
HWADDR=fa:16:3e:d7:f5:e6
IPADDR=10.0.0.103
MTU=1496
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 network-scripts]# cat ifcfg-eth1:1
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth1:1
HWADDR=fa:16:3e:d7:f5:e6
IPV6ADDR=2000:10::b
IPV6INIT=yes
IPV6_DEFAULTGW=2000:10::250
MTU=1496
NETMASK=64
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
[root@rhel2 network-scripts]# cat ifcfg-eth2
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEFROUTE=yes
DEVICE=eth2
HWADDR=fa:16:3e:7a:38:5e
IPV6ADDR=2000:123::7
IPV6INIT=yes
IPV6_DEFAULTGW=2000:10::250
MTU=1446
NETMASK=64
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
~~~
Hi, I discovered another bug in cloud-init while testing (and filed it): https://bugs.launchpad.net/cloud-init/+bug/1684349 I also identified the code that is responsible for subinterfaces: /usr/lib/python2.7/site-packages/cloudinit/net/sysconfig.py ~~~ @classmethod def _render_physical_interfaces(cls, network_state, iface_contents): physical_filter = renderer.filter_by_physical for iface in network_state.iter_interfaces(physical_filter): iface_name = iface['name'] iface_subnets = iface.get("subnets", []) iface_cfg = iface_contents[iface_name] route_cfg = iface_cfg.routes if len(iface_subnets) == 1: cls._render_subnet(iface_cfg, route_cfg, iface_subnets[0]) elif len(iface_subnets) > 1: for i, iface_subnet in enumerate(iface_subnets, start=len(iface_cfg.children)): iface_sub_cfg = iface_cfg.copy() iface_sub_cfg.name = "%s:%s" % (iface_name, i) iface_cfg.children.append(iface_sub_cfg) cls._render_subnet(iface_sub_cfg, route_cfg, iface_subnet) ~~~ And a git blame on the master branch reveals: ~~~ f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 295) iface_subnets = iface.get("subnets", []) f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 296) iface_cfg = iface_contents[iface_name] f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 297) route_cfg = iface_cfg.routes f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 298) if len(iface_subnets) == 1: f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 299) cls._render_subnet(iface_cfg, route_cfg, iface_subnets[0]) f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 300) elif len(iface_subnets) > 1: da25385d (Scott Moser 2017-02-17 12:05:38 -0500 301) for i, isubnet in enumerate(iface_subnets, da25385d (Scott Moser 2017-02-17 12:05:38 -0500 302) start=len(iface_cfg.children)): f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 303) iface_sub_cfg = iface_cfg.copy() f47d811a (Joshua Harlow 2016-06-10 14:26:30 -0700 304) iface_sub_cfg.name = "%s:%s" % (iface_name, i) f81d6c7b (Lars Kellogg-Stedman 2017-02-17 08:55:05 -0500 305) iface_cfg.children.append(iface_sub_cfg) da25385d (Scott Moser 2017-02-17 12:05:38 -0500 306) cls._render_subnet(iface_sub_cfg, route_cfg, isubnet) ~~~ Introduced by commit: ~~~ commit f47d811af4f2ae0977db076b3eedb27634da2132 Author: Joshua Harlow <harlowja> Date: Fri Jun 10 14:26:30 2016 -0700 Add a sysconfig renderer ~~~ IMO, we should replace _render_subnet with a method _render_subnets and we should get rid of the .length condition. The new code needs to be able to parse: ~~~ {'subnets': [{u'routes': [{u'netmask': 0, u'network': u'::', u'gateway': u'2000:10::250'}], u'netmask': 64, 'address': u'2000:123::7', u'type': 'static', 'ipv6': True}], 'name': 'eth2', 'address': None, 'mtu': 1446, 'mode': 'manual', 'mac_address': u'fa:16:3e:0d:bc:68', 'type': 'physical', 'gateway': None, 'inet': 'inet'} eth1 {'subnets': [{u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'10.0.0.250'}], u'netmask': u'255.255.255.0', u'type': 'static', 'ipv4': True, 'address': u'10.0.0.111'}, {u'routes': [{u'netmask': 0, u'network': u'::', u'gateway': u'2000:10::250'}], u'netmask': 64, 'address': u'2000:10::9', u'type': 'static', 'ipv6': True}], 'name': 'eth1', 'address': None, 'mtu': 1496, 'mode': 'manual', 'mac_address': u'fa:16:3e:18:ca:3d', 'type': 'physical', 'gateway': None, 'inet': 'inet'} eth0 {'subnets': [{u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'192.168.0.1'}], u'netmask': u'255.255.255.0', u'type': 'static', 'ipv4': True, 'address': u'192.168.0.11'}, {u'routes': [{u'netmask': 0, u'network': u'::', u'gateway': u'2000:192:168::1'}], u'netmask': 64, 'address': u'2000:192:168::4', u'type': 'static', 'ipv6': True}], 'name': 'eth0', 'address': None, 'mtu': 1446, 'mode': 'manual', 'mac_address': u'fa:16:3e:84:0c:82', 'type': 'physical', 'gateway': None, 'inet': 'inet'} ~~~ And a subnets array: [ {u'routes': [{u'netmask': u'0.0.0.0', u'network': u'0.0.0.0', u'gateway': u'10.0.0.250'}], u'netmask': u'255.255.255.0', u'type': 'static', 'ipv4': True, 'address': u'10.0.0.111'}, {u'routes': [{u'netmask': 0, u'network': u'::', u'gateway': u'2000:10::250'}], u'netmask': 64, 'address': u'2000:10::9', u'type': 'static', 'ipv6': True} ] So, in code: ~~~ @classmethod def _render_physical_interfaces(cls, network_state, iface_contents): physical_filter = renderer.filter_by_physical for iface in network_state.iter_interfaces(physical_filter): iface_name = iface['name'] print iface_name print iface iface_subnets = iface.get("subnets", []) iface_cfg = iface_contents[iface_name] route_cfg = iface_cfg.routes if len(iface_subnets) >= 1: cls._render_subnets(iface_cfg, route_cfg, iface_subnets[0]) #elif len(iface_subnets) > 1: # for i, iface_subnet in enumerate(iface_subnets, # start=len(iface_cfg.children)): # iface_sub_cfg = iface_cfg.copy() # iface_sub_cfg.name = "%s:%s" % (iface_name, i) # iface_cfg.children.append(iface_sub_cfg) # cls._render_subnet(iface_sub_cfg, route_cfg, iface_subnet) ~~~ The next step though is to convert iface_subnet to iface_subnets ... and I hope that you see here that this is not so trivial: * while the subnets array contains 'equal' subnets which can be static, dhcp, etc. ... and is not hierarchical ... note that the RHEL network config scripts have a hierarchy, by using IPV6ADDRESS and secondary addresses, and by using IPADDR and IPADDR2, etc ... and what happens if dhcp and static are mixed? I am tired now. Looking into this tomorrow. - Andreas Hi, I fixed the code and also updated the unit tests (which contained invalid IPv6 alias files, no surprise here). This is the merge request: https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/322998 The merge is currently waiting for review. Here are the changes (from master, but the sysconfig.py file has not diverged too much between the newest RHEL version and master): diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py index 504e4d0..babc5cd 100644 --- a/cloudinit/net/sysconfig.py +++ b/cloudinit/net/sysconfig.py @@ -59,6 +59,9 @@ class ConfigMap(object): def __setitem__(self, key, value): self._conf[key] = value + def __getitem__(self, key): + return self._conf[key] + def drop(self, key): self._conf.pop(key, None) @@ -211,27 +214,67 @@ class Renderer(renderer.Renderer): iface_cfg[new_key] = old_value @classmethod - def _render_subnet(cls, iface_cfg, route_cfg, subnet): - subnet_type = subnet.get('type') - if subnet_type == 'dhcp6': - iface_cfg['DHCPV6C'] = True - iface_cfg['IPV6INIT'] = True - iface_cfg['BOOTPROTO'] = 'dhcp' - elif subnet_type in ['dhcp4', 'dhcp']: - iface_cfg['BOOTPROTO'] = 'dhcp' - elif subnet_type == 'static': - iface_cfg['BOOTPROTO'] = 'static' - if subnet_is_ipv6(subnet): - iface_cfg['IPV6ADDR'] = subnet['address'] + def _render_subnets(cls, iface_cfg, subnets): + # setting base values + # iface_cfg['DHCPV6C'] = False + # iface_cfg['IPV6INIT'] = False + iface_cfg['BOOTPROTO'] = 'none' + + # modifying base values according to subnets + for i, subnet in enumerate(subnets,start=len(iface_cfg.children)): + subnet_type = subnet.get('type') + if subnet_type == 'dhcp6': iface_cfg['IPV6INIT'] = True + iface_cfg['DHCPV6C'] = True + iface_cfg['BOOTPROTO'] = 'dhcp' + elif subnet_type in ['dhcp4', 'dhcp']: + iface_cfg['BOOTPROTO'] = 'dhcp' + elif subnet_type == 'static': + if iface_cfg['BOOTPROTO'] == 'none': + iface_cfg['BOOTPROTO'] = 'static' + if subnet_is_ipv6(subnet): + iface_cfg['IPV6INIT'] = True else: - iface_cfg['IPADDR'] = subnet['address'] - else: - raise ValueError("Unknown subnet type '%s' found" - " for interface '%s'" % (subnet_type, + raise ValueError("Unknown subnet type '%s' found" + " for interface '%s'" % (subnet_type, iface_cfg.name)) - if 'netmask' in subnet: - iface_cfg['NETMASK'] = subnet['netmask'] + + # set IPv4 and IPv6 static addresses + ipv4_index = -1 + ipv6_index = -1 + for i, subnet in enumerate(subnets,start=len(iface_cfg.children)): + subnet_type = subnet.get('type') + if subnet_type == 'dhcp6': + continue + elif subnet_type in ['dhcp4', 'dhcp']: + continue + elif subnet_type == 'static': + if subnet_is_ipv6(subnet): + ipv6_index = ipv6_index + 1 + if 'netmask' in subnet and subnet['netmask'] != "": + ipv6_cidr = subnet['address'] + '/' + subnet['netmask'] + else: + ipv6_cidr = subnet['address'] + if ipv6_index == 0: + iface_cfg['IPV6ADDR'] = ipv6_cidr + elif ipv6_index == 1: + iface_cfg['IPV6ADDR_SECONDARIES'] = ipv6_cidr + else: + iface_cfg['IPV6ADDR_SECONDARIES'] = iface_cfg['IPV6ADDR_SECONDARIES'] + " " + ipv6_cidr + else: + ipv4_index = ipv4_index + 1 + if ipv4_index == 0: + iface_cfg['IPADDR'] = subnet['address'] + if 'netmask' in subnet: + iface_cfg['NETMASK'] = subnet['netmask'] + else: + iface_cfg['IPADDR' + str(ipv4_index)] = subnet['address'] + if 'netmask' in subnet: + iface_cfg['NETMASK' + str(ipv4_index)] = subnet['netmask'] + + @classmethod + def _render_subnet_routes(cls, iface_cfg, route_cfg, subnets): + for i, subnet in enumerate(subnets,start=len(iface_cfg.children)): for route in subnet.get('routes', []): if subnet.get('ipv6'): gw_cfg = 'IPV6_DEFAULTGW' @@ -295,15 +338,9 @@ class Renderer(renderer.Renderer): iface_subnets = iface.get("subnets", []) iface_cfg = iface_contents[iface_name] route_cfg = iface_cfg.routes - if len(iface_subnets) == 1: - cls._render_subnet(iface_cfg, route_cfg, iface_subnets[0]) - elif len(iface_subnets) > 1: - for i, isubnet in enumerate(iface_subnets, - start=len(iface_cfg.children)): - iface_sub_cfg = iface_cfg.copy() - iface_sub_cfg.name = "%s:%s" % (iface_name, i) - iface_cfg.children.append(iface_sub_cfg) - cls._render_subnet(iface_sub_cfg, route_cfg, isubnet) + + cls._render_subnets(iface_cfg, iface_subnets) + cls._render_subnet_routes(iface_cfg, route_cfg, iface_subnets) Hi, Can we do something to speed up the merge of the upstream commit and possibly get the customer a hotfix RPM? - Andreas https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/323089 https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/322992 Andreas, The original merge request, https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/322998, is marked as "superseded", and the first url from comment #13, https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/323089, leads to a 404. Would you mind posting updated links? Thanks! Hi, Merge request: https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/323340 Also note that along the way I had discovered a cast error somewhere in cloud-init https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/322992 Feel free to provide your own code. What the customer cares about is that we eliminate the use of alias files (which in the case of IPv4 are considered outdated, and in the case of IPv6 are invalid). Also note that IPv6 routes need to be provided in route6-<interface> and cannot be provided in route-<interface> files. Hi, I'm just checking to see if we have any news for this. - Andreas (In reply to Andreas Karis from comment #16) > Hi, > > I'm just checking to see if we have any news for this. > > - Andreas No news, the upstream patch is still pending review. Created attachment 1279727 [details]
new patch against master
I've reworked the patch so it now applies against master, and I ran the unittests against the resultant tree with no differences from the previous results, but beyond that, I'm not sure how to test that this is sane. Could somebody more familiar have a look/check?
Hi, I created a new merge request against master with a rebace: https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/324195 *rebase Hi, I identified the issue with default route creation. I still have to test this more thoroughly and asked the customer to confirm as well. Upstream bug: https://bugs.launchpad.net/cloud-init/+bug/1696176 Merge request: https://code.launchpad.net/~akaris/cloud-init/+git/cloud-init/+merge/325174 Do I open a new BZ for this or are we going to move this one back to ASSIGNED? Thanks, - Andreas Reading the upstream bug, it's not clear if this is an actual problem or just a cosmetic issue: "Since f38fa41317602908139aa96e930b634f65e39555 , default routes get added to both ifcfg-* and route-* and route6-* files. Default routes should only go to ifcfg-* files, otherwise the information is redundant." Does the redundant information cause an operational problem (no default route is set) or does it result in error messages but no operational impact, etc? Thanks. It's mostly cosmetic from my point of view:
a) redundant routes in IPv4 sysconfig files cause no issue at all
* if GATEWAY is set in ifcfg-* and a default route is configured in route-*, then only one default route will be configured
b) redundant routes in IPv6 sysconfig files cause duplicate default route which AFAICT doesn't cause issues:
* if IPV6_DEFAULTGW is set in in ifcfg-* and a default route is configured in route6-*, then 2 default routes with the same gateway but different metric will be configured (which shouldn't cause an issue, though)
default via 2000:192:168:200::1 dev eth2 metric 1
default via 2000:192:168:200::1 dev eth2 metric 1024
c) With dual-stack networks, the current code will create case b) for IPv6, while it does not create case a) for IPv4. This is due to
gw_key = 'GATEWAY0'
nm_key = 'NETMASK0'
addr_key = 'ADDRESS0'
Due to the fact that I hardcoded the '0' in there, and for the IPv4 and IPv6 default route, it will pass that one twice, meaning that it assigns the same index to the IPv4 and IPv6 route, meaning that the first one will not appear in the array of routes (it will however still be configured as a default route). So accidentally, in dual stack networks, IPv4 does behave as it should - only creating a GATEWAY in ifcfg-* and not creating a route in route-*
*** Bug 1457971 has been marked as a duplicate of this bug. *** Code verification done on rhel 7.4 guest. cloud-init: cloud-init[1750]: Cloud-init v. 0.7.9 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:2275 |