Bug 1236219 - /usr/share/nova/interfaces.template is broken (does not render correctly) and not useful with cloud-init
/usr/share/nova/interfaces.template is broken (does not render correctly) and...
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova (Show other bugs)
12.0 (Pike)
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Rajesh Tailor
Joe H. Rahme
: Reopened, Triaged, ZStream
Depends On:
  Show dependency treegraph
Reported: 2015-06-26 17:58 EDT by Lars Kellogg-Stedman
Modified: 2018-03-16 11:29 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-06-05 13:18:57 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Lars Kellogg-Stedman 2015-06-26 17:58:45 EDT
We distribute /usr/share/nova/interfaces.template that looks like this:

DEVICE="{{ name }}"
IPADDR={{ address }}
NETMASK={{ netmask }}
BROADCAST={{ broadcast }}
GATEWAY={{ gateway }}
DNS1={{ dns }}

#if $use_ipv6
IPV6ADDR={{ address_v6 }}
#end if

There are two problems here:

1. The get_injected_network_template function (in nova.virt.netutils) does not call this with top-level keys matching the ones used in the template, so all the references there (e.g., {{ name }}) expand to the empty string.  The template actually receives a top-level 'interfaces' key with a list of network interfaces to inject.  We could fix this template, but...

2. cloud-init expects a debian-ish "/etc/network/interfaces" format (see, e.g., cloudinit/distros/rhel.py).  So in fact we need a template that looks like this:

{% for interface in interfaces %}
auto {{ interface.name }}
iface {{ interface.name }} inet static
  address {{ interface.address }}
  netmask {{ interface.netmask }}
  broadcast {{ interface.broadcast }}
  gateway {{ interface.gateway }}
  dns-nameservers {{ interface.dns }}
{% endfor %}
Comment 3 Luca Miccini 2018-02-21 11:10:13 EST
Reopening this as it is still relevant for OSP12. 

Instances using configdrive (provider network, no dhcp possible) end up with:

$ cat /mnt/test/openstack/content/0000

#if $use_ipv6

IMHO we should consider using the default template: 


or provide a way to inject it into the nova container during the deployment (ideally without having to create new container).

Additionally I think we should document that in order for cloud-init to make use of such template the flat_injected parameter should be set to true, either extending puppet-nova or by using nova_config like:

        value: true
Comment 4 Rajesh Tailor 2018-03-09 06:51:09 EST
Hi Luca,

I tried to reproduce it with below steps, but couldn't get the exact content of interface mentioned by you in comment#3.

1) After fresh deployment of OSP12, I have set flat_injected=true in /etc/nova/nova.conf on overcloud nodes.

Then I have created a provider network.

openstack network create --external --provider-network-type vlan --provider-physical-network datacentre --provider-segment 10 external

2) Create subnet without dhcp.
openstack subnet create --network external --subnet-range --allocation-pool start=,end= --dns-nameserver --no-dhcp ext_subnet

3) upload cirros image to glance.
openstack image create --file cirros-0.4.0-x86_64-disk.img centos_img

4) I have created required security group, applied rules to security group and created keypair as well.
openstack security group create sg1

openstack security group rule create --protocol icmp sg1

openstack security group rule create --protocol tcp --dst-port 22 sg1

openstack keypair create test_keys > ~/test_keys.pem

4) Create instance with config drive.
openstack server create --image centos_img --flavor m1.test --nic net-id=external --key-name test_keys --security-group sg1 --config-drive true test_vm

After instance creation, I couldn't access vm through assigned IP.

I could see instance disk config on /var/lib/nova/instance/<instance_id>/disk.config location, I tried to mount it to see its contents, but couldn't find content directory in it.

It seems that I might have missed something or done something wrong, would you please provide proper reproduction steps.
Comment 5 Luca Miccini 2018-03-09 07:38:31 EST
Hi Rajesh,

the only difference that I can see is that I deploy with:

        value: true
        value: true

but regardless of whether I set config_drive_cdrom to true or false I consistently see a "sr0" device in my instance. I've tested both cirros 3.5 and cirros 4, same result:

/dev/root resized successfully [took 1.26s]
  ____               ____  ____
 / __/ __ ____ ____ / __ \/ __/
/ /__ / // __// __// /_/ /\ \ 
\___//_//_/  /_/   \____/___/ 

login as 'cirros' user. default password: 'gocubsgo'. use 'sudo' for root.
test-vm3 login: cirros
$ sudo -i
# lsblk
sr0      11:0    1  464K  0 rom  
vda     253:0    0    1G  0 disk 
|-vda1  253:1    0 1015M  0 part /
`-vda15 253:15   0    8M  0 part 
# mount /dev/sr0 /mnt/

# find /mnt

Maybe I can have a look at your environment? 
I am using the 12.0-20180124.1 images fwiw.
Comment 6 Luca Miccini 2018-03-12 09:54:53 EDT
I've tried with the latest docker images available (20180309) and the configdrive now contains valid ip/mask info for both a single nic and multiple nics:

# cat /mnt/openstack/content/0000
# Injected by Nova on instance boot
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
    hwaddress ether fa:16:3e:52:54:c9

auto eth1
iface eth1 inet static
    hwaddress ether fa:16:3e:f0:62:6c

I've tested ubuntu 18, centos7 and cirros and only cirros is still showing the issue:

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether fa:16:3e:52:54:c9 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f816:3eff:fe52:54c9/64 scope link 
       valid_lft forever preferred_lft forever

I've tried using the nova-provided interfaces.template and cirros still does not configure the ip address, so I would consider this an issue with cirros-init as cloud-init seems to be doing the right thing regardless of the configdrive format.

Comment 7 Rajesh Tailor 2018-03-14 01:41:08 EDT

I have also tried with OSP12 as well as master branch.
I think your finding are correct, there is no issue with centos images. I could
get valid ip/mask with configdrive.

Can we close this issue in that case ?
Comment 8 Luca Miccini 2018-03-15 03:24:19 EDT
(In reply to Rajesh Tailor from comment #7)
> Hi,
> I have also tried with OSP12 as well as master branch.
> I think your finding are correct, there is no issue with centos images. I
> could
> get valid ip/mask with configdrive.
> Can we close this issue in that case ?

I think we should try to track down the change/patch that fixed this issue and see if it is backportable to newton. 
I don't have much time today but I'll try to have a look at the code, any help would be appreciated :)
Comment 9 Rajesh Tailor 2018-03-16 02:20:02 EDT
I tried to check the changelog for nova, yet couldn't find anything related, which looks like fixed this issue.
Did you get anything related ?

Is the issue persist in newton ?
In that case, we might need to clone this BZ or open a new BZ for RHOS10.

Note You need to log in before you can comment on or make changes to this bug.