Bug 1473890 - cloud-init sysconfig renderer does not render DNS or GATEWAY [NEEDINFO]
cloud-init sysconfig renderer does not render DNS or GATEWAY
Status: CLOSED EOL
Product: Fedora
Classification: Fedora
Component: cloud-init (Show other bugs)
26
All Linux
unspecified Severity high
: ---
: ---
Assigned To: Garrett Holmstrom
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 1489270 1492726 1539760
  Show dependency treegraph
 
Reported: 2017-07-22 00:19 EDT by Richard Chan
Modified: 2018-05-29 07:51 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-05-29 07:51:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rmccabe: needinfo? (gholms)


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1645404 None None None 2017-07-23 03:15 EDT
Launchpad 1645597 None None None 2017-07-23 03:14 EDT
Launchpad 1686856 None None None 2017-07-23 03:12 EDT
Launchpad 1705804 None None None 2017-07-22 01:13 EDT

  None (edit)
Description Richard Chan 2017-07-22 00:19:45 EDT
Description of problem:
When cloud-init reads NoCloud datasource meta-data with ENI network data it does not render GATEWAY and DNS1, DN2, DNS3 in /etc/sysconfig/network-scripts/ifcfg-eth0.

The resulting VM has incomplete network configuration: no default gateway and nameserver configured. Surely this affects Fedora Atomic Hosts as well?

Version-Release number of selected component (if applicable):

cloud-init-0.7.9-7.fc26.noarch

How reproducible:
Always

Steps to Reproduce:
1. Use fedora f26 cloud image with cloud-init enabled. Attach NoCloud drive with meta-data file:
instance-id: iid-nocloud
hostname: nocloud
network-interfaces: |
  auto eth0
  iface eth0 inet static
  address 192.168.125.154
  netmask 255.255.255.0
  gateway 192.168.125.1
  dns-nameservers 192.168.125.1 8.8.8.8 8.8.4.4 1.2.3.4
  dns-search example.com foo.biz bar.info foobar.toomany
   
2.
3.

Actual results:
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEVICE=eth0
IPADDR=192.168.125.154
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no



Expected results:
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=static
DEVICE=eth0
DNS1=192.168.125.1
DNS2=8.8.8.8
DNS3=8.8.4.4
GATEWAY=192.168.125.1
HELLO=world
IPADDR=192.168.125.154
NETMASK=255.255.255.0
ONBOOT=yes
SEARCH="example.com foo.biz bar.info"
TYPE=Ethernet


Additional info:
Problem can be reproduced on upstream ubuntu/zesty branch (presumably the latest)

In cloudinit/net/sysconfig.py I do not see anywhere that DNSX or GATEWAY can 
be rendered

The network_state object created from cloudinit/net/eni.py:convert_eni_data doesn't have type == 'nameserver' or type == 'route'. All the data from ENI input is in the 'subnets' key of the interface.  

Global rendering of /etc/resolv.conf will never take place. However it would be better to get DNSX= in ifcfg-eth0, and have /etc/resolv.conf generated by network.service or NetworkManager.service.

The sysconfig renderer seems to be semi-broken in this respect. It needs something like:

    @classmethod
    def _render_iface_shared(cls, iface, iface_cfg):
        print("XXA:", iface, iface_cfg)
        for k, v in cls.iface_defaults:
            iface_cfg[k] = v

        for (old_key, new_key) in [('mac_address', 'HWADDR'), ('mtu', 'MTU')]:
            old_value = iface.get(old_key)
            if old_value is not None:
                iface_cfg[new_key] = old_value

        if 'subnets' in iface:
            for subnet in iface['subnets']:
                if 'dns_search' in subnet:
                    search_str = ""
                    for i, k in enumerate(subnet['dns_search']):
                        if i > 3:
                            break
                        if i > 0:
                            search_str = search_str + " "
                        search_str = search_str + k
                    iface_cfg['SEARCH'] = search_str

                if 'dns_nameservers' in subnet:
                    ns_str = ""
                    for i, k in enumerate(subnet['dns_nameservers']):
                        if i == 3:
                            break
                        iface_cfg['DNS'+str(i+1)] = k
        
                if 'gateway' in subnet:
                    iface_cfg['GATEWAY'] = subnet['gateway']
Comment 1 Richard Chan 2017-07-22 00:21:34 EDT
upstream ubuntu/zesty branch has diverged a lot for F26 with netplan etc. It seems to have the same problem though.
Comment 2 Richard Chan 2017-07-22 00:23:40 EDT
network_state object: the way ENI is converted to network_config and then to network_state means that the dns and routes keys are always empty in simple configurations.


Internal State
!!python/object:cloudinit.net.network_state.NetworkState
_network_state:
    dns:
        nameservers: []
        search: []
    interfaces:
        eth0:
            address: null
            gateway: null
            inet: inet
            mac_address: null
            mode: manual
            mtu: null
            name: eth0
            subnets:
            -   _orig_eni_name: eth0
                address: 192.168.125.154
                control: auto
                dns_nameservers:
                - 192.168.125.1
                - 8.8.8.8
                - 8.8.4.4
                - 1.2.3.4
                dns_search:
                - example.com
                - foo.biz
                - bar.info
                - foobar.toomany
                gateway: 192.168.125.1
                netmask: 255.255.255.0
                type: static
            type: physical
    routes: []
_version: 1
Comment 3 Richard Chan 2017-07-22 00:28:21 EDT
Ooops in the Description please ignore the line

HELLO=world

That was a test line injected when I was recoding sysconfig.py.
Comment 4 Richard Chan 2017-07-22 00:36:09 EDT
Upstream has a test tool tools/net-convert.py: it runs on F26 by this diff
--- net-convert.py      2017-07-21 21:40:58.850805568 +0800
+++ abcd-convert.py     2017-07-22 11:54:27.095124513 +0800
@@ -9,7 +9,6 @@
 from cloudinit.sources.helpers import openstack

 from cloudinit.net import eni
-from cloudinit.net import netplan
 from cloudinit.net import network_state
 from cloudinit.net import sysconfig

@@ -69,13 +68,11 @@
     print(yaml.dump(ns, default_flow_style=False, indent=4))
     if args.output_kind == "eni":
         r_cls = eni.Renderer
-    elif args.output_kind == "netplan":
-        r_cls = netplan.Renderer
     else:
         r_cls = sysconfig.Renderer

     r = r_cls()
-    r.render_network_state(ns, target=args.directory)
+    r.render_network_state(network_state=ns, target=args.directory)



The data was generated by

python3 net-convert.py   --network-data eni.txt  --kind eni --output-kind sysconfig -d target/

eni.txt:
auto eth0
iface eth0 inet static
address 192.168.125.154
netmask 255.255.255.0
gateway 192.168.125.1
dns-nameservers 192.168.125.1 8.8.8.8 8.8.4.4 1.2.3.4
dns-search tbs example.com foo.biz bar.info
Comment 5 Richard Chan 2017-07-22 01:25:46 EDT
network_config object: {'version': 1, 'config': [{'type': 'physical', 'name': 'eth0', 'subnets': [{'_orig_eni_name': 'eth0', 'type': 'static', 'control': 'auto', 'address': '192.168.125.154', 'netmask': '255.255.255.0', 'gateway': '192.168.125.1', 'dns_nameservers': ['192.168.125.1', '8.8.8.8', '8.8.4.4', '1.2.3.4'], 'dns_search': ['example.com', 'foo.biz', 'bar.info', 'foobar.toomany']}]}]}
Comment 6 Richard Chan 2017-07-22 01:34:19 EDT
Typo: SEARCH should be replaced by DOMAIN
Comment 7 Richard Chan 2017-07-26 00:16:52 EDT
There is a lot of churn and refactoring in upstream. Any chance we could get into Fedora as a temporary workaround and rebasing?

Minimally invasive, fixes rendering of /etc/resolv.conf and gateway.


diff --git a/cloudinit/net/eni.py b/cloudinit/net/eni.py
index b06ffac..a8983f2 100644
--- a/cloudinit/net/eni.py
+++ b/cloudinit/net/eni.py
@@ -261,6 +261,7 @@ def _ifaces_to_net_config_data(ifaces):
     config = ifaces_to_net_config_data(ifaces)
     state = parse_net_config_data(config)."""
     devs = {}
+    nameservers = []
     for name, data in ifaces.items():
         # devname is 'eth0' for name='eth0:1'
         devname = name.partition(":")[0]
@@ -285,13 +286,19 @@ def _ifaces_to_net_config_data(ifaces):
                 subnet[copy_key] = data[copy_key]
 
         if 'dns' in data:
+            ns = { 'type': 'nameserver' }
             for n in ('nameservers', 'search'):
                 if n in data['dns'] and data['dns'][n]:
                     subnet['dns_' + n] = data['dns'][n]
+                    if n == 'nameservers':
+                        ns['address'] = data['dns'][n]
+                    else:
+                        ns['search'] = data['dns'][n]
+            nameservers.append(ns)
         devs[devname]['subnets'].append(subnet)
 
     return {'version': 1,
-            'config': [devs[d] for d in sorted(devs)]}
+            'config': [devs[d] for d in sorted(devs)] + nameservers}
 
 
 class Renderer(renderer.Renderer):
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index 095a32b..e9fb7a9 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -234,6 +234,13 @@ class Renderer(renderer.Renderer):
                                                       iface_cfg.name))
         if 'netmask' in subnet:
             iface_cfg['NETMASK'] = subnet['netmask']
+        if 'gateway' in subnet:
+            if subnet.get('ipv6'):
+                gw_cfg = 'IPV6_DEFAULTGW'
+            else:
+                gw_cfg = 'GATEWAY'
+            iface_cfg[gw_cfg] = subnet['gateway']
+            iface_cfg['DEFROUTE'] = True
         for route in subnet.get('routes', []):
             if subnet.get('ipv6'):
                 gw_cfg = 'IPV6_DEFAULTGW'
@@ -261,7 +268,10 @@ class Renderer(renderer.Renderer):
                 iface_cfg['DEFROUTE'] = True
                 if 'gateway' in route:
                     iface_cfg[gw_cfg] = route['gateway']
-                route_cfg.has_set_default = True
+                if subnet.get('ipv4'):
+                    route_cfg.has_set_default_ipv4 = True
+                else:
+                    route_cfg.has_set_default_ipv6 = True
             else:
                 gw_key = 'GATEWAY%s' % route_cfg.last_idx
                 nm_key = 'NETMASK%s' % route_cfg.last_idx
Comment 8 Richard Chan 2017-07-26 00:22:04 EDT
With this patch the same NoCloud /meta-data file will render correcty on both RHEL and Fedora for static IP addressing.
Comment 9 Ryan McCabe 2017-09-27 10:07:17 EDT
Still waiting on upstream to take the fixes. The latest release (17.1) and the master branch are still broken.
Comment 10 long.cheung 2018-03-05 05:44:10 EST
Any update on this bugs?
i am using Fedora 27 server still have this problem.

meta-data input
instance-id: 5337159125
local-hostname: testnocloudgateway
network-interfaces: |
  auto ens3
  iface ens3 inet static
  address 103.225.10.40
  netmask 255.255.255.0
  gateway 103.225.10.1
  dns-nameservers 8.8.8.8

Results
BOOTPROTO=static
DEVICE=ens3
IPADDR=103.225.10.40
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no

still missing gateway and dns server
Comment 11 long.cheung 2018-03-05 05:48:19 EST
cloud-init Version
[root@testnocloudgateway ~]# yum list cloud-init*
Last metadata expiration check: 0:00:42 ago on Mon 05 Mar 2018 06:47:13 PM HKT.
Installed Packages
cloud-init.noarch                                                                    0.7.9-9.fc27                                                                     @fedora
Comment 12 Ryan McCabe 2018-03-05 11:21:38 EST
Garrett, you can grab the patches to fix this (and a few other issues) for 0.7.9 from https://people.redhat.com/rmccabe/cloud-init/cloud-init-0.7.9-24.el7.src.rpm
Comment 13 Fedora End Of Life 2018-05-03 04:09:19 EDT
This message is a reminder that Fedora 26 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 26. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '26'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 26 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
Comment 14 Fedora End Of Life 2018-05-29 07:51:54 EDT
Fedora 26 changed to end-of-life (EOL) status on 2018-05-29. Fedora 26
is no longer maintained, which means that it will not receive any
further security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.