1492726 – Cloud-init fails to set the default gateway on RHEL 7.4 VM

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1492726 - Cloud-init fails to set the default gateway on RHEL 7.4 VM

Summary: Cloud-init fails to set the default gateway on RHEL 7.4 VM

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	cloud-init
Sub Component:
Version:	7.4
Hardware:	Unspecified
OS:	Unspecified
Priority:	urgent
Severity:	urgent
Target Milestone:	rc
Target Release:	---
Assignee:	Ryan McCabe
QA Contact:	Vladimir
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1539760 (view as bug list)
Depends On:	1473890
Blocks:	1495441 1519271 1539760 1540094
TreeView+	depends on / blocked

Reported:	2017-09-18 14:22 UTC by Simone Tiraboschi
Modified:	2024-11-21 02:54 UTC (History)
CC List:	32 users (show)
Fixed In Version:	cloud-init-0.7.9-19.el7
Doc Type:	Bug Fix
Doc Text:	Previously, the cloud-init service did not apply the default gateway parameters for Red Hat Enterprise Linux 7 guest virtual machines, which caused the guests to be impossible to log in to using the Red Hat Virtualization and Red Hat OpenStack Platform clients. The gateway parameters are now applied correctly, and the affected login processes now work as expected.
Clone Of:
Clones:	1539760 1540094 (view as bug list)
Environment:
Last Closed:	2018-04-10 14:05:07 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Launchpad	1686856	None	None	None	2017-09-19 05:31:02 UTC
Red Hat Bugzilla	1492791	unspecified	CLOSED	Engine VM has no external connectivity due to unconfigured default gateway if deployed with static IP	2021-02-22 00:41:40 UTC
Red Hat Product Errata	RHEA-2018:0806	normal	SHIPPED_LIVE	cloud-init bug fix and enhancement update	2018-04-10 12:40:54 UTC

Internal Links: 1492791

Description Simone Tiraboschi 2017-09-18 14:22:12 UTC

Description of problem:
on latest version of cloud-init, it does not apply the default gateway on RHEL 7.4 vm.
Seen using the no-cloud provider with a static networking configuration.

 [root@enginevm ~]# nmcli con show "System eth0" | grep -i GATEWAY
 connection.gateway-ping-timeout:        0
 ipv4.gateway:                           --
 ipv6.gateway:                           --
 IP4.GATEWAY:                            --
 IP6.GATEWAY:                            fe80::c4ee:3eff:fed5:fad9
 [root@enginevm ~]# nmcli con modify "System eth0" ipv4.gateway
 Error: value for 'ipv4.gateway' is missing.
 [root@enginevm ~]# 
 [root@enginevm ~]# nmcli con show "System eth0" | grep -i GATEWAY
 connection.gateway-ping-timeout:        0
 ipv4.gateway:                           --
 ipv6.gateway:                           --
 IP4.GATEWAY:                            --
 IP6.GATEWAY:                            fe80::c4ee:3eff:fed5:fad9
 [root@enginevm ~]# nmcli con modify "System eth0" ipv4.gateway 192.168.1.1
 [root@enginevm ~]# nmcli con reload "System eth0"
 [root@enginevm ~]# nmcli con up "System eth0"
 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3)
 [root@enginevm ~]# nmcli con show "System eth0" | grep -i GATEWAY
 connection.gateway-ping-timeout:        0
 ipv4.gateway:                           192.168.1.1
 ipv6.gateway:                           --
 IP4.GATEWAY:                            192.168.1.1
 IP6.GATEWAY:                            fe80::c4ee:3eff:fed5:fad9
 [root@enginevm ~]# mount /dev/sr0 /mnt/
 mount: /dev/sr0 is write-protected, mounting read-only
 [root@enginevm ~]# cat /mnt/meta-data 
 instance-id: d8b22f43-1565-44e2-916f-f211c7e07f13
 local-hostname: enginevm.localdomain
 network-interfaces: |
   auto eth0
   iface eth0 inet static
     address 192.168.1.204
     network 192.168.1.0
     netmask 255.255.255.0
     broadcast 192.168.1.255
     gateway 192.168.1.1

under /var/log/messages:
Sep 18 09:47:34 localhost cloud-init: Cloud-init v. 0.7.9 running 'init' at Mon, 18 Sep 2017 09:47:34 +0000. Up 8.48 seconds.
Sep 18 09:47:34 localhost cloud-init: ci-info: +++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++
Sep 18 09:47:34 localhost cloud-init: ci-info: +--------+------+---------------+---------------+-------+-------------------+
Sep 18 09:47:34 localhost cloud-init: ci-info: | Device |  Up  |    Address    |      Mask     | Scope |     Hw-Address    |
Sep 18 09:47:34 localhost cloud-init: ci-info: +--------+------+---------------+---------------+-------+-------------------+
Sep 18 09:47:34 localhost cloud-init: ci-info: |  lo:   | True |   127.0.0.1   |   255.0.0.0   |   .   |         .         |
Sep 18 09:47:34 localhost cloud-init: ci-info: |  lo:   | True |       .       |       .       |   d   |         .         |
Sep 18 09:47:34 localhost cloud-init: ci-info: | eth0:  | True | 192.168.1.204 | 255.255.255.0 |   .   | 00:16:3e:10:dc:25 |
Sep 18 09:47:34 localhost cloud-init: ci-info: | eth0:  | True |       .       |       .       |   d   | 00:16:3e:10:dc:25 |
Sep 18 09:47:34 localhost cloud-init: ci-info: +--------+------+---------------+---------------+-------+-------------------+
Sep 18 09:47:34 localhost cloud-init: ci-info: +++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++
Sep 18 09:47:34 localhost cloud-init: ci-info: +-------+-------------+---------+---------------+-----------+-------+
Sep 18 09:47:34 localhost cloud-init: ci-info: | Route | Destination | Gateway |    Genmask    | Interface | Flags |
Sep 18 09:47:34 localhost cloud-init: ci-info: +-------+-------------+---------+---------------+-----------+-------+
Sep 18 09:47:34 localhost cloud-init: ci-info: |   0   | 192.168.1.0 | 0.0.0.0 | 255.255.255.0 |    eth0   |   U   |
Sep 18 09:47:34 localhost cloud-init: ci-info: +-------+-------------+---------+---------------+-----------+-------+
Sep 18 09:47:34 localhost systemd: Started Dynamic System Tuning Daemon.
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.7326] ifcfg-rh: update /etc/sysconfig/network-scripts/ifcfg-eth0 (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03,"System eth0")
Sep 18 09:47:34 localhost systemd: Started Postfix Mail Transport Agent.
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.8337] device (eth0): state change: activated -> deactivating (reason 'user-requested') [100 110 39]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.8341] manager: NetworkManager state is now DISCONNECTING
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.8382] audit: op="device-disconnect" interface="eth0" ifindex=2 pid=1384 uid=0 result="success"
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.8384] device (eth0): state change: deactivating -> disconnected (reason 'user-requested') [110 30 39]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.8406] manager: NetworkManager state is now DISCONNECTED
Sep 18 09:47:34 localhost nm-dispatcher: req:3 'down' [eth0]: new request (4 scripts)
Sep 18 09:47:34 localhost cloud-init: Device 'eth0' successfully disconnected.
Sep 18 09:47:34 localhost nm-dispatcher: req:3 'down' [eth0]: start running ordered scripts...
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9309] device (eth0): Activation: starting connection 'System eth0' (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03)
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9310] audit: op="connection-activate" uuid="5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03" name="System eth0" pid=1423 uid=0 result="success"
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9310] device (eth0): state change: disconnected -> prepare (reason 'none') [30 40 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9311] manager: NetworkManager state is now CONNECTING
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9313] device (eth0): state change: prepare -> config (reason 'none') [40 50 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9461] device (eth0): state change: config -> ip-config (reason 'none') [50 70 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9513] device (eth0): state change: ip-config -> ip-check (reason 'none') [70 80 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9521] device (eth0): state change: ip-check -> secondaries (reason 'none') [80 90 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9522] device (eth0): state change: secondaries -> activated (reason 'none') [90 100 0]
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9523] manager: NetworkManager state is now CONNECTED_LOCAL
Sep 18 09:47:34 localhost cloud-init: Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/2)
Sep 18 09:47:34 localhost NetworkManager[829]: <info>  [1505728054.9570] device (eth0): Activation: successful, device activated.
Sep 18 09:47:34 localhost nm-dispatcher: req:4 'up' [eth0]: new request (4 scripts)
Sep 18 09:47:34 localhost nm-dispatcher: req:4 'up' [eth0]: start running ordered scripts...

Cloud-init configuration:
[root@enginevm ~]# cat /mnt/meta-data 
instance-id: d8b22f43-1565-44e2-916f-f211c7e07f13
local-hostname: enginevm.localdomain
network-interfaces: |
  auto eth0
  iface eth0 inet static
    address 192.168.1.204
    network 192.168.1.0
    netmask 255.255.255.0
    broadcast 192.168.1.255
    gateway 192.168.1.1


Version-Release number of selected component (if applicable):
cloud-init.x86_64               0.7.9-9.el7.centos.2    @anaconda 

How reproducible:
100%

Steps to Reproduce:
1. try to configure static ipv4 networking with a gateway with cloud-init and no-cloud provider
2.
3.

Actual results:
it ignores the gateway parameter

Expected results:
it applies also the gateway parameters

Additional info:

Comment 2 Simone Tiraboschi 2017-09-18 15:06:58 UTC

[root@enginevm ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 
# Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=none
DEVICE=eth0
IPADDR=192.168.1.204
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no
DNS1=192.168.1.1
DNS2=8.8.8.8
DOMAIN=localdomain

Comment 3 Sandro Bonazzola 2017-09-19 05:31:47 UTC

Proposing for backport to 7.4 too

Comment 7 Ryan McCabe 2017-09-27 14:09:02 UTC

Moving back to assigned. The fix upstream is incomplete. Still waiting for them to pull the complete fixes in. The latest release (17.1) and the master branch are still broken.

Comment 8 Ryan McCabe 2017-09-27 14:35:51 UTC

..and moving back to POST. I had stale bits from 0.7.9 lying around that were being used while I was testing. The upstream fix does seem to resolve GATEWAY not being written out.

Comment 13 Vladimir 2017-11-16 14:35:18 UTC

Verified from oVirt side using 
oVirt version 4.2.0-0.0.master.20171112130303.git8bc889c.el7.centos
VDSM Version: vdsm-4.20.7-1.gitc9cf1ee.el7.centos 
cloud-init version 0.7.9-14.el7

Comment 14 Israel Pinto 2017-11-20 08:11:24 UTC

Just setting the verify after vlad tested it see comment 13

Comment 15 Vladimir 2017-11-30 13:12:54 UTC

Hi, Ryan

I've just checked cloud-init with IPv6 and there seems to ba a bug there

Here is the network_json 

cat network_data.json 
{
  "links" : [ {
    "name" : "eth1",
    "id" : "eth1",
    "type" : "vif"
  } ],
  "services" : [ {
    "address" : "1.2.3.4",
    "type" : "dns-nameserver"
  }, {
    "address" : "search.test.foo",
    "type" : "dns-search"
  } ],
  "networks" : [ {
    "netmask" : "255.255.255.0",
    "link" : "eth1",
    "id" : "eth1",
    "ip_address" : "10.0.0.1",
    "type" : "ipv4",
    "gateway" : "10.0.0.254"
  }, {
    "netmask" : "64",
    "link" : "eth1",
    "id" : "eth1",
    "ip_address" : "2620:0052:0000:2342:021a:4aff:fe16:8890",
    "type" : "ipv6",
    "gateway" : "2620:52:0:2342::1"
  } ]

And here is # Created by cloud-init on instance boot automatically, do not edit.
#
BOOTPROTO=none
DEVICE=eth1
GATEWAY=2620:52:0:2342::1
IPADDR=10.0.0.1
IPV6ADDR=2620:0052:0000:2342:021a:4aff:fe16:8890/64
IPV6INIT=yes
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
USERCTL=no

cloud-init-0.7.9-18.el7.x86_64

Comment 17 Vladimir 2017-12-01 12:55:51 UTC

Verified on 4.2.0-0.5.master.el7
cloud-init-0.7.9-19test1.el7.x86_64

Gateway was set correctly

Comment 19 Israel Pinto 2017-12-20 11:30:41 UTC

(In reply to Vladimir from comment #17)
> Verified on 4.2.0-0.5.master.el7
> cloud-init-0.7.9-19test1.el7.x86_64
> 
> Gateway was set correctly

moving to verify

Comment 20 Felipe Alfaro Solana 2017-12-21 08:55:18 UTC

Any progress on this? It is very annoying having to resort to using DHCP specially on simple virtualized environments without DNS integration or mDNS support.

Comment 21 Ryan McCabe 2017-12-21 14:22:45 UTC

(In reply to Felipe Alfaro Solana from comment #20)
> Any progress on this? It is very annoying having to resort to using DHCP
> specially on simple virtualized environments without DNS integration or mDNS
> support.

Yes, it is fixed in the latest RHEL 7.5 build. You can grab it from brew. If you need it sooner than that and can make the case to PM, you could request a 7.4.z release.

Comment 22 Felipe Alfaro Solana 2017-12-21 14:33:42 UTC

I don't think I have access to brew (I'm not a Red Hat engineer).

Comment 23 Ryan McCabe 2017-12-21 15:06:08 UTC

Sorry, should have checked that first. You can grab the latest build here: https://people.redhat.com/rmccabe/cloud-init/cloud-init-0.7.9-20.el7.x86_64.rpm

Comment 26 Kriston Rehberg 2018-01-23 05:40:59 UTC

This affected me when upgrading a CentOS 7 virtual machine at DigitalOcean.com.  When upgrading a virtual machine to another pricing plan on DigitalOcean, their cloud-init script correctly sets the GATEWAY but this is ignored due to this bug and this results in complete lack of network connectivity in or out.

The only solution, as far as I can tell, is to log into the graphical console and manually edit the /etc/sysconfig/network-scripts/ifcfg-eth0 and add the missing GATEWAY line.  That works if your provider offers console access and I need to point out that AWS and Azure do not offer console access.

I suspect that the cloud providers that don't offer graphical console access will have their virtual machines "bricked" by this unfortunate oversight.

Comment 27 Ryan McCabe 2018-01-29 20:24:47 UTC

*** Bug 1539760 has been marked as a duplicate of this bug. ***

Comment 31 errata-xmlrpc 2018-04-10 14:05:07 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0806

Note You need to log in before you can comment on or make changes to this bug.

agoulart
aherr
amedeo.salvati
baptiste.agasse
danken
dvd
eraviv
fdelorey
fdinitto
felipe.alfaro
fgarciad
gianluca.cecchi
ipinto
jbadiapa
jgreguske
jherrman
jhunsaker
lars
lmiksik
matthieu.pieters
mavital
me
mmagr
mrunge
pvilayat
redhat
rmccabe
sebastiaan
snagar
vhutsky
vshypygu
xiachen