Bug 1288819

Shahar,

Sorry for the delayed response; I must have missed the notification when this bug was created.

I think this comment may address your issue:

https://bugzilla.redhat.com/show_bug.cgi?id=1235602#c3

Please let me know if that resolves the problem.

(In reply to Lars Kellogg-Stedman from comment #1)
Hi Lars,
I checked again and we do use the nocloud debian network style in the meda data file as you can see in comment #1 (we don't and cannot inject network file into the VM).
Does the network line syntax looks good to you?

How exactly are you booting the guest and associating the user-data with it? Can you provide me with a reproducible test case? That would make it much easier to diagnose.  Thanks!

Created attachment 1116171 [details]
Cloud init input sample

Added attachment with the cloud init input.
The input is attach as iso image to the VM.
In this sample there is only network configuration which is not working for us - other input such as host name ssh-keys etc works fine.
The network issue is only for setting static/dynamic nic - setting dns and search domain works fine.

If I take your attachment and unpack it:

$ mkdir config
$ tar -x -f cloud-init.tar.gz -C config

Modify it to work in my network environment:

$ find config -type f -print | xargs sed -i 's/192.168.1/192.168.10/g'

Generate a config drive:

$ mkisofs -V cidata -o config.iso -rJ config

And then use that to boot an instance:

$ virt-install -n citest -w bridge=mybridge,model=virtio --disk vol=default/rhel-guest-7.2.qcow2 -r 1024 --cdrom config.iso --boot hd

It all Just Works.

Of particular interest in the logs is:

Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] rhel.py[DEBUG]: Translated ubuntu style network settings auto eth0
                                                       iface eth0 inet static
                                                         address 192.168.10.100
                                                         netmask 255.255.255.0
                                                         gateway 192.168.10.1
                                                        into {'eth0': {'auto': True, 'netmask': '255.255.255.0', 'bootproto': 'static', 'gateway': '192.168.10.1', 'address': '192.168.10.100'}}
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network-scripts/ifcfg-eth0 (quiet=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Read 146 bytes from /etc/sysconfig/network-scripts/ifcfg-eth0
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [420] 195 bytes
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network (quiet=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Read 30 bytes from /etc/sysconfig/network
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network - wb: [420] 30 bytes
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False)
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] __init__.py[DEBUG]: Attempting to run bring up interface eth0 using command ['ifup', 'eth0']
Jan 19 12:47:53 localhost.localdomain cloud-init[521]: [CLOUDINIT] util.py[DEBUG]: Running command ['ifup', 'eth0'] with allowed return codes [0] (shell=False, capture=True)
Jan 19 12:47:53 localhost.localdomain NetworkManager[528]: <info>  ifcfg-rh: update /etc/sysconfig/network-scripts/ifcfg-eth0 (5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03,"System eth0")

Which results in the following /etc/sysconfig/network-scripts/ifcfg-eth0:

DEVICE=eth0
BOOTPROTO=static
BOOTPROTOv6=dhcp
ONBOOT=yes
TYPE=Ethernet
USERCTL=yes
PEERDNS=yes
IPV6INIT=yes
PERSISTENT_DHCLIENT=1
NETMASK=255.255.255.0
IPADDR=192.168.10.100
GATEWAY=192.168.10.1

Also, note that you're not using the "nocloud" data source.  If you're attaching that data as an ISO image, you're using the ConfigDrive data source.  You're trying to modify the datasource using a "runcmd" action, but this isn't going to have any effect -- by the time cloud-init sees this command, it has already processed the available data source(s).

I can confirm that the script is define and works well.

I have couple of questions:
1. can I associate nic by its mac address? (doing it by name is prone to errors due to client naming)
2. does cloud init suppose to work in rhel with NetworkMnager on (which is by default)

> 1. can I associate nic by its mac address? (doing it by name
> is prone to errors due to client naming)

In most cloud environments, the MAC address of an interface is generally not known in advance, so cloud-init has no facility for identifying interfaces by MAC address.

That said, most cloud-optimized images are configured to use legacy network naming so that the first discovered interface is "eth0", the second is "eth1", etc.  Since the order in which interfaces are specified is usually deterministic, interface naming in virtual environments is generally much less of a problem than on physical hardware.

If you really needed to identify and configure interfaces by MAC address, you could accomplish that by passing a shell script to cloud-init that implements the necessary logic.  You can quickly look up an interface name by mac address by doing something like...

  interface=$(grep $macaddr /sys/class/net/*/address | cut -d/ -f5)

> 2. does cloud init suppose to work in rhel with NetworkMnager 
> on (which is by default)

It should.  cloud-init simply generates an /etc/sysconfig/network-scripts/ifcfg-xxx file and calls "ifup", which ought to integrate with NetworkManager just fine.

Test with RHEL_72 and RHEL_71 cloud init package:
RHEL_7.2 cloud-init-0.7.6-6.el7.x86_64
RHEL_7.1 cloud-init-0.7.6-6.el7.x86_64

Scenarios:
1. Setting NIC to static and start on boot,name server to 1.2.3.4 and search to foo.com 
-----------------------------
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:1a:4a:16:01:51 brd ff:ff:ff:ff:ff:ff
    inet 10.35.163.49/24 brd 10.35.163.255 scope global dynamic eth0
       valid_lft 42921sec preferred_lft 42921sec
    inet6 fe80::21a:4aff:fe16:151/64 scope link tentative dadfailed 
       valid_lft forever preferred_lft forever
----------------------------
/etc/resolv.conf 
# Generated by NetworkManager
nameserver 10.35.28.28
nameserver 10.35.28.1
nameserver 10.34.32.3
search scl.lab.tlv.redhat.com foo.com
-----------------------------
# cat /etc/sysconfig/network-scripts/ifcfg-eth0 
DEVICE=eth0
BOOTPROTO=static
BOOTPROTOv6=dhcp
ONBOOT=yes
TYPE=Ethernet
USERCTL=yes
PEERDNS=yes
IPV6INIT=yes
PERSISTENT_DHCLIENT=1
NETMASK=255.255.255.0
IPADDR=192.168.254.10
GATEWAY=192.168.254.1
----------------------------------
Host Payload:
{
  "network-interfaces" : "auto eth0\niface eth0 inet static\n  address 192.168.254.10\n  netmask 255.255.255.0\n  gateway 192.168.254.1\n  dns-nameservers 1.2.3.4\n  dns-search foo.com\n",
  "availability_zone" : "nova",
  "launch_index" : "0",
  "meta" : {
    "role" : "server",
    "dsmode" : "local",
    "essential" : "false"
  },
  "network_config" : {
    "path" : "/etc/network/interfaces",
    "content_path" : "/content/0000"
  },
  "uuid" : "6241f53d-0841-47d5-b71a-e98dc43737ff"
}#cloud-config
output:
  all: '>> /var/log/cloud-init-output.log'
password: '1'
disable_root: 0
runcmd:
- 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list:
  ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
ssh_pwauth: true
chpasswd:
  expire: false
user: root


2. Update with RunOnce to DHCP
The IP didn't updated the VM IP with the static IP
Host payload:
{
  "network-interfaces" : "auto eth0\niface eth0 inet dhcp\n",
  "availability_zone" : "nova",
  "launch_index" : "0",
  "meta" : {
    "role" : "server",
    "dsmode" : "local",
    "essential" : "false"
  },
  "network_config" : {
    "path" : "/etc/network/interfaces",
    "content_path" : "/content/0000"
  },
  "uuid" : "99e93ad6-db09-497b-80d2-d95c4e389155"
}#cloud-config
output:
  all: '>> /var/log/cloud-init-output.log'
disable_root: 0
runcmd:
- 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list:
  ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
ssh_pwauth: true
chpasswd:
  expire: false

(In reply to Israel Pinto from comment #9)
> Test with RHEL_72 and RHEL_71 cloud init package:
> RHEL_7.2 cloud-init-0.7.6-6.el7.x86_64
> RHEL_7.1 cloud-init-0.7.6-6.el7.x86_64
> 
> Scenarios:
> 1. Setting NIC to static and start on boot,name server to 1.2.3.4 and search
> to foo.com 
> -----------------------------
> # ip a
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
>     link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>     inet 127.0.0.1/8 scope host lo
>        valid_lft forever preferred_lft forever
>     inet6 ::1/128 scope host 
>        valid_lft forever preferred_lft forever
> 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
> UP qlen 1000
>     link/ether 00:1a:4a:16:01:51 brd ff:ff:ff:ff:ff:ff
>     inet 10.35.163.49/24 brd 10.35.163.255 scope global dynamic eth0
>        valid_lft 42921sec preferred_lft 42921sec
>     inet6 fe80::21a:4aff:fe16:151/64 scope link tentative dadfailed 
>        valid_lft forever preferred_lft forever
> ----------------------------
> /etc/resolv.conf 
> # Generated by NetworkManager
> nameserver 10.35.28.28
> nameserver 10.35.28.1
> nameserver 10.34.32.3
> search scl.lab.tlv.redhat.com foo.com
> -----------------------------
> # cat /etc/sysconfig/network-scripts/ifcfg-eth0 
> DEVICE=eth0
> BOOTPROTO=static
> BOOTPROTOv6=dhcp
> ONBOOT=yes
> TYPE=Ethernet
> USERCTL=yes
> PEERDNS=yes
> IPV6INIT=yes
> PERSISTENT_DHCLIENT=1
> NETMASK=255.255.255.0
> IPADDR=192.168.254.10
> GATEWAY=192.168.254.1
> ----------------------------------
> Host Payload:
> {
>   "network-interfaces" : "auto eth0\niface eth0 inet static\n  address
> 192.168.254.10\n  netmask 255.255.255.0\n  gateway 192.168.254.1\n 
> dns-nameservers 1.2.3.4\n  dns-search foo.com\n",
>   "availability_zone" : "nova",
>   "launch_index" : "0",
>   "meta" : {
>     "role" : "server",
>     "dsmode" : "local",
>     "essential" : "false"
>   },
>   "network_config" : {
>     "path" : "/etc/network/interfaces",
>     "content_path" : "/content/0000"
>   },
>   "uuid" : "6241f53d-0841-47d5-b71a-e98dc43737ff"
> }#cloud-config
> output:
>   all: '>> /var/log/cloud-init-output.log'
> password: '1'
> disable_root: 0
> runcmd:
> - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo
> ''datasource_list:
>   ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
> ssh_pwauth: true
> chpasswd:
>   expire: false
> user: root
> 
> 
> 2. Update with RunOnce to DHCP
> The IP didn't updated the VM IP with the static IP
> Host payload:
> {
>   "network-interfaces" : "auto eth0\niface eth0 inet dhcp\n",
>   "availability_zone" : "nova",
>   "launch_index" : "0",
>   "meta" : {
>     "role" : "server",
>     "dsmode" : "local",
>     "essential" : "false"
>   },
>   "network_config" : {
>     "path" : "/etc/network/interfaces",
>     "content_path" : "/content/0000"
>   },
>   "uuid" : "99e93ad6-db09-497b-80d2-d95c4e389155"
> }#cloud-config
> output:
>   all: '>> /var/log/cloud-init-output.log'
> disable_root: 0
> runcmd:
> - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo
> ''datasource_list:
>   ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
> ssh_pwauth: true
> chpasswd:
>   expire: false
cloud init package at rhel_7.1 is:cloud-init-0.7.6-2.el7.x86_64
(copy paste mistake)

The problems that Me and Israel encounter are:
1. when setting static ip with NetworkMnager on we got dhcp from NetworkMnanager - only after restarting the network service we got the static ip.
2. re-running cloud init on the same nic this time with dhcp the ifcfg-eth0 had both configuration the old static ip and the new dhcp.

*** Bug 1288105 has been marked as a duplicate of this bug. ***

IMHO the open issue on cloud-init side is better explained in bug 1363633.
Please reopen if I am mistaken.