Bug 1288105

Summary: [Cloud Init] Networks configuration isn't set on guest
Product: [oVirt] ovirt-engine Reporter: Israel Pinto <ipinto>
Component: BLL.VirtAssignee: Shahar Havivi <shavivi>
Status: CLOSED DUPLICATE QA Contact: Israel Pinto <ipinto>
Severity: medium Docs Contact:
Priority: high    
Version: 3.6.0.3CC: bugs, ipinto, mavital, mgoldboi, michal.skrivanek, shavivi, tjelinek
Target Milestone: ovirt-3.6.5Keywords: AutomationBlocker
Target Release: ---Flags: michal.skrivanek: ovirt-3.6.z?
mgoldboi: planning_ack+
ipinto: devel_ack?
mavital: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-11 09:14:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1288819    
Bug Blocks:    

Description Israel Pinto 2015-12-03 14:38:17 UTC 
Description of problem:
Networks configuration doesn't set on guest:
1. Add NIC and set it to start on boot: NIC doesn't not start 
2. DNS servers are not set in /etc/resolv.conf
3. In order to set DNS search domains you MUST add new NIC
Note: There is open BZ on action from GUI also: https://bugzilla.redhat.com/show_bug.cgi?id=1284767

Version-Release number of selected component (if applicable):
RHEVM Version: 3.6.0.3-0.1.el6
VDSM: vdsm-4.17.10.1-0.el7ev
libvirt: libvirt-1.2.17-13.el7
cloud init version: 
cloud-init-0.7.6-6.el7.x86_64
cloud-utils-growpart-0.27-13.el7.noarch
VM:
rhel-guest-image-7.2 
OS: Red Hat Enterprise Linux Server release 7.2 (Maipo)

Steps to Reproduce:
1. Import rhel-guest-image as template 
2. Create VM from template via REST with cloud init.
and Start VM with use could init.
3. check network configuration


Actual results:
1. Nic exists and down
2. DNS search and servers did not set

Expected results:
1. Nic exists and up
2. DNS search and servers set on guest

Additional info:
payload on host(network configuration exist):
[root@virt-nested-vm10 latest]# cat meta_data.json 
{
  "launch_index" : "0",
  "availability_zone" : "nova",
  "network-interfaces" : "auto eth4\niface eth4 inet dhcp\n  dns-nameservers 1.2.3.4\n  dns-search foo.test.com\n",
  "name" : "cloud_init",
  "network_config" : {
    "content_path" : "/content/0000",
    "path" : "/etc/network/interfaces"
  },
  "hostname" : "cloud_init",
  "uuid" : "abc77421-d584-4051-ba57-6d5e46a0ea9b",
  "meta" : {
    "essential" : "false",
    "role" : "server",
    "dsmode" : "local"
  }
}[root@virt-nested-vm10 latest]# cat user_data 
#cloud-config
ssh_pwauth: true
timezone: NZ
disable_root: 0
output:
  all: '>> /var/log/cloud-init-output.log'
user: cloud_user
password: *********
chpasswd:
  expire: false
runcmd:
- 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list:
  ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
write_files:
-  content: test_cloud_init
   path: /tmp/test.txt



REST request to create VM:
1. Nic exists and down
 CREATE request content is --  url:https://engine-addr/api/vms body:<vm>
    <name>cloud_init_vm</name>
    <description>cloud_init_vm</description>
    <os type="rhel_6x64"/>
    <cluster href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5" id="3ff0d97b-d6bc-4278-9329-1c174f4fdcc5">
        <actions>
            <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/resetemulatedmachine" rel="resetemulatedmachine"/>
        </actions>
        <name>golden_env_mixed_1</name>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/networks" rel="networks"/>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/permissions" rel="permissions"/>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/glustervolumes" rel="glustervolumes"/>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/glusterhooks" rel="glusterhooks"/>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/affinitygroups" rel="affinitygroups"/>
        <link href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5/cpuprofiles" rel="cpuprofiles"/>
        <cpu id="Intel Conroe Family">
            <architecture>X86_64</architecture>
        </cpu>
        <data_center href="/api/datacenters/c4fda5b4-35d3-4f7a-82a6-628404b01b5d" id="c4fda5b4-35d3-4f7a-82a6-628404b01b5d"/>
        <memory_policy>
            <overcommit percent="200"/>
            <transparent_hugepages>
                <enabled>true</enabled>
            </transparent_hugepages>
        </memory_policy>
        <scheduling_policy href="/api/schedulingpolicies/b4ed2332-a7ac-4d5f-9596-99a439cb2812" id="b4ed2332-a7ac-4d5f-9596-99a439cb2812">
            <name>none</name>
            <policy>none</policy>
        </scheduling_policy>
        <version major="3" minor="6"/>
        <error_handling>
            <on_error>migrate</on_error>
        </error_handling>
        <virt_service>true</virt_service>
        <gluster_service>false</gluster_service>
        <threads_as_cores>false</threads_as_cores>
        <tunnel_migration>false</tunnel_migration>
        <trusted_service>false</trusted_service>
        <ha_reservation>false</ha_reservation>
        <optional_reason>false</optional_reason>
        <maintenance_reason_required>false</maintenance_reason_required>
        <ballooning_enabled>false</ballooning_enabled>
        <ksm>
            <enabled>true</enabled>
            <merge_across_nodes>true</merge_across_nodes>
        </ksm>
        <required_rng_sources/>
        <fencing_policy>
            <enabled>true</enabled>
            <skip_if_sd_active>
                <enabled>false</enabled>
            </skip_if_sd_active>
            <skip_if_connectivity_broken>
                <enabled>false</enabled>
                <threshold>50</threshold>
            </skip_if_connectivity_broken>
        </fencing_policy>
        <migration>
            <auto_converge>inherit</auto_converge>
            <compressed>inherit</compressed>
        </migration>
    </cluster>
    <display>
        <type>spice</type>
    </display>
    <template id="9f8f335f-f65b-4f2d-8844-415d999e2441"/>
    <initialization>
        <host_name>cloud_init</host_name>
        <timezone>NZ</timezone>
        <dns_servers>1.2.3.4</dns_servers>
        <dns_search>foo.test.com</dns_search>
        <nic_configurations>
            <nic_configuration>
                <name>eth4</name>
                <boot_protocol>dhcp</boot_protocol>
                <on_boot>true</on_boot>
            </nic_configuration>
        </nic_configurations>
        <root_password>1234</root_password>
        <custom_script>write_files:
-  content: test_cloud_init
   path: /tmp/test.txt
   permissions: '0644'</custom_script>
        <user_name>cloud_user</user_name>
    </initialization>
</vm>
Start VM:
 request content is --  url:/api/vms/e3c7f041-8a72-49a9-ac82-2b995e05a6b1/start body:<action>
    <async>false</async>
    <grace_period>
        <expiry>10</expiry>
    </grace_period>
    <use_cloud_init>true</use_cloud_init>
</action>

VM XML:
 <vm href="/api/vms/e3c7f041-8a72-49a9-ac82-2b995e05a6b1" >
......
        </actions>
        <name>cloud_init_vm</name>
        <description>cloud_init_vm</description>      
        <type>desktop</type>
        <status>
            <state>powering_up</state>
        </status>
        <memory>1073741824</memory>
        <cpu>
            <topology sockets="1" cores="1"/>
            <architecture>X86_64</architecture>
        </cpu>
        <cpu_shares>0</cpu_shares>
        <bios>
            <boot_menu>
                <enabled>false</enabled>
            </boot_menu>
        </bios>
        <os type="rhel_6x64">
            <boot dev="hd"/>
        </os>
        <cluster href="/api/clusters/3ff0d97b-d6bc-4278-9329-1c174f4fdcc5" id="3ff0d97b-d6bc-4278-9329-1c174f4fdcc5"/>
        <creation_time>2015-12-03T15:59:24.329+02:00</creation_time>
        <origin>ovirt</origin>
        <stateless>false</stateless>
        <delete_protected>false</delete_protected>
        <high_availability>
            <enabled>false</enabled>
            <priority>0</priority>
        </high_availability>
        <display>
            <type>spice</type>
            <address>10.35.161.156</address>
            <port>5900</port>
            <secure_port>5901</secure_port>
            <monitors>1</monitors>
            <single_qxl_pci>false</single_qxl_pci>
            <allow_override>false</allow_override>
            <smartcard_enabled>false</smartcard_enabled>
            <file_transfer_enabled>true</file_transfer_enabled>
            <copy_paste_enabled>true</copy_paste_enabled>
            <disconnect_action>LOCK_SCREEN</disconnect_action>
        </display>
        <sso>
            <methods>
                <method id="GUEST_AGENT"/>
            </methods>
        </sso>
        <timezone>Etc/GMT</timezone>
        <usb>
            <enabled>false</enabled>
        </usb>
        <migration_downtime>-1</migration_downtime>
        <start_paused>false</start_paused>
        <cpu_profile href="/api/cpuprofiles/54d4aa0d-cc16-4f9a-b4e7-bea08614da92" id="54d4aa0d-cc16-4f9a-b4e7-bea08614da92"/>
        <migration>
            <auto_converge>inherit</auto_converge>
            <compressed>inherit</compressed>
        </migration>
        <io>
            <threads>0</threads>
        </io>
        <time_zone>
            <name>Etc/GMT</name>
        </time_zone>
        <small_icon href="/api/icons/93b8f29b-6870-4fcb-a62a-17ba2944bf24" id="93b8f29b-6870-4fcb-a62a-17ba2944bf24"/>
        <large_icon href="/api/icons/82840f9d-f12f-41eb-a528-4fcc231de444" id="82840f9d-f12f-41eb-a528-4fcc231de444"/>
        <memory_policy>
            <guaranteed>1073741824</guaranteed>
        </memory_policy>
        <host href="/api/hosts/b140d8a7-5153-4ae6-8c86-fb1dcf42567d" id="b140d8a7-5153-4ae6-8c86-fb1dcf42567d"/>
        <template href="/api/templates/9f8f335f-f65b-4f2d-8844-415d999e2441" id="9f8f335f-f65b-4f2d-8844-415d999e2441"/>
        <start_time>2015-12-03T15:59:40.830+02:00</start_time>
        <stop_time>2015-12-03T15:59:33.399+02:00</stop_time>
        <run_once>false</run_once>
        <initialization>
            <host_name>cloud_init</host_name>
            <timezone>NZ</timezone>
            <regenerate_ssh_keys>false</regenerate_ssh_keys>
            <dns_servers>1.2.3.4</dns_servers>
            <dns_search>foo.test.com</dns_search>
            <nic_configurations>
                <nic_configuration>
                    <name>eth4</name>
                    <ip/>
                    <boot_protocol>dhcp</boot_protocol>
                    <on_boot>true</on_boot>
                </nic_configuration>
            </nic_configurations>
            <root_password>******</root_password>
            <custom_script>write_files:
-  content: test_cloud_init
   path: /tmp/test.txt
   permissions: '0644'</custom_script>
            <user_name>cloud_user</user_name>
        </initialization>
        <placement_policy/>
        <next_run_configuration_exists>false</next_run_configuration_exists>
        <numa_tune_mode>interleave</numa_tune_mode>
    </vm>
Comment 1 Michal Skrivanek 2015-12-07 13:46:15 UTC 
does it work on 7.1 host? i.e. is it really a regression in 7.2 cloud-init? you may try to downgrade cloud-init package only then
Comment 2 Israel Pinto 2015-12-07 13:59:34 UTC 
The cloud init package is part of rhel guest image.
I tested with rhel-guest-image-7.2.
Comment 3 Yaniv Kaul 2016-01-10 13:41:04 UTC 
(In reply to Israel Pinto from comment #2)
> The cloud init package is part of rhel guest image.
> I tested with rhel-guest-image-7.2.

Israel - Michal asked to downgrade either cloud init or use an older release of rhel-guest-image to rule out a possible regression in cloud-init itself. Did you try?
Comment 6 Israel Pinto 2016-01-25 14:56:17 UTC 
(In reply to Yaniv Kaul from comment #3)
> (In reply to Israel Pinto from comment #2)
> > The cloud init package is part of rhel guest image.
> > I tested with rhel-guest-image-7.2.
> 
> Israel - Michal asked to downgrade either cloud init or use an older release
> of rhel-guest-image to rule out a possible regression in cloud-init itself.
> Did you try?

I tested with rhel-guest-image 7.1 
cloud_init package: cloud-init-0.7.6-6.el7.x86_64
and I see the same problem.
See more details at: https://bugzilla.redhat.com/show_bug.cgi?id=1288819#c9
Comment 7 Israel Pinto 2016-01-25 15:00:16 UTC 
(In reply to Israel Pinto from comment #6)
> (In reply to Yaniv Kaul from comment #3)
> > (In reply to Israel Pinto from comment #2)
> > > The cloud init package is part of rhel guest image.
> > > I tested with rhel-guest-image-7.2.
> > 
> > Israel - Michal asked to downgrade either cloud init or use an older release
> > of rhel-guest-image to rule out a possible regression in cloud-init itself.
> > Did you try?
> 
> I tested with rhel-guest-image 7.1 
> cloud_init package: cloud-init-0.7.6-6.el7.x86_64
> and I see the same problem.
> See more details at: https://bugzilla.redhat.com/show_bug.cgi?id=1288819#c9

cloud init package at rhel_7.1 is:cloud-init-0.7.6-2.el7.x86_64
(copy paste mistake)
Comment 9 Tomas Jelinek 2016-01-29 07:29:52 UTC 
It is caused by an issue with CloudInit that has it's settings overridden by NetworkManager (tracked in dependent bug: https://bugzilla.redhat.com/show_bug.cgi?id=1288819). 

The workaround is to disable NetworkManager on the guest OS
Comment 10 Yaniv Kaul 2016-02-01 14:34:59 UTC 
(In reply to Tomas Jelinek from comment #9)
> It is caused by an issue with CloudInit that has it's settings overridden by
> NetworkManager (tracked in dependent bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=1288819). 
> 
> The workaround is to disable NetworkManager on the guest OS

Israel - please test the workaround. If it works, please close the bug as duplicate of bug 1288819 - nothing we can do here (not even a 'depend on').
Comment 11 Israel Pinto 2016-02-03 13:52:12 UTC 
I test also with Network Manger disable and i get the results.
NOTE:
Cloud init should sun one time on VM and if i need to connect to VM and disable the Network Manager it is solve the problem.
Comment 12 Tomas Jelinek 2016-02-11 09:14:36 UTC 
based on comment 11 disabling the NetworkManager solves the issue: e.g. this is the same issue as 1288819 => closing as duplicate

*** This bug has been marked as a duplicate of bug 1288819 ***