Description of problem:

Upgrading RHV-H restores some ifcfg files that have been previously removed. It mixes current with previous network configuration in the host.

See the details and reproduction:

1. Install RHV-H host with some previous version, 2 versions back.

# nodectl info | grep current
current_layer: rhvh-4.1-0.20170706.0+1

2. Add the host to the engine, configure 1 extra network (network-2, using vlan 50) via Hosts->Networks->Setup Host Networks

# ll /etc/sysconfig/network-scripts/ifcfg-*
-rw-rw-r--. 1 root root 134 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-eth0
-rw-rw-r--. 1 root root 146 Oct 12 14:12 /etc/sysconfig/network-scripts/ifcfg-eth0.50
-rw-r--r--. 1 root root 254 Sep 12  2016 /etc/sysconfig/network-scripts/ifcfg-lo
-rw-rw-r--. 1 root root 150 Oct 12 14:12 /etc/sysconfig/network-scripts/ifcfg-network-2
-rw-rw-r--. 1 root root 201 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt

# ll /var/lib/vdsm/persistence/netconf/nets/
total 8
-rw-r--r--. 1 root root 224 Oct 12 14:12 network-2
-rw-r--r--. 1 root root 257 Oct 12 14:12 ovirtmgmt

2. Upgrade host, but not to very lastest (we need one more upgrade)

# yum install redhat-virtualization-host-image-update-4.1-20170808.0.el7_4
# reboot
# nodectl info | grep current
current_layer: rhvh-4.1-0.20170808.0+1

3. Remove the extra network from the host via Hosts->Networks->Setup Host Networks

# ll /etc/sysconfig/network-scripts/ifcfg-*
-rw-rw-r--. 1 root root 134 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-eth0
-rw-r--r--. 1 root root 254 May  3 20:17 /etc/sysconfig/network-scripts/ifcfg-lo
-rw-rw-r--. 1 root root 201 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt

# ll /var/lib/vdsm/persistence/netconf/nets/
total 4
-rw-r--r--. 1 root root 257 Oct 12 15:15 ovirtmgmt

4. Upgrade host to very latest, then check networks. See who is back from the dead:

# nodectl info | grep current
current_layer: rhvh-4.1-0.20171002.0+1

# ll /etc/sysconfig/network-scripts/ifcfg-*
-rw-rw-r--. 1 root root 134 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-eth0
-rw-rw-r--. 1 root root 146 Oct 12 15:33 /etc/sysconfig/network-scripts/ifcfg-eth0.50
-rw-r--r--. 1 root root 254 May  3 20:17 /etc/sysconfig/network-scripts/ifcfg-lo
-rw-rw-r--. 1 root root 150 Oct 12 15:33 /etc/sysconfig/network-scripts/ifcfg-network-2
-rw-rw-r--. 1 root root 201 Oct 12 14:11 /etc/sysconfig/network-scripts/ifcfg-ovirtmgmt

But persitence looks fine:

# ll /var/lib/vdsm/persistence/netconf/nets/
total 4
-rw-r--r--. 1 root root 257 Oct 12 15:15 ovirtmgmt

Still, this is enough to break host network config on reboot, it case it brings back some conflicting configurations.

This is the weird part (to me). Why it copies these files from the rhvh-4.1-0.20170808 if they should not even exist there as per step 3?

[DEBUG] (migrate_etc) Migrating etc (<LV 'rhvh_gsslab-24-125/rhvh-4.1-0.20170808.0+1' /> -> <LV 'rhvh_gsslab-24-125/rhvh-4.1-0.20171002.0+1' />)
[DEBUG] (migrate_etc) Calling binary: (['mount', u'/dev/rhvh_gsslab-24-125/rhvh-4.1-0.20171002.0+1', u'/tmp/mnt.qdl2d'],) {}
[DEBUG] (migrate_etc) Calling binary: (['mount', u'/dev/rhvh_gsslab-24-125/rhvh-4.1-0.20170808.0+1', u'/tmp/mnt.kG22U'],) {}

[DEBUG] (migrate_etc) Calling: (['cp', '-a', '-r', u'/tmp/mnt.kG22U///etc/sysconfig/network-scripts/ifcfg-eth0.50', u'/tmp/mnt.qdl2d///etc/sysconfig/network-scripts/ifcfg-eth0.50']
[DEBUG] (migrate_etc) Calling: (['cp', '-a', '-r', u'/tmp/mnt.kG22U///etc/sysconfig/network-scripts/ifcfg-network-2', u'/tmp/mnt.qdl2d///etc/sysconfig/network-scripts/ifcfg-network-2'],)

Version-Release number of selected component (if applicable):
redhat-virtualization-host-image-update-4.1-20171002

How reproducible:
100%, as above.