Created attachment 547623 [details]
ovirt.log

Problem results from not cleaning up files and persistence when a network is reconfigured and multipath holding the files busy.

Created attachment 551965 [details]
Patch

Note:  I could only reproduce this by booting iso, configuring, installing to disk, and on boot from disk, changing the network device.

Testing:

boot iso
set password
configure networking (eth0)
install locally and reboot
boot from disk
configure network (eth1)
check ifconfig output -- should see only breth1, lo, eth1

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
In certain situations, network reconfiguration would fail to cleanup after itself resulting multiple bridges and incorrectly configured interfaces.  The previous network files are now explicitly cleaned up correctly.

Verified on 5.8-20120111.0:
1. configure eth0 during install.
2. install locally and reboot.
3. login and check ifconfig output, see breth0, eth0.
4. ifup eth1.
5. configure eth1 in "setup".
6. check ifconfig output, only see breth1, eth1.

Re-open this bug, change network deleted the loopback device lo, which will cause vdsmd failed to run:

[root@unused ~]# grep -A3 -B3 down /etc/init.d/vdsmd
     start)
        if ! ifconfig lo | grep -q UP;
        then
            log_failure_msg "VDSMD: lo interface is down, can't run !"
            echo "VDSMD: lo interface is down, can't run !" > /dev/kmsg
            exit 1
        fi


Should keep lo device when changed network.

diff --git a/scripts/ovirt-config-networking b/scripts/ovirt-config-networking
index 0d57b8c..c543605 100755
--- a/scripts/ovirt-config-networking
+++ b/scripts/ovirt-config-networking
@@ -491,9 +491,9 @@ else
                     if [ -n "${NIC}" ] && [[ "${NICS}" =~ "${NIC}" ]]; then
                         # ensure all network devices and dhclient processes are stopped
                         service network stop; killall dhclient
-                        remove_config /etc/sysconfig/network-scripts/ifcfg-*
+                        remove_config /etc/sysconfig/network-scripts/ifcfg-*eth*
                         service multipathd restart 2>&1 >/dev/null
-                        rm -f /etc/sysconfig/network-scripts/ifcfg-*
+                        rm -f /etc/sysconfig/network-scripts/ifcfg-*eth*
                         configure_interface $NIC
                     else
                         printf "\nInvalid option selected.\n"

verified on 5.8-20120130.0 build:
the loopback device lo is reserved after change network, and rhevh itself show up in RHEVM correctly(vdsmd is running).

    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-In certain situations, network reconfiguration would fail to cleanup after itself resulting multiple bridges and incorrectly configured interfaces.  The previous network files are now explicitly cleaned up correctly.+In certain situations, network reconfiguration would fail to clean up before loading the new configuration. This resulted  in multiple bridges and incorrectly configured interfaces being created. The existing network configuration files are now explicitly removed prior to loading the new configuration.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0168.html