Created attachment 1026732 [details] log collector With oVirt Live 3.6 nightly the host persists in non operational state, not allowing to complete successfully the setup. Attaching sull sosreport.
Engine reports: 2015-05-18 16:25:01,807 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-97) [69477ea4] Host 'local_host' is set to Non-Operational, it is missing the following networks: 'ovirtmgmt' but brctl show: bridge name bridge id STP enabled interfaces ;vdsmdummy; 8000.000000000000 no ovirtmgmt 8000.0a2c82c7fb54 no dummy0 and ip addr: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: p4p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000 link/ether e8:de:27:03:20:1d brd ff:ff:ff:ff:ff:ff 3: em1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether f8:b1:56:c8:96:1f brd ff:ff:ff:ff:ff:ff inet 192.168.1.107/24 brd 192.168.1.255 scope global dynamic em1 valid_lft 85637sec preferred_lft 85637sec inet6 fe80::fab1:56ff:fec8:961f/64 scope link valid_lft forever preferred_lft forever 4: bond0: <BROADCAST,MULTICAST,MASTER> mtu 1500 qdisc noop state DOWN link/ether ee:87:9b:7d:5a:c2 brd ff:ff:ff:ff:ff:ff 5: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovirtmgmt state UNKNOWN link/ether 0a:2c:82:c7:fb:54 brd ff:ff:ff:ff:ff:ff 6: ovirtmgmt: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP link/ether 0a:2c:82:c7:fb:54 brd ff:ff:ff:ff:ff:ff inet 10.0.0.1/24 brd 10.0.0.255 scope global ovirtmgmt valid_lft forever preferred_lft forever inet6 fe80::82c:82ff:fec7:fb54/64 scope link valid_lft forever preferred_lft forever 7: ;vdsmdummy;: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether b6:80:44:32:1b:b9 brd ff:ff:ff:ff:ff:ff
The ovirtmgmt bridge exists, but Vdsm does not report the ovirtmgt /network/. networks = {} this can happen if the the network vdsm-ovirtmgmt is not defined in libvirt. Can you provide the output of `virsh -r net-list` to confirm this? How does ovirt-live create the ovirtmgmt network? This regression might be due to https://gerrit.ovirt.org/40568 net: Remove 3.0.0 network upgrade I am guessing that ovirt-live created the bridge and depended upon vdsm upgrading it to a fully-fledged network. If this is the case, the network should be created by accessing the vdsm API (or by the very least, by defining the proper libvirt network on top of an ovirtmgmt bridge).
(In reply to Dan Kenigsberg from comment #2) > The ovirtmgmt bridge exists, but Vdsm does not report the ovirtmgt /network/. > > networks = {} > > this can happen if the the network vdsm-ovirtmgmt is not defined in libvirt. > > Can you provide the output of `virsh -r net-list` to confirm this? # LC_ALL=C virsh -r net-list Name State Autostart Persistent ---------------------------------------------------------- ;vdsmdummy; active no no > How does ovirt-live create the ovirtmgmt network? This regression might be > due to https://gerrit.ovirt.org/40568 net: Remove 3.0.0 network upgrade > I am guessing that ovirt-live created the bridge and depended upon vdsm > upgrading it to a fully-fledged network. If this is the case, the network > should be created by accessing the vdsm API (or by the very least, by > defining the proper libvirt network on top of an ovirtmgmt bridge). You're right. It created the bridge during the boot creating: - ifcfg-dummy0 - ifcfg-ovirtmgmt - route-ovirtmgmt - rule-ovirtmgmt So if I understand correctly, this is not supported anymore in general, so it's not a EL7 specific issue.
In order to re-create the missing libvirt network: >>> import sys >>> sys.path.append("/usr/share/vdsm") >>> from network.configurators import ifcfg >>> configWriter = ifcfg.ConfigWriter() >>> configWriter.createLibvirtNetwork(network='ovirtmgmt', bridged=True, skipBackup=True) # LC_ALL=C virsh -r net-list Name State Autostart Persistent ---------------------------------------------------------- ;vdsmdummy; active no no vdsm-ovirtmgmt active yes yes Dan, is above network.configurators.ifcfg deprecated or still supported in 3.6?
(In reply to Sandro Bonazzola from comment #3) > So if I understand correctly, this is not supported anymore in general, so > it's not a EL7 specific issue. el6 is not supported on 3.6 onward. So this problem is el7 (and Fedora) only. (In reply to Sandro Bonazzola from comment #4) > > Dan, is above network.configurators.ifcfg deprecated or still supported in > 3.6? the ifcfg configurator is currently broken on master, I'm afraid (bug 1223457). But it is a serious bug that would be fixed asap. ifcfg is going to be supported in 3.x. However, you are talking here about something else: about creating a vdsm network outside vdsm. THIS is something that should be avoided, if possible. Is it possible to use Vdsm's setupNetwork API in order to configure the network? If so, please do that. If it is impossible (due to bootstrapping of oVirt Live), one needs to mimick vdsm's creation of libvirt's network e.g virsh net-define with <network> <name>vdsm-ovirtmgmt</name> <uuid>6970749c-6401-4b7f-86f3-f1cea7291482</uuid> <forward mode='bridge'/> <bridge name='dummy0'/> </network>
The correct way to bootstrap vdsm networks is to create them via vdsm api. This will be supported for a long time and will not break easily. Running this will mimic the creation of such a network from the engine: (python) from vdsm import vdscli NOCHK = {'connectivityCheck': False} NET = 'ovirtmgmt' c = vdscli.connect() c.setupNetworks({NET: {'bootproto':'dhcp', 'nic':'dummy0', 'STP': 'no', 'bridged': True, 'mtu': '1500'}}, {}, NOCHK) c.setSafeNetworkConfig()
This requires VDSM to be configured and on all-in-one this happens when running host-deploy. Need to verify if we can use above code.
This bug report has Keywords: Regression or TestBlocker. Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Verified with: ovirt-live-el7-3.6.1_rc3.iso Host is successfully installed. # virsh -r net-list Name State Autostart Persistent ---------------------------------------------------------- ;vdsmdummy; active no no vdsm-ovirtmgmt active yes yes
oVirt 3.6.0 has been released and the bz verified, moving to closed current release.