Bug 1296614 - ovirtmgmt bridge is not configured on newly deployed host
ovirtmgmt bridge is not configured on newly deployed host
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Network (Show other bugs)
3.6.2
Unspecified Unspecified
unspecified Severity high (vote)
: ovirt-3.6.2
: 3.6.2.5
Assigned To: Dan Kenigsberg
Meni Yakove
:
: 1296613 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-07 11:35 EST by Martin Perina
Modified: 2016-02-18 06:05 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-18 06:05:42 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
danken: ovirt‑3.6.z?
rule-engine: planning_ack?
danken: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine and vdsm logs (342.41 KB, application/x-gzip)
2016-01-07 11:35 EST, Martin Perina
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 51577 master MERGED engine: dhcp network should has ip info 2016-01-10 10:40 EST
oVirt gerrit 51601 ovirt-engine-3.6 MERGED engine: dhcp network should has ip info 2016-01-10 11:30 EST
oVirt gerrit 51602 ovirt-engine-3.6.2 MERGED engine: dhcp network should has ip info 2016-01-10 11:32 EST

  None (edit)
Description Martin Perina 2016-01-07 11:35:03 EST
Created attachment 1112496 [details]
engine and vdsm logs

Description of problem:

When I try to install new host using oVirt UI or REST, ovirtmgmt bridge is not configured during host deploy and although installation finished successfully, host status is changed to NonOperational with error: 

 Host does not comply with cluster networks, following networks are missing on host: 'ovirtmgmt'


Both engine and host uses latest Centos 7, RPM installation behaves the same as dev env.

Version-Release number of selected component (if applicable):

ovirt-engine master
vdsm master

How reproducible:

100%

Steps to Reproduce:
1. Install new host into engine
2.
3.

Actual results:

Bridge ovirtmgt is not configured on host -> host is marked as Non Operational 

Expected results:

Host is configured properly and in status Up after successful install

Additional info:
Comment 1 Dan Kenigsberg 2016-01-08 03:27:03 EST
Martin, as a workaround, can you setup an ovitmgmt network manually?
Before you do that, could you run `virsh -r net-list` on your host?


vdsm-4.17.999-442.gitec3bd67.el7.centos

2016-01-07 17:20:33,636 INFO  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-1) [718a970d] Lock Acquired to object 'EngineLock:{exclusiveLocks='[72962378-0be1-4fc9-a785-a607aa9d3a1a=<HOST_NETWORK, ACTION_TYPE_FAILED_SETUP_NETWORKS_IN_PROGRESS>]', sharedLocks='null'}'
2016-01-07 17:20:33,702 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-1) [718a970d] Validation of action 'HostSetupNetworks' failed for user admin@internal. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORK_ATTACHMENT_IP_CONFIGURATION_INCOMPATIBLE_BOOT_PROTOCOL_AND_IP_ADDRESS_DETAILS,$networkName ovirtmgmt,$interfaceName enp5s0f0,$BootProtocol dhcp
2016-01-07 17:20:33,702 INFO  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-1) [718a970d] Lock freed to object 'EngineLock:{exclusiveLocks='[72962378-0be1-4fc9-a785-a607aa9d3a1a=<HOST_NETWORK, ACTION_TYPE_FAILED_SETUP_NETWORKS_IN_PROGRESS>]', sharedLocks='null'}'
2016-01-07 17:20:33,709 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-1) [718a970d] Failed to configure management network: Failed to configure management network on host dev-18 due to setup networks failure.

Vdsm Caps has

'enp5s0f0': {'addr': '10.34.63.178', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ipv6addrs': ['2620:52:0:223c:82c1:6eff:fe6b:7e6c/64', 'fe80::82c1:6eff:fe6b:7e6c/64'], 'mtu': '1500', 'dhcpv4': True, 'netmask': '255.255.252.0', 'dhcpv6': False, 'ipv4addrs': ['10.34.63.178/22'], 'cfg': {'IPV6INIT': 'no', 'NM_CONTROLLED': 'no', 'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'HWADDR': '80:c1:6e:6b:7e:6c', 'BOOTPROTO': 'dhcp', 'DEVICE': 'enp5s0f0', 'ONBOOT': 'yes'}, 'hwaddr': '80:c1:6e:6b:7e:6c', 'speed': 1000, 'gateway': '10.34.63.254'}

Can you see the incompatibility, Martin Mucha?
Comment 2 Martin Perina 2016-01-08 04:04:59 EST
Here's the output of 'virsh -r net-list':

Name                 State      Autostart     Persistent
----------------------------------------------------------
 ;vdsmdummy;          active     no            no
 vdsm-ovirtmgmt       active     yes           yes


After that I did manual ovirtmgmt setup, restarted the host and activated it and everything now works as expected.
Comment 3 Martin Mucha 2016-01-08 04:36:12 EST
on latest master on my environment, I dropped db, rebuild, ran engine setup and then added my host using rest. Host is up after this. ovirtmgmt bridge seems to be present

brctl show
bridge name	bridge id		STP enabled	interfaces
;vdsmdummy;		8000.000000000000	no		
network		8000.56a495973726	no		dummy_40
ovirtmgmt		8000.bc305be60b8f	no		em1

virsh -r net-list
 Name                 State      Autostart     Persistent
----------------------------------------------------------
 ;vdsmdummy;          active     no            no
 vdsm-network         active     yes           yes
 vdsm-ovirtmgmt       active     yes           yes


//since there's also vdsm-network, it's possible, that ovirtmgmt is configured from previous installation?
Comment 4 Simone Tiraboschi 2016-01-09 08:10:21 EST
The issue could be here (seen on a different deploy):

[root@integ-test-engine03 ~]# grep CanDoAction /var/log/ovirt-engine/engine.log 
2016-01-08 17:36:37,923 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-3) [7d0e6bb] CanDoAction of action 'HostSetupNetworks' failed for user admin@internal. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORK_ATTACHMENT_IP_CONFIGURATION_INCOMPATIBLE_BOOT_PROTOCOL_AND_IP_ADDRESS_DETAILS,$networkName ovirtmgmt,$interfaceName enp2s0f0,$BootProtocol dhcp
2016-01-08 17:41:46,800 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-15) [7a2edb80] CanDoAction of action 'HostSetupNetworks' failed for user admin@internal. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORK_ATTACHMENT_IP_CONFIGURATION_INCOMPATIBLE_BOOT_PROTOCOL_AND_IP_ADDRESS_DETAILS,$networkName ovirtmgmt,$interfaceName enp2s0f0,$BootProtocol dhcp
2016-01-08 17:48:51,720 WARN  [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (org.ovirt.thread.pool-7-thread-9) [6b60a115] CanDoAction of action 'HostSetupNetworks' failed for user admin@internal. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,NETWORK_ATTACHMENT_IP_CONFIGURATION_INCOMPATIBLE_BOOT_PROTOCOL_AND_IP_ADDRESS_DETAILS,$networkName ovirtmgmt,$interfaceName enp2s0f0,$BootProtocol dhcp
2016-01-08 18:03:49,422 WARN  [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-3) [] CanDoAction of action 'LoginAdminUser' failed for user admin@internal. Reasons: USER_FAILED_TO_AUTHENTICATE_WRONG_USERNAME_OR_PASSWORD
Comment 5 Yevgeny Zaspitsky 2016-01-10 02:56:43 EST
Martin,
Was your engine environment upgraded from 3.5?
Can the engine DB be restored for us to check its state?
Comment 6 Martin Perina 2016-01-11 03:04:49 EST
*** Bug 1296613 has been marked as a duplicate of this bug. ***
Comment 7 Martin Perina 2016-01-11 04:09:26 EST
(In reply to Yevgeny Zaspitsky from comment #5)
> Martin,
> Was your engine environment upgraded from 3.5?
> Can the engine DB be restored for us to check its state?

No, it was clean installation, I just created new DC and Cluster (both with default values) and tried to add a host.

Provided patch fixed the issue for real host, I will try it also with Lago and post results.

Thanks
Comment 8 Michael Burman 2016-01-21 05:43:58 EST
Verified on - 3.6.2.6-0.1.el6 and vdsm-4.17.18-0.el7ev

Note You need to log in before you can comment on or make changes to this bug.