Created attachment 1173362 [details] Logs Description of problem: Add host failed - failed to configure ovirtmgmt network on host. Seems like it caused by the fix for the brain split bug. Version-Release number of selected component (if applicable): 3.6.7.5-0.1.el6.noarch vdsm-4.17.31-0.el7ev.noarch Steps to Reproduce: 1. Add host to rhev-m 3.6.7 Actual results: Add host failed Additional info: Seems to be related to the brain split bug fix.
This is duplicate of BZ #1348103
Yes, though it's new in 3.6.7 with JSON-RPC
I was able to reproduce issue on RHEV-H & RHEL7 hosts as following: 1. Both hosts added to engine - no problem 2. Put hosts to maintanance 3. Re-provision hosts OS from same source ( PXE ) 4. Edit hosts to re-fetch new fingerprint of server 5. Reinstall hosts - button in engine 6. Hosts failed to reinstall - Packages & versions: rhevm-3.6.7-6 vdsm-jsonrpc-java-1.1.12-1.el6ev.noarch Hosts: RHEV-H: 7.2-20160627.3.el7ev RHEL7: vdsm 4.17.31-0.el7ev
*** Bug 1350718 has been marked as a duplicate of this bug. ***
*** Bug 1351226 has been marked as a duplicate of this bug. ***
*** Bug 1329166 has been marked as a duplicate of this bug. ***
*** Bug 1352859 has been marked as a duplicate of this bug. ***
failed with: rhevm-3.6.8-0.1.el6.noarch vdsm-4.17.33-1.el7ev.noarch this time it failed adding new host
In this case I see in the vdsm.log: jsonrpc.Executor/4::ERROR::2016-07-14 13:17:37,189::API::1652::vds::(_rollback) connectivity check failed Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1650, in _rollback yield rollbackCtx File "/usr/share/vdsm/API.py", line 1502, in setupNetworks supervdsm.getProxy().setupNetworks(networks, bondings, options) File "/usr/share/vdsm/supervdsm.py", line 50, in __call__ return callMethod() File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda> **kwargs) File "<string>", line 2, in setupNetworks File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod raise convert_to_error(kind, result) ConfigNetworkError: (10, 'connectivity check failed') Edward - can you take a look?
Moving to assigned based on comment #19
In general, the error 'connectivity check failed' means that VDSM successfully applied the last setup request, however, it can no longer 'hear' the Engine so it fails and reverts back to the last known configuration. From the vdsm.log: 'nics': {'eth0': {'addr': '10.34.60.4', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ipv6addrs': ['2620:52:0:223c:21a:4aff:fed0:3009/64', 'fe80::21a:4aff:fed0:3009/64'], 'mtu': '1500', 'dhcpv4': True, 'netmask': '255.255.252.0', 'dhcpv6': False, 'ipv4addrs': ['10.34.60.4/22'], 'cfg': {'PEERROUTES': 'yes', 'IPV6INIT': 'yes', 'NAME': 'eth0', 'IPADDR': '10.34.60.4', 'NETBOOT': 'yes', 'IPV6_PEERDNS': 'yes', 'DEFROUTE': 'yes', 'PEERDNS': 'yes', 'IPV4_FAILURE_FATAL': 'no', 'IPV6_AUTOCONF': 'yes', 'PREFIX': '22', 'BOOTPROTO': 'static', 'IPV6_DEFROUTE': 'yes', 'GATEWAY': '10.34.63.254', 'HWADDR': '00:1A:4A:D0:30:09', 'IPV6_FAILURE_FATAL': 'no', 'DNS1': '10.34.63.229', 'IPV6_PEERROUTES': 'yes', 'TYPE': 'Ethernet', 'ONBOOT': 'yes', 'UUID': 'a11ac764-5abb-4a6d-9892-5ce82b83e12e'} From some reason BOOTPROTO is set with 'static' instead of 'none'. And dhcpv4 is reported as True. So we have a collision here. How is the host eth0 nic configured before it is added? Is it DHCP or static? If it was DHCP originally, we need to understand if dhclient request has been answered and if the correct address has been re-assign to it. Please provide supervdsm.log to look into it further.
Regardless, I've verified it on Nelly's env, when vdsm-jsonrpc-java 1.1.12 is installed. So moving back to ON_QA, and if needed open another bug on network on the specific issue.
nic eth0 was manually configured to static in ifcfg-eth0 before installing vdsm 'static' is completely valid bootproto and it can be used for readability... in fact, you either put 'dhcp' in ifcfg for dhcp, or anything else for static IP Anyway, i tried it with bootproto 'none' and i'm still getting the same error (with vdsm-jsonrpc-java 1.1.12) I'm not saying this is not an environment issue, but I can't verify this currently. I don't think creating a new bug for with the same title for the same version, with the same reproduction steps and the same error message is a good idea.
So I'm moving that to Network. This is different than the original errors in the log, although the title is identical.
'static' may be valid for the ifcfg scripts end result, but not nessesery how VDSM interprets it. Documentation states what values it should get. But I am not sure if this is the problem The problem is that a static IP was set, but VDSM detects it as dynamic (dhcp). Please collect this info before adding the host: - 'ip addr' - A caps report (vdsClient 0 -s getVdsCaps)
(In reply to movciari from comment #24) > I don't think creating a new bug for with the same title for the > same version, with the same reproduction steps and the same error message is > a good idea. Michal, if the underlying problem is different, and resolving team is different, it is better to modify the existing summary line to be more specific, and open an fresh bug.
Verified on - 3.6.8-0.1.el6 with vdsm-4.17.33-1.el7ev.noarch and vdsm-jsonrpc-java-1.1.12-1.el6ev.noarch. The verification done only for the origin report.
*** Bug 1357615 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-1509.html