Description of problem: After enabling ovirt-4.2-pre repo on a ppc64le 4.1 CentOS host and starting an upgrade from the UI, the host fails to activate due to missing openvswitch binaries and stays in Initializing state Version-Release number of selected component (if applicable): vdsm-4.20.8-1.el7.centos.ppc64le How reproducible: reproduced on ppc64le so far Steps to Reproduce: 1. enable 4.2-pre repo on an existing 4.1 host yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release42-pre.rpm 2. in the admin portal (4.1.7) select installation -> upgrade Actual results: Host stuck in Initializing state with error: command GetCapabilitiesVDS failed: Internal JSON-RPC error: {'reason': "[Errno 2] No such file or directory: '/usr/share/openvswitch/scripts/ovs-ctl'"} Expected results: Host upgraded successfully
Why is openvswitch missing, Evgeni? Do you know?
Hi Dan, I'm not sure but it's not installed on this ppc system and never was. Will attach the OTOPI log in a second.
Did some research: the system was delpoyed a year ago and was initially installed from master (which is similar to 4.1-pre) and later upgraded to 4.1.6 using plain yum. Not back then not now did openvswitch get installed, nor is it a dependency for VDSM. In my opinion, if a VDSM does not depend on openvswitch, it should not fail when that is missing. Alternatively, it should be added as a dependency so that it's installed when upgrading from previous versions.
Checked another ppc64le hypervisor that we have and it's also lacking openvswitch while all x86_64 hosts seem to have it so this may be specific for this platform. Snippet from vdsm.log for reference: 2017-11-28 16:10:39,450+0000 ERROR (jsonrpc/7) [jsonrpc.JsonRpcServer] Internal server error (__init__:611) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request res = method(**params) File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, in _dynamicMethod result = fn(*methodArgs) File "<string>", line 2, in getCapabilities File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1317, in getCapabilities c = caps.get() File "/usr/lib/python2.7/site-packages/vdsm/host/caps.py", line 168, in get net_caps = supervdsm.getProxy().network_caps() File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__ return callMethod() File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda> **kwargs) File "<string>", line 2, in network_caps File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod raise convert_to_error(kind, result) OSError: [Errno 2] No such file or directory: '/usr/share/openvswitch/scripts/ovs-ctl' Installing openvswitch manually allows the host to immediately get activated and fixes the issue. Still, the upgrade (even using yum) should do that automatically.
We've fixed the immediate problem of Vdsm not responsing if openvswitch is not installed. Would you file another bug for "require openvswitch on ppc, too"?
Does v4.20.9 equals vdsm-4.20.8-79.gitd2a37c7.el7.centos.x86_64 ??
$ git describe d2a37c7 v4.20.9 yes
Verified on - vdsm-4.20.8-79.gitd2a37c7.el7.centos.x86_64 Vdsm is responding if openvswitch is not installed.
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017. Since the problem described in this bug report should be resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report.