Created attachment 1075775 [details] logs from ppc64le host Description of problem: cannot add ppc64le host into cluster / vdsm-network.service: control process exited, code=exited status=1. ~~~ Failed to install Host ibm-p8-rhevm-hv-02. Failed to execute stage 'Closing up': Command '/bin/systemctl' failed to execute. ~~~ ~~~ 2015-09-22 06:33:43 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:940 execute-output: ('/bin/systemctl', 'start', 'vdsmd.service') stderr: A dependency job for vdsmd.service failed. See 'journalctl -xe' for details. 2015-09-22 06:33:43 DEBUG otopi.context context._executeMethod:155 method exception Traceback (most recent call last): File "/tmp/ovirt-5nF8L2TvA1/pythonlib/otopi/context.py", line 145, in _executeMethod method['method']() File "/tmp/ovirt-5nF8L2TvA1/otopi-plugins/ovirt-host-deploy/vdsm/packages.py", line 224, in _start self.services.state('vdsmd', True) File "/tmp/ovirt-5nF8L2TvA1/otopi-plugins/otopi/services/systemd.py", line 141, in state 'start' if state else 'stop' File "/tmp/ovirt-5nF8L2TvA1/otopi-plugins/otopi/services/systemd.py", line 80, in _executeServiceCommand raiseOnError=raiseOnError File "/tmp/ovirt-5nF8L2TvA1/pythonlib/otopi/plugin.py", line 945, in execute command=args[0], RuntimeError: Command '/bin/systemctl' failed to execute ~~~ ~~~ -- Logs begin at Tue 2015-09-22 04:58:13 EDT, end at Tue 2015-09-22 07:00:01 EDT. -- Sep 22 06:33:43 ibm-p8-rhevm-hv-02.klab.eng.bos.redhat.com systemd[1]: Starting Virtual Desktop Server Manager network restoration... -- Subject: Unit vdsm-network.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit vdsm-network.service has begun starting up. Sep 22 06:33:43 ibm-p8-rhevm-hv-02.klab.eng.bos.redhat.com systemd[1]: vdsm-network.service: control process exited, code=exited status=1 Sep 22 06:33:43 ibm-p8-rhevm-hv-02.klab.eng.bos.redhat.com systemd[1]: Failed to start Virtual Desktop Server Manager network restoration. -- Subject: Unit vdsm-network.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit vdsm-network.service has failed. -- ~~~ Version-Release number of selected component (if applicable): sanlock-3.2.4-1.el7.ppc64le mom-0.5.0-1.el7ev.noarch vdsm-4.17.7-1.el7ev.noarch libvirt-python-1.2.17-2.el7.ppc64le rhevm-backend-3.6.0-0.16.master.el6.noarch How reproducible: 100% Steps to Reproduce: 1. install rhel7.2 with following repos: Repo-baseurl : http://download.englab.brq.redhat.com/pub/rhel/rel-eng/RHEL-7.2-20150917.0/compose/Server/ppc64le/os/ Repo-baseurl : http://download.englab.brq.redhat.com/pub/rhel/rel-eng/RHEL-7.2-20150917.0/compose/Server-optional/ppc64le/os/ Repo-baseurl : http://download.lab.bos.redhat.com/rel-eng/repos/rhevh-rhel-7.2-candidate/ppc64le/ Repo-baseurl : http://10.34.63.204/pub/mirror/rhevm/3.6/3.6.0-13/el7Server Repo-baseurl : http://download.lab.bos.redhat.com/rel-eng/repos/rhel-7.2-candidate/ppc64le/ ^^^ last one needed as there's no sanlock etc in rhevh-rhel-7.2-candidate 2. yum install vdsm 3. add into engine via admin portal Actual results: failure probably because of vdsm-network Expected results: should work Additional info: for petitboot Device: (*) Specify paths/URLs manually Kernel: http://download.eng.bos.redhat.com/rel-eng/RHEL-7.2-20150917.0/compose/Server/ppc64le/os/ppc/ppc64/vmlinuz Initrd: http://download.eng.bos.redhat.com/rel-eng/RHEL-7.2-20150917.0/compose/Server/ppc64le/os/ppc/ppc64/initrd.img Device tree: <empty> Boot arguments: ro ip=dhcp inst.text inst.headless inst.repo=http://download.eng.bos.redhat.com/rel-eng/RHEL-7.2-20150917.0/compose/Server/ppc64le/os/ inst.selinux=0
we can find the same issue when install vdsm on RHEL-7.2-20150917.0. it will block the virt-who vdsm mode testing.
This bug could related to [1] which occurs on x86_64 [1] https://bugzilla.redhat.com/show_bug.cgi?format=multiple&id=1264667 upgrade.log contains: MainThread::ERROR::2015-09-22 06:33:43,723::upgrade::94::upgrade::(apply_upgrade) Failed to run upgrade-unified-persistence Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/tool/upgrade.py", line 92, in apply_upgrade upgrade.run(ns, args) File "/usr/lib/python2.7/site-packages/vdsm/tool/unified_persistence.py", line 193, in run run() File "/usr/lib/python2.7/site-packages/vdsm/tool/unified_persistence.py", line 44, in run networks, bondings = _getNetInfo() File "/usr/lib/python2.7/site-packages/vdsm/tool/unified_persistence.py", line 131, in _getNetInfo networks = _processNetworks(netinfo) File "/usr/lib/python2.7/site-packages/vdsm/tool/unified_persistence.py", line 53, in _processNetworks defaultGateway = getDefaultGateway() File "/usr/lib/python2.7/site-packages/vdsm/netinfo.py", line 319, in getDefaultGateway output = routeShowGateways('main') File "/usr/lib/python2.7/site-packages/vdsm/ipwrapper.py", line 537, in routeShowGateways return _execCmd(command) File "/usr/lib/python2.7/site-packages/vdsm/ipwrapper.py", line 526, in _execCmd returnCode, output, error = execCmd(command) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 664, in execCmd deathSignal=deathSignal, childUmask=childUmask) File "/usr/lib64/python2.7/site-packages/cpopen/__init__.py", line 52, in __init__ stderr=stderr) File "/usr/lib64/python2.7/subprocess.py", line 712, in __init__ restore_sigpipe=restore_sigpipe) TypeError: _execute_child_v276() got an unexpected keyword argument 'restore_sigpipe'
Jiri, which version of cpopen was used? It's possible, that [1] can fix it (or broke it). [1] https://gerrit.ovirt.org/#/c/46300/9
it's a duplicated issue with https://bugzilla.redhat.com/show_bug.cgi?id=1264667 According to Oved Ourfali's comment, I downgrade the python package from 2.7.5-33 to 2.7.5-32, vdsm-tool configure --force can be executed successfully, vdsm also can be restarted normally. so maybe it's not a ovirt issue, because we didn't install any ovirt packages, we just prepare a RHEL-7.2-20150917.0-Server os, and then add http://bob.eng.lab.tlv.redhat.com/builds/vt16/el7/ as the yum repo for installing vdsm packages. it's also not a python-cpopen issue, because this package is the same version in RHEL-7.2-20150910.2 and RHEL-7.2-20150917.0. RHEL-7.2-20150910.2 [PASS] => python-2.7.5-32 => python-cpopen-1.3-5 => vdsm-4.16.21-1.el7ev RHEL-7.2-20150917.0 [FAILED] => python-2.7.5-33 => python-cpopen-1.3-5 => vdsm-4.16.21-1.el7ev
Can you please try to recreate with the updated cpopen package and python version? Please close as duplicate if this works.
Please see #5 as ppc64le hosts are under your control for now. Thx.
I not encounter such problem with: vdsm-4.17.8-1.el7ev.noarch python-cpopen-1.4-0.el7ev.ppc64le python-2.7.5-33.el7.ppc64le
Closing as a duplicate. *** This bug has been marked as a duplicate of bug 1264667 ***