Description of problem: update vdsm Version-Release number of selected component: vdsm-python-4.12.0-18.git78903dd.fc19 Additional info: reporter: libreport-2.1.6 cmdline: /usr/bin/python /usr/bin/vdsm-tool service-disable ksmtuned executable: /usr/bin/vdsm-tool kernel: 3.10.4-300.fc19.x86_64 runlevel: unknown type: Python uid: 0 Truncated backtrace: service.py:353:_runAlts:ServiceNotExistError: ServiceNotExistError: Tried all alternatives but failed: ServiceNotExistError: ksmtuned is not native systemctl service ServiceNotExistError: ksmtuned is not a SysV service Traceback (most recent call last): File "/usr/bin/vdsm-tool", line 143, in <module> sys.exit(main()) File "/usr/bin/vdsm-tool", line 140, in main return tool_command[cmd]["command"](*args[1:]) File "/usr/lib64/python2.7/site-packages/vdsm/tool/service.py", line 405, in service_disable return _runAlts(_srvDisableAlts, srvName) File "/usr/lib64/python2.7/site-packages/vdsm/tool/service.py", line 353, in _runAlts ('\n'.join(str(e) for errs in errors.values() for e in errs))) ServiceNotExistError: ServiceNotExistError: Tried all alternatives but failed: ServiceNotExistError: ksmtuned is not native systemctl service ServiceNotExistError: ksmtuned is not a SysV service Local variables in innermost frame: errors: defaultdict(<type 'list'>, {'_systemctlDisable': [ServiceNotExistError()], '_chkconfigDisable': [ServiceNotExistError()]}) e: ServiceNotExistError() srvName: 'ksmtuned' args: () kwarg: {} srv: 'ksmtuned' alt: <function _chkconfigDisable at 0x7f4d8bb35320> alts: [<function _systemctlDisable at 0x7f4d8bb27a28>, <function _chkconfigDisable at 0x7f4d8bb35320>]
Created attachment 788986 [details] File: backtrace
Created attachment 788987 [details] File: environ
Hi Zhou, it seems that where ksmtuned is not present (e.g. fedora19), we fail during service-disable of CONFLICTING_SERVICES. Do you have a fix for this? Maybe: commit ad15e4590b32ec5c02f4206b87d05d9ec0baba81 Author: Zhou Zheng Sheng <zhshzhou.ibm.com> Date: Sat Jul 13 22:44:07 2013 +0800 vdsmd and supervdsmd: extract common init tasks and add init adapter Do you think we could backport this to ovirt-3.3?
Hi Federico. In commit ad15e45, Sthe ksmtuned.service is declared a conflict in the Systemd unit file. Systemd is tolerant on non-existing service name in conflict field. So I think this commit can fix the bug. The commit itself contains two small bugs that only can be triggered on RHEL6, so Yaniv submit commit aab8d20, 9f9f954 and 07be44a to fix them. If we backport the init adapter patch, Yaniv's fixes should be backported as well. Hi Yaniv, do you think it is correct to backport all the following commits? commit ad15e45 vdsmd and supervdsmd: extract common init tasks and add init adapter commit 8ca1e14 packaging: rename supervdsmServer.py to supervdsmServer and add environment file commit f4aab4f Packageing: Move SystemD and SysVInit files to respective sub-dirs commit 07be44a Removing has_systemd from sysv script commit 9f9f954 Adding requires to python-argparse commit aab8d20 Fixing start return the return value of vdsm execution
(In reply to Zhou Zheng Sheng from comment #4) > Hi Federico. > > In commit ad15e45, Sthe ksmtuned.service is declared a conflict in the > Systemd unit file. Systemd is tolerant on non-existing service name in > conflict field. So I think this commit can fix the bug. The commit itself > contains two small bugs that only can be triggered on RHEL6, so Yaniv submit > commit aab8d20, 9f9f954 and 07be44a to fix them. If we backport the init > adapter patch, Yaniv's fixes should be backported as well. As far as I know the service file is: [Unit] ... Conflicts=libvirt-guests.service ksmtuned.service [Service] ... ExecStart=/lib/systemd/systemd-vdsmd start Which means that even if we declare conflicts here (ksmtuned.service), we still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init file). And in the init file again we try to stop the CONFLICTING_SERVICES as shutdown_conflicting_srv doesn't care if we're using systemd or SysV, therefore failing in "vdsm-tool service-disable ksmtuned"
(In reply to Federico Simoncelli from comment #5) > As far as I know the service file is: > > [Unit] > ... > Conflicts=libvirt-guests.service ksmtuned.service > > [Service] > ... > ExecStart=/lib/systemd/systemd-vdsmd start > > Which means that even if we declare conflicts here (ksmtuned.service), we > still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init > file). > > And in the init file again we try to stop the CONFLICTING_SERVICES as > shutdown_conflicting_srv doesn't care if we're using systemd or SysV, > therefore failing in "vdsm-tool service-disable ksmtuned" After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init tasks and add init adapter", ExecStart is not calling /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve vdsm directly, so there will be no problem for ksmtuned in CONFLICTING_SERVICES in vdsmd.init.in.
(In reply to Zhou Zheng Sheng from comment #6) > (In reply to Federico Simoncelli from comment #5) > > As far as I know the service file is: > > > > [Unit] > > ... > > Conflicts=libvirt-guests.service ksmtuned.service > > > > [Service] > > ... > > ExecStart=/lib/systemd/systemd-vdsmd start > > > > Which means that even if we declare conflicts here (ksmtuned.service), we > > still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init > > file). > > > > And in the init file again we try to stop the CONFLICTING_SERVICES as > > shutdown_conflicting_srv doesn't care if we're using systemd or SysV, > > therefore failing in "vdsm-tool service-disable ksmtuned" > > After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init > tasks and add init adapter", ExecStart is not calling > /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve > vdsm directly, so there will be no problem for ksmtuned in > CONFLICTING_SERVICES in vdsmd.init.in. That is *exactly* why I asked you if that was the fix, and if it is, then you should probably backport it to ovirt-3.3.
(In reply to Federico Simoncelli from comment #7) > (In reply to Zhou Zheng Sheng from comment #6) > > (In reply to Federico Simoncelli from comment #5) > > > As far as I know the service file is: > > > > > > [Unit] > > > ... > > > Conflicts=libvirt-guests.service ksmtuned.service > > > > > > [Service] > > > ... > > > ExecStart=/lib/systemd/systemd-vdsmd start > > > > > > Which means that even if we declare conflicts here (ksmtuned.service), we > > > still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init > > > file). > > > > > > And in the init file again we try to stop the CONFLICTING_SERVICES as > > > shutdown_conflicting_srv doesn't care if we're using systemd or SysV, > > > therefore failing in "vdsm-tool service-disable ksmtuned" > > > > After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init > > tasks and add init adapter", ExecStart is not calling > > /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve > > vdsm directly, so there will be no problem for ksmtuned in > > CONFLICTING_SERVICES in vdsmd.init.in. > > That is *exactly* why I asked you if that was the fix, and if it is, then > you should probably backport it to ovirt-3.3. Sorry, so if I understand correctly we need all these dependencies: commit ad15e45 vdsmd and supervdsmd: extract common init tasks and add init adapter commit 8ca1e14 packaging: rename supervdsmServer.py to supervdsmServer and add environment file commit f4aab4f Packageing: Move SystemD and SysVInit files to respective sub-dirs commit 07be44a Removing has_systemd from sysv script commit 9f9f954 Adding requires to python-argparse commit aab8d20 Fixing start return the return value of vdsm execution Resetting the needinfo on Yaniv.
*** Bug 1004630 has been marked as a duplicate of this bug. ***
I saw that issue more than once. Federico is right, it might be the case before merging ad15e45 . Before merging it, we should make it work smoothly of course, or instead of merging it I'll try to figure how can we correct the dependency issue and update the bug asap
Before ad15e459 we never failed shutdown_conflicting_srv if one of the stop operations failed, but this code does not run over systemd at all.. over systemd we use the "Conflicts" field, and the execute operation on master branch is: ExecStart=@VDSMDIR@/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null "@VDSMDIR@/vdsm" Over ovirt-3.3 branch we haven't merged ad15e459 and we still use "ExecStart=/lib/systemd/systemd-vdsmd start", but also shutdown_conflicting_srv always return 0. so I'm not sure that's the case here
(In reply to Yaniv Bronhaim from comment #11) > Over ovirt-3.3 branch we haven't merged ad15e459 and we still use > "ExecStart=/lib/systemd/systemd-vdsmd start", but also > shutdown_conflicting_srv always return 0. so I'm not sure that's the case > here In this case though shutdown_conflicting_srv returns 0 and swallows all the exceptions, abrt always detects un-caught python exception regardless how shutdown_conflicting_srv deals with it. I think this behaviour does not affect the startup of VDSM but it's a bit annoying for the user to see abrt warnings.
This particular issue is currently blocking the 3.3 release. Can we prioritize getting this fixed in the 3.3 branch? Thanks
Does it effect the start of vdsm when this error is raised? If not, and it just reports the error (as Zhou says in comment #12), it shouldn't be a blocker to 3.3 branch, and also shouldn't be fixed, as it is just a report for a real issue. Anyhow, in master branch we already stopped using vdsm-tool in sysv init script, so after stabilizing master branch, we'll be able to pull those changes also to 3.3 branch.
IIUC, this breaks ovirt-node on reboot. Testing without a manual workaround causes vdsm to fail to start on ovirt-node. With a manual workaround put in place for this bug (remove ksmtuned from the systemd-vdsmd file), vdsm starts correctly.
I did not try it out, but with systemd around, vdsmd init script should not bother to stop conflicting services - it's already done by virture of declaring them as conflicting. diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 56f07bd..b38791d 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -186,7 +186,9 @@ start() { local ret_val "@PYTHON@" "@VDSMDIR@/hooks.pyc" before_vdsm_start - shutdown_conflicting_srv + if ! has_systemd; then + shutdown_conflicting_srv + fi if ! "@LIBEXECDIR@/vdsm-gencerts.sh" --check; then printf $"Configuring a self-signed VDSM host certificate: "