Bug 999664 - [abrt] vdsm-python-4.12.0-18.git78903dd.fc19: service.py:353:_runAlts:ServiceNotExistError: ServiceNotExistError: Tried all alternatives but failed:
Summary: [abrt] vdsm-python-4.12.0-18.git78903dd.fc19: service.py:353:_runAlts:Service...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: vdsm
Version: 19
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Federico Simoncelli
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: abrt_hash:5186b49ac356bffe55c25b7f56b...
: 1004630 (view as bug list)
Depends On:
Blocks: 918494
TreeView+ depends on / blocked
 
Reported: 2013-08-21 19:44 UTC by Bob McCarthy
Modified: 2013-09-16 14:31 UTC (History)
9 users (show)

Fixed In Version: vdsm-4.12.1-2
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-09-16 14:31:44 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: backtrace (1.29 KB, text/plain)
2013-08-21 19:44 UTC, Bob McCarthy
no flags Details
File: environ (130 bytes, text/plain)
2013-08-21 19:44 UTC, Bob McCarthy
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 19046 0 None None None Never

Description Bob McCarthy 2013-08-21 19:44:11 UTC
Description of problem:
update vdsm

Version-Release number of selected component:
vdsm-python-4.12.0-18.git78903dd.fc19

Additional info:
reporter:       libreport-2.1.6
cmdline:        /usr/bin/python /usr/bin/vdsm-tool service-disable ksmtuned
executable:     /usr/bin/vdsm-tool
kernel:         3.10.4-300.fc19.x86_64
runlevel:       unknown
type:           Python
uid:            0

Truncated backtrace:
service.py:353:_runAlts:ServiceNotExistError: ServiceNotExistError: Tried all alternatives but failed:
ServiceNotExistError: ksmtuned is not native systemctl service
ServiceNotExistError: ksmtuned is not a SysV service

Traceback (most recent call last):
  File "/usr/bin/vdsm-tool", line 143, in <module>
    sys.exit(main())
  File "/usr/bin/vdsm-tool", line 140, in main
    return tool_command[cmd]["command"](*args[1:])
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/service.py", line 405, in service_disable
    return _runAlts(_srvDisableAlts, srvName)
  File "/usr/lib64/python2.7/site-packages/vdsm/tool/service.py", line 353, in _runAlts
    ('\n'.join(str(e) for errs in errors.values() for e in errs)))
ServiceNotExistError: ServiceNotExistError: Tried all alternatives but failed:
ServiceNotExistError: ksmtuned is not native systemctl service
ServiceNotExistError: ksmtuned is not a SysV service

Local variables in innermost frame:
errors: defaultdict(<type 'list'>, {'_systemctlDisable': [ServiceNotExistError()], '_chkconfigDisable': [ServiceNotExistError()]})
e: ServiceNotExistError()
srvName: 'ksmtuned'
args: ()
kwarg: {}
srv: 'ksmtuned'
alt: <function _chkconfigDisable at 0x7f4d8bb35320>
alts: [<function _systemctlDisable at 0x7f4d8bb27a28>, <function _chkconfigDisable at 0x7f4d8bb35320>]

Comment 1 Bob McCarthy 2013-08-21 19:44:16 UTC
Created attachment 788986 [details]
File: backtrace

Comment 2 Bob McCarthy 2013-08-21 19:44:22 UTC
Created attachment 788987 [details]
File: environ

Comment 3 Federico Simoncelli 2013-09-05 09:21:52 UTC
Hi Zhou, it seems that where ksmtuned is not present (e.g. fedora19), we fail during service-disable of CONFLICTING_SERVICES.

Do you have a fix for this? Maybe:

commit ad15e4590b32ec5c02f4206b87d05d9ec0baba81
Author: Zhou Zheng Sheng <zhshzhou.ibm.com>
Date:   Sat Jul 13 22:44:07 2013 +0800

    vdsmd and supervdsmd: extract common init tasks and add init adapter

Do you think we could backport this to ovirt-3.3?

Comment 4 Zhou Zheng Sheng 2013-09-05 09:48:44 UTC
Hi Federico.

In commit ad15e45, Sthe ksmtuned.service is declared a conflict in the Systemd unit file. Systemd is tolerant on non-existing service name in conflict field. So I think this commit can fix the bug. The commit itself contains two small bugs that only can be triggered on RHEL6, so Yaniv submit commit aab8d20, 9f9f954 and 07be44a to fix them. If we backport the init adapter patch, Yaniv's fixes should be backported as well.

Hi Yaniv, do you think it is correct to backport all the following commits?

commit ad15e45 vdsmd and supervdsmd: extract common init tasks and add init adapter
commit 8ca1e14 packaging: rename supervdsmServer.py to supervdsmServer and add environment file
commit f4aab4f Packageing: Move SystemD and SysVInit files to respective sub-dirs
commit 07be44a Removing has_systemd from sysv script
commit 9f9f954 Adding requires to python-argparse
commit aab8d20 Fixing start return the return value of vdsm execution

Comment 5 Federico Simoncelli 2013-09-05 10:22:28 UTC
(In reply to Zhou Zheng Sheng from comment #4)
> Hi Federico.
> 
> In commit ad15e45, Sthe ksmtuned.service is declared a conflict in the
> Systemd unit file. Systemd is tolerant on non-existing service name in
> conflict field. So I think this commit can fix the bug. The commit itself
> contains two small bugs that only can be triggered on RHEL6, so Yaniv submit
> commit aab8d20, 9f9f954 and 07be44a to fix them. If we backport the init
> adapter patch, Yaniv's fixes should be backported as well.

As far as I know the service file is:

[Unit]
...
Conflicts=libvirt-guests.service ksmtuned.service

[Service]
...
ExecStart=/lib/systemd/systemd-vdsmd start

Which means that even if we declare conflicts here (ksmtuned.service), we still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init file).

And in the init file again we try to stop the CONFLICTING_SERVICES as shutdown_conflicting_srv doesn't care if we're using systemd or SysV, therefore failing in "vdsm-tool service-disable ksmtuned"

Comment 6 Zhou Zheng Sheng 2013-09-05 10:28:27 UTC
(In reply to Federico Simoncelli from comment #5)
> As far as I know the service file is:
> 
> [Unit]
> ...
> Conflicts=libvirt-guests.service ksmtuned.service
> 
> [Service]
> ...
> ExecStart=/lib/systemd/systemd-vdsmd start
> 
> Which means that even if we declare conflicts here (ksmtuned.service), we
> still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init
> file).
> 
> And in the init file again we try to stop the CONFLICTING_SERVICES as
> shutdown_conflicting_srv doesn't care if we're using systemd or SysV,
> therefore failing in "vdsm-tool service-disable ksmtuned"

After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init tasks and add init adapter", ExecStart is not calling /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve vdsm directly, so there will be no problem for ksmtuned in CONFLICTING_SERVICES in vdsmd.init.in.

Comment 7 Federico Simoncelli 2013-09-05 10:43:39 UTC
(In reply to Zhou Zheng Sheng from comment #6)
> (In reply to Federico Simoncelli from comment #5)
> > As far as I know the service file is:
> > 
> > [Unit]
> > ...
> > Conflicts=libvirt-guests.service ksmtuned.service
> > 
> > [Service]
> > ...
> > ExecStart=/lib/systemd/systemd-vdsmd start
> > 
> > Which means that even if we declare conflicts here (ksmtuned.service), we
> > still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init
> > file).
> > 
> > And in the init file again we try to stop the CONFLICTING_SERVICES as
> > shutdown_conflicting_srv doesn't care if we're using systemd or SysV,
> > therefore failing in "vdsm-tool service-disable ksmtuned"
> 
> After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init
> tasks and add init adapter", ExecStart is not calling
> /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve
> vdsm directly, so there will be no problem for ksmtuned in
> CONFLICTING_SERVICES in vdsmd.init.in.

That is *exactly* why I asked you if that was the fix, and if it is, then you should probably backport it to ovirt-3.3.

Comment 8 Federico Simoncelli 2013-09-05 10:47:08 UTC
(In reply to Federico Simoncelli from comment #7)
> (In reply to Zhou Zheng Sheng from comment #6)
> > (In reply to Federico Simoncelli from comment #5)
> > > As far as I know the service file is:
> > > 
> > > [Unit]
> > > ...
> > > Conflicts=libvirt-guests.service ksmtuned.service
> > > 
> > > [Service]
> > > ...
> > > ExecStart=/lib/systemd/systemd-vdsmd start
> > > 
> > > Which means that even if we declare conflicts here (ksmtuned.service), we
> > > still run /lib/systemd/systemd-vdsmd afterwards (which is the vdsm init
> > > file).
> > > 
> > > And in the init file again we try to stop the CONFLICTING_SERVICES as
> > > shutdown_conflicting_srv doesn't care if we're using systemd or SysV,
> > > therefore failing in "vdsm-tool service-disable ksmtuned"
> > 
> > After we applied commit ad15e4 "vdsmd and supervdsmd: extract common init
> > tasks and add init adapter", ExecStart is not calling
> > /lib/systemd/systemd-vdsmd any longer, it calls daemon adapter which execve
> > vdsm directly, so there will be no problem for ksmtuned in
> > CONFLICTING_SERVICES in vdsmd.init.in.
> 
> That is *exactly* why I asked you if that was the fix, and if it is, then
> you should probably backport it to ovirt-3.3.

Sorry, so if I understand correctly we need all these dependencies:

commit ad15e45 vdsmd and supervdsmd: extract common init tasks and add init adapter
commit 8ca1e14 packaging: rename supervdsmServer.py to supervdsmServer and add environment file
commit f4aab4f Packageing: Move SystemD and SysVInit files to respective sub-dirs
commit 07be44a Removing has_systemd from sysv script
commit 9f9f954 Adding requires to python-argparse
commit aab8d20 Fixing start return the return value of vdsm execution

Resetting the needinfo on Yaniv.

Comment 9 Mike Burns 2013-09-05 13:49:58 UTC
*** Bug 1004630 has been marked as a duplicate of this bug. ***

Comment 10 Yaniv Bronhaim 2013-09-08 08:48:33 UTC
I saw that issue more than once. Federico is right, it might be the case before merging ad15e45 . Before merging it, we should make it work smoothly of course, or instead of merging it I'll try to figure how can we correct the dependency issue and update the bug asap

Comment 11 Yaniv Bronhaim 2013-09-08 15:12:37 UTC
Before ad15e459 we never failed shutdown_conflicting_srv if one of the stop operations failed,  but this code does not run over systemd at all.. over systemd we use the "Conflicts" field, and the execute operation on master branch is:
ExecStart=@VDSMDIR@/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null "@VDSMDIR@/vdsm"

Over ovirt-3.3 branch we haven't merged ad15e459 and we still use "ExecStart=/lib/systemd/systemd-vdsmd start", but also shutdown_conflicting_srv always return 0. so I'm not sure that's the case here

Comment 12 Zhou Zheng Sheng 2013-09-09 06:13:00 UTC
(In reply to Yaniv Bronhaim from comment #11)
> Over ovirt-3.3 branch we haven't merged ad15e459 and we still use
> "ExecStart=/lib/systemd/systemd-vdsmd start", but also
> shutdown_conflicting_srv always return 0. so I'm not sure that's the case
> here

In this case though shutdown_conflicting_srv returns 0 and swallows all the exceptions, abrt always detects un-caught python exception regardless how shutdown_conflicting_srv deals with it. I think this behaviour does not affect the startup of VDSM but it's a bit annoying for the user to see abrt warnings.

Comment 13 Mike Burns 2013-09-09 13:34:23 UTC
This particular issue is currently blocking the 3.3 release.  Can we prioritize getting this fixed in the 3.3 branch?  

Thanks

Comment 14 Yaniv Bronhaim 2013-09-09 14:30:22 UTC
Does it effect the start of vdsm when this error is raised? If not, and it just reports the error (as Zhou says in comment #12), it shouldn't be a blocker to 3.3 branch, and also shouldn't be fixed, as it is just a report for a real issue.

Anyhow, in master branch we already stopped using vdsm-tool in sysv init script, so after stabilizing master branch, we'll be able to pull those changes also to 3.3 branch.

Comment 15 Mike Burns 2013-09-09 15:19:51 UTC
IIUC, this breaks ovirt-node on reboot.  Testing without a manual workaround causes vdsm to fail to start on ovirt-node.  With a manual workaround put in place for this bug (remove ksmtuned from the systemd-vdsmd file), vdsm starts correctly.

Comment 16 Dan Kenigsberg 2013-09-09 16:00:53 UTC
I did not try it out, but with systemd around, vdsmd init script should not bother to stop conflicting services - it's already done by virture of declaring them as conflicting.

diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in
index 56f07bd..b38791d 100755
--- a/vdsm/vdsmd.init.in
+++ b/vdsm/vdsmd.init.in
@@ -186,7 +186,9 @@ start() {
     local ret_val
     "@PYTHON@" "@VDSMDIR@/hooks.pyc" before_vdsm_start
 
-    shutdown_conflicting_srv
+    if ! has_systemd; then
+        shutdown_conflicting_srv
+    fi
 
     if ! "@LIBEXECDIR@/vdsm-gencerts.sh" --check; then
         printf $"Configuring a self-signed VDSM host certificate: "


Note You need to log in before you can comment on or make changes to this bug.