Bug 690177 - tuned switch (stopping of ktune) profile problem
tuned switch (stopping of ktune) profile problem
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: tuned (Show other bugs)
15
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Jan Vcelak
Fedora Extras Quality Assurance
:
: 690194 692765 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-03-23 10:19 EDT by Jan Ščotka
Modified: 2013-03-08 23:23 EST (History)
16 users (show)

See Also:
Fixed In Version: tuned-0.2.21-1.fc15
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-06-29 18:02:26 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jan Ščotka 2011-03-23 10:19:01 EDT
Description of problem:
I've found, that in https://fedoraproject.org/wiki/QA:Testcase_Power_Management_tuned_basic testcase
switch to  tuned-adm profile laptop-battery-powersave was okay,
but any another switch profile fail or better to say waited for something. It can be caused due to my encrypted disk, but I'm not sure
I've had to kill commands by ctrl+c, 

Actual results:
tuned-adm profile throughput-performance
Stopping ktune (via systemctl):  


^CStopping tuned (via systemctl):                          [  OK  ]
Switching to profile 'throughput-performance'
Starting ktune (via systemctl):  
^CStarting tuned (via systemctl):  ^C


Expected results:
everything working, without problem

Additional info:
first turn on of laptop-battery-powersave caused need for passwd for partition:
# tuned-adm profile laptop-battery-powersave
Stopping ktune (via systemctl):                            [  OK  ]
Stopping tuned (via systemctl):                            [  OK  ]
Switching to profile 'laptop-battery-powersave'
Starting ktune (via systemctl):  Please enter passphrase for disk HITACHI_HTS722010K9SA00 (luks-6b2e13e1-3e4b-498c-a3c9-b9683996d2cc) on /shome! **********
                                                           [  OK  ]
Starting tuned (via systemctl):
Comment 1 Jan Vcelak 2011-03-23 11:25:28 EDT
Hi, I will need a little bit more information.

- Does turning tuned off work? (tuned-adm off)
- Can you try switching back to following profiles?
  - default
  - spindown-disk
  - latency-powersave
  - enterprise-storage

Thank you.
Comment 2 Jaroslav Škarvada 2011-03-23 16:06:28 EDT
It seems to be systemd (or tuned/systemd) related, same with the tuned-adm off  (used systemd-20-1.fc15).
Comment 3 Jan Vcelak 2011-03-24 10:52:13 EDT
*** Bug 690194 has been marked as a duplicate of this bug. ***
Comment 4 Jan Vcelak 2011-03-24 10:53:39 EDT
It seems to hang on systemd-ask-tty when restarting cpuspeed service.
Comment 5 Honza Horak 2011-03-24 11:31:42 EDT
Hi, I've the same problem, but without the additional info from comment #0 (I've encrypted only /home, which shouldn't be involved at all).

> - Does turning tuned off work? (tuned-adm off)
Yes, but the switching still doesn't work after turning off.

> - Can you try switching back to following profiles?
>   - default
>   - spindown-disk
>   - latency-powersave
>   - enterprise-storage
The first switch after system reboot works, but any additional switch hangs.
Comment 6 Jan Vcelak 2011-04-02 10:00:06 EDT
*** Bug 692765 has been marked as a duplicate of this bug. ***
Comment 7 Jan Vcelak 2011-05-24 07:10:03 EDT
It seems, that we encounter a deadlock. I'm really suspecting systemd. Lennart, please, can you take a look at this and tell us your opinion?

This occurs only when 'cpuspeed' package is installed. Ktune uses this daemon to switch CPU governors. (Service restart invoked from other init script.) The bug can be demonstrated using 'laptop-battery-powersave' profile.

[root@f15 ~]# rpm -qa systemd\* tuned cpuspeed
systemd-debuginfo-26-1.fc15.x86_64
tuned-0.2.20-1.fc15.noarch
systemd-26-1.fc15.x86_64
systemd-sysv-26-1.fc15.x86_64
cpuspeed-1.5-15.fc15.x86_64
systemd-units-26-1.fc15.x86_64

[root@f15 ~]# tuned-adm active
Current active profile: off
Service tuned: disabled, stopped
Service ktune: disabled, stopped
[root@f15 ~]# tuned-adm profile laptop-battery-powersave
Stopping ktune (via systemctl):                            [  OK  ]
Stopping tuned (via systemctl):                            [  OK  ]
Switching to profile 'laptop-battery-powersave'
Starting ktune (via systemctl):                            [  OK  ]
Starting tuned (via systemctl):                            [  OK  ]
[root@f15 ~]# service ktune restart
Restarting ktune (via systemctl):  (HANG)

Processes tree:

[root@f15 ~]# pstree
systemd─┬─6*[agetty]
        ├─auditd───{auditd}
        ├─crond
        ├─dbus-daemon───{dbus-daemon}
        ├─dhclient
        ├─ktune───tunedadm.sh───service───cpuspeed───systemctl
        ├─login───bash
        ├─rsyslogd───3*[{rsyslogd}]
        ├─2*[sendmail]
        ├─sshd─┬─sshd───bash───service───ktune───systemctl───systemd-tty-ask
        │      └─sshd───bash───pstree
        ├─tuned
        └─udevd───2*[udevd]

After interruption by ^C:

[root@f15 ~]# pstree
systemd─┬─6*[agetty]
        ├─auditd───{auditd}
        ├─crond
        ├─dbus-daemon───{dbus-daemon}
        ├─dhclient
        ├─ktune───tunedadm.sh───service───cpuspeed───systemctl
        ├─login───bash
        ├─rsyslogd───3*[{rsyslogd}]
        ├─2*[sendmail]
        ├─sshd─┬─sshd───bash
        │      └─sshd───bash───pstree
        ├─tuned
        └─udevd───2*[udevd]

Other services are affected as well, even if we kill hanging systemctl:

[root@f15 ~]# service nginx start
Starting nginx (via systemctl):  ^C
[root@f15 ~]# systemctl start nginx.service
^C
[root@f15 ~]# killall systemctl
[root@f15 ~]# pstree
systemd─┬─6*[agetty]
        ├─auditd───{auditd}
        ├─crond
        ├─dbus-daemon───{dbus-daemon}
        ├─dhclient
        ├─login───bash
        ├─rsyslogd───3*[{rsyslogd}]
        ├─2*[sendmail]
        ├─sshd─┬─sshd───bash───pstree
        │      └─sshd───bash
        ├─tuned
        └─udevd───2*[udevd]
[root@f15 ~]# service nginx start
Starting nginx (via systemctl):  ^C
[root@f15 ~]# pstree
systemd─┬─6*[agetty]
        ├─auditd───{auditd}
        ├─crond
        ├─dbus-daemon───{dbus-daemon}
        ├─dhclient
        ├─login───bash
        ├─rsyslogd───3*[{rsyslogd}]
        ├─2*[sendmail]
        ├─sshd─┬─sshd───bash───pstree
        │      └─sshd───bash
        ├─tuned
        └─udevd───2*[udevd]
Comment 8 Jan Vcelak 2011-05-24 08:42:12 EDT
I just found out, that presence of this bug also prevents the system to shutdown/reboot.
Comment 9 Benjamin Wild 2011-05-29 14:05:56 EDT
confirmed. enabling tuned prevents my fedora 15 installation from shutting down.
Comment 10 Michal Schmidt 2011-05-31 11:16:05 EDT
(In reply to comment #7)
> This occurs only when 'cpuspeed' package is installed. Ktune uses this daemon
> to switch CPU governors. (Service restart invoked from other init script.)

I see. The /etc/init.d/ktune initscript eventually calls into /etc/tune-profiles/functions where "service cpuspeed restart" is used.

service ktune stop
-> service cpuspeed restart

There is an ordering dependency between ktune.service and cpuspeed.service:
$ systemctl show -p After ktune.service
After=... cpuspeed.service ...

That's because the ktune initscript does not have an LSB header, so it gets ordered according to chkconfig priorities: ktune 27, cpuspeed 13.

When the restart of cpuspeed is requested, systemd will attempt to honour the ordering dependency, i.e. to stop ktune before cpuspeed. It sees ktune is already being stopped, so it waits. But the stopping of ktune waits on the restart of cpuspeed. Deadlock! (The job to stop ktune.service will timeout after 5 minutes and mark the service failed.)

To prevent the deadlock you can:
 - add LSB headers to /etc/init.d/ktune, avoiding the ordering dependency
   to cpuspeed; or
 - use "systemctl --ignore-dependencies restart cpuspeed.service"
   in /etc/tune-profiles/functions instead of plain "service cpuspeed restart".
Comment 11 Jan Vcelak 2011-05-31 17:05:23 EDT
Thank you, Michal. Will be fixed as suggested.
Comment 12 Michel Alexandre Salim 2011-06-14 02:06:34 EDT
Is there any ETA for the fix? I'm having to replace the two lines that restart cpuspeed in the functions file on all my machines
Comment 13 Jan Vcelak 2011-06-14 10:41:04 EDT
Soon Michel, I'm occupied with other bugs. Please, give me a week.
Comment 14 Jan Vcelak 2011-06-21 08:15:25 EDT
Finally, fixed in tuned-0.2.21-1.fc15
Comment 15 Fedora Update System 2011-06-21 08:18:41 EDT
tuned-0.2.21-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/tuned-0.2.21-1.fc15
Comment 16 Fedora Update System 2011-06-21 19:54:02 EDT
Package tuned-0.2.21-1.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing tuned-0.2.21-1.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/tuned-0.2.21-1.fc15
then log in and leave karma (feedback).
Comment 17 Fedora Update System 2011-06-29 18:02:15 EDT
tuned-0.2.21-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.