Bug 972457 - agetty idle I/O polling causes elevated CPU usage
Summary: agetty idle I/O polling causes elevated CPU usage
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: util-linux
Version: 19
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Karel Zak
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 973501 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-06-09 15:03 UTC by Alexander Demenshin
Modified: 2013-06-14 04:46 UTC (History)
5 users (show)

Fixed In Version: util-linux-2.23.1-3.fc19
Clone Of:
Environment:
Last Closed: 2013-06-14 04:46:06 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Alexander Demenshin 2013-06-09 15:03:44 UTC
Description of problem:

When running F19 (and, most likely, all previous versions - see below) as KVM guest, active agetty constantly consumes a bit of CPU (1% in my case), which is elevated to ca. 4-5% of host CPU usage. Given that most virtualized systems are running headless but will have at least one agetty running, several of them will consume significant amount of CPU time at the host even when idling.

Version-Release number of selected component (if applicable):

util-linux-2.23.1-2.fc19.x86_64

How reproducible:

100% (Proxmox 3.0 KVM, though it may affect any QEMU-KVM or even other VMs/hypervisors).

Steps to Reproduce:
1. Run F19 inside KVM
2. Login and logout on console (though in my case login is not necessary)
3. Make sure that the guest is idle
4. Observe CPU usage in host and guest.

Actual results:

Checking CPU usage reveals that agetty in guest system is constantly consuming ca. 1% of CPU time (may depend on CPU type, it is Core i5-3570T in my case), host system (KVM process) is consuming ca. 4-5% of CPU. Running 6 active agetty processes leads to ca. 12-13% CPU usage in host (each still consuming ca. 1% in guest).

Expected results:

CPU usage should be very close to 0% while idling (as it is when agetty is not running).

Additional info:

ltrace of running agetty reveals the following loop:

16:36:30.110604 usleep(1000)                                                                                  = <void>
16:36:30.112047 read(0 <no return ...>
error: maximum array length seems negative
, "\n", 1)                                                                                                    = -1
16:36:30.112494 __errno_location()                                                                            = 0x7f02ac6756c0
16:36:30.112829 usleep(1000)                                                                                  = <void>
16:36:30.114235 read(0 <no return ...>
error: maximum array length seems negative
, "\n", 1)                                                                                                    = -1
16:36:30.114748 __errno_location()                                                                            = 0x7f02ac6756c0

This loop (function get_logname in agetty.c) is the cause of elevated CPU usage, as it is repeated every 1ms - actually, just running usleep(1000) in a loop (without any I/O) will lead to this result.

Probably, this can (and should) be fixed outside of agetty (at hypervisor/VM/kernel level, as it is related to nanosleep implementation), but for the time being it makes sense to get rid of this loop in agetty to avoid unnecessary CPU consumption in virtualized environments.

PS: There are many reports about Linux KVM guests consuming up to 10% of host CPU when idle even on tickless kernels, so I believe that usleep() loop on agetty may be the cause.

Comment 1 Zdenek Kabelac 2013-06-11 18:37:22 UTC
Yep - agetty is getting very visible in the 'top' list.
It can actually beat even Firefox now with the amount of used CPU time.
(And my fan rotates unnecessarily)

Comment 2 Karel Zak 2013-06-12 19:07:39 UTC
*** Bug 973501 has been marked as a duplicate of this bug. ***

Comment 3 Florian Weimer 2013-06-12 19:12:43 UTC
It's not related to KVM, it happens on bare metal, too.

Comment 4 Florian Weimer 2013-06-12 19:30:40 UTC
Probably introduced by:

commit 3aa6b68f7e19fa3e1c2bba75bee921a98b7b46af
Author: Werner Fink <werner>
Date:   Mon May 9 15:52:36 2011 +0200

    agetty: proper session on the terminal line
    
    Ensure a proper session on the terminal line, that is do a
    vhangup() and become the controlling terminal.  After this
    determine if the terminal line a virtual console by using
    the ioctl TIOCMGET to get the status modem bits of a serial
    line which is a invalid argument on a virtual console.
    
    Signed-off-by: Werner Fink <werner>

I have verified that the triggering error code is EAGAIN, and the descriptor has indeed the O_NONBLOCK flag set.

Comment 5 Alexander Demenshin 2013-06-12 20:17:34 UTC
(In reply to Florian Weimer from comment #3)

> It's not related to KVM, it happens on bare metal, too.

Indeed... But the effect is most noticeable in KVM, though.

Comment 6 Karel Zak 2013-06-13 09:46:22 UTC
(In reply to Florian Weimer from comment #4)
> Probably introduced by:

No, it was my mistake introduced by --local-line patch (backported to f19). 

I have fixed the code, so O_NONBLOCK should be used only for serial lines with -L (CLOCAL) flags. I have also increased the usleep() to 0.25s to make it less expensive on systems with ugly serial lines.

Comment 7 Fedora Update System 2013-06-13 10:01:08 UTC
util-linux-2.23.1-3.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/util-linux-2.23.1-3.fc19

Comment 8 Fedora Update System 2013-06-13 18:02:56 UTC
Package util-linux-2.23.1-3.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing util-linux-2.23.1-3.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-10745/util-linux-2.23.1-3.fc19
then log in and leave karma (feedback).

Comment 9 Fedora Update System 2013-06-14 04:46:06 UTC
util-linux-2.23.1-3.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.