Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
As can be seen in kernel/time/tick-oneshot.c:tick_dev_program_event(), clock_event_device:min_delta_ns, which represents the granularity of the clockevent timer increments, can grow till overflow and can never be reduced.
One possible observable consequence of that is, if it ever overflows, the loop in this function becomes endless, because
expires = ktime_add_ns(now, dev->min_delta_ns);
gives either negative expires or expires less than now, either of which resulting in error return from clockevents_program_event() which causes the loop to start over.
Version-Release number of selected component (if applicable):
2.6.32-504.3.3.el6.x86_64
How reproducible:
The endless loop on one of the CPUs was seen once in a virtual machine in Parallels Server. The exact details are still investigated.
Additional info:
These problems have been addressed by the following commits in the mainline kernel:
commit 80a05b9ffa7dc13f6693902dd8999a2b61a3a0d7
Author: Thomas Gleixner <tglx>
Date: Fri Mar 12 17:34:14 2010 +0100
clockevents: Sanitize min_delta_ns adjustment and prevent overflows
The current logic which handles clock events programming failures can
increase min_delta_ns unlimited and even can cause overflows.
Sanitize it by:
- prevent zero increase when min_delta_ns == 1
- limiting min_delta_ns to a jiffie
- bail out if the jiffie limit is hit
- add retries stats for /proc/timer_list so we can gather data
Reported-by: Uwe Kleine-Koenig <u.kleine-koenig>
Signed-off-by: Thomas Gleixner <tglx>
commit d1748302f70be7469809809283fe164156a34231
Author: Martin Schwidefsky <schwidefsky.com>
Date: Tue Aug 23 15:29:42 2011 +0200
clockevents: Make minimum delay adjustments configurable
The automatic increase of the min_delta_ns of a clockevents device
should be done in the clockevents code as the minimum delay is an
attribute of the clockevents device.
In addition not all architectures want the automatic adjustment, on a
massively virtualized system it can happen that the programming of a
clock event fails several times in a row because the virtual cpu has
been rescheduled quickly enough. In that case the minimum delay will
erroneously be increased with no way back. The new config symbol
GENERIC_CLOCKEVENTS_MIN_ADJUST is used to enable the automatic
adjustment. The config option is selected only for x86.
Signed-off-by: Martin Schwidefsky <schwidefsky.com>
Cc: john stultz <johnstul.com>
Link: http://lkml.kernel.org/r/20110823133142.494157493@de.ibm.com
Signed-off-by: Thomas Gleixner <tglx>
Unfortunately, no.
We've seen several (very few) sporadic reproductions during routine automated testing of Parallels Server, but have been unable yet to identify the exact scenario of how min_delta_ns can grow up to those pathological values initially.
(In reply to Roman Kagan from comment #4)
> Unfortunately, no.
>
> We've seen several (very few) sporadic reproductions during routine
> automated testing of Parallels Server, but have been unable yet to identify
> the exact scenario of how min_delta_ns can grow up to those pathological
> values initially.
Okay, I've run this in our testing suite and don't see any issues so I'm going to do some additional testing over the weekend.
P.
Comment 9RHEL Program Management
2015-01-26 12:29:54 UTC
This request was evaluated by Red Hat Product Management for
inclusion in a Red Hat Enterprise Linux release. Product
Management has requested further review of this request by
Red Hat Engineering, for potential inclusion in a Red Hat
Enterprise Linux release for currently deployed products.
This request is not yet committed for inclusion in a release.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://rhn.redhat.com/errata/RHSA-2015-1272.html
Comment 17Dr. David Alan Gilbert
2018-11-28 15:12:05 UTC
*** Bug 1538078 has been marked as a duplicate of this bug. ***
Description of problem: As can be seen in kernel/time/tick-oneshot.c:tick_dev_program_event(), clock_event_device:min_delta_ns, which represents the granularity of the clockevent timer increments, can grow till overflow and can never be reduced. One possible observable consequence of that is, if it ever overflows, the loop in this function becomes endless, because expires = ktime_add_ns(now, dev->min_delta_ns); gives either negative expires or expires less than now, either of which resulting in error return from clockevents_program_event() which causes the loop to start over. Version-Release number of selected component (if applicable): 2.6.32-504.3.3.el6.x86_64 How reproducible: The endless loop on one of the CPUs was seen once in a virtual machine in Parallels Server. The exact details are still investigated. Additional info: These problems have been addressed by the following commits in the mainline kernel: commit 80a05b9ffa7dc13f6693902dd8999a2b61a3a0d7 Author: Thomas Gleixner <tglx> Date: Fri Mar 12 17:34:14 2010 +0100 clockevents: Sanitize min_delta_ns adjustment and prevent overflows The current logic which handles clock events programming failures can increase min_delta_ns unlimited and even can cause overflows. Sanitize it by: - prevent zero increase when min_delta_ns == 1 - limiting min_delta_ns to a jiffie - bail out if the jiffie limit is hit - add retries stats for /proc/timer_list so we can gather data Reported-by: Uwe Kleine-Koenig <u.kleine-koenig> Signed-off-by: Thomas Gleixner <tglx> commit d1748302f70be7469809809283fe164156a34231 Author: Martin Schwidefsky <schwidefsky.com> Date: Tue Aug 23 15:29:42 2011 +0200 clockevents: Make minimum delay adjustments configurable The automatic increase of the min_delta_ns of a clockevents device should be done in the clockevents code as the minimum delay is an attribute of the clockevents device. In addition not all architectures want the automatic adjustment, on a massively virtualized system it can happen that the programming of a clock event fails several times in a row because the virtual cpu has been rescheduled quickly enough. In that case the minimum delay will erroneously be increased with no way back. The new config symbol GENERIC_CLOCKEVENTS_MIN_ADJUST is used to enable the automatic adjustment. The config option is selected only for x86. Signed-off-by: Martin Schwidefsky <schwidefsky.com> Cc: john stultz <johnstul.com> Link: http://lkml.kernel.org/r/20110823133142.494157493@de.ibm.com Signed-off-by: Thomas Gleixner <tglx>