Bug 10509 - floating point calculations yield NaN although they shouldn't
Summary: floating point calculations yield NaN although they shouldn't
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: kernel
Version: 6.2
Hardware: i386
OS: Linux
Target Milestone: ---
Assignee: Michael K. Johnson
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 2000-04-01 18:34 UTC by Cord Kielhorn
Modified: 2008-05-01 15:37 UTC (History)
1 user (show)

Clone Of:
Last Closed: 2000-04-22 07:19:06 UTC

Attachments (Terms of Use)

Description Cord Kielhorn 2000-04-01 18:34:29 UTC
This happens on at least PIII (Katmai) SMP machines with kernel-2.2.14-5.0
(the pre 6.2 one from rawhide, but the 6.2 version is not different with
respect to that (btw: why didn't you increment the release number between
rawhide's 2.2.14-5.0 and zoot's 2.2.14-5.0?):
Under seldom circumstances floating point calculations yield NaN although
they haven't before in the same run. For example the same calculation
yields the correct result for several million times when it suddenly
"Nans". Maybe sometimes floating point calculations even yield a wrong but
non-Nan result, we haven't checked that extensively. We have observed this
behaviour on several machines, so it shouldn't be a hardware problem. The
processors are 500 MHz Katmai with CPU id 0672 and microcode patches 000B
and 0010. It seems to be a problem with the fxsave and fxrestore fast
extended FPU save and restore feature used in kernel-2.2.14-5.0's
linux-2.2.12-PIII.patch: if one rebuilds the kernel with CONFIG_X86_FX
unconfigured the problems disappear.


Comment 1 Doug Ledford 2000-04-22 07:19:59 UTC
The errata kernel doesn't include the PIII patches and as such solves this
problem.  It's also beleived to be fixed internally in the latest version of the
PIII patches.

Note You need to log in before you can comment on or make changes to this bug.