Bug 10509 - floating point calculations yield NaN although they shouldn't
floating point calculations yield NaN although they shouldn't
Status: CLOSED ERRATA
Product: Red Hat Linux
Classification: Retired
Component: kernel (Show other bugs)
6.2
i386 Linux
medium Severity high
: ---
: ---
Assigned To: Michael K. Johnson
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2000-04-01 13:34 EST by Cord Kielhorn
Modified: 2008-05-01 11:37 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2000-04-22 03:19:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Cord Kielhorn 2000-04-01 13:34:29 EST
This happens on at least PIII (Katmai) SMP machines with kernel-2.2.14-5.0
(the pre 6.2 one from rawhide, but the 6.2 version is not different with
respect to that (btw: why didn't you increment the release number between
rawhide's 2.2.14-5.0 and zoot's 2.2.14-5.0?):
Under seldom circumstances floating point calculations yield NaN although
they haven't before in the same run. For example the same calculation
yields the correct result for several million times when it suddenly
"Nans". Maybe sometimes floating point calculations even yield a wrong but
non-Nan result, we haven't checked that extensively. We have observed this
behaviour on several machines, so it shouldn't be a hardware problem. The
processors are 500 MHz Katmai with CPU id 0672 and microcode patches 000B
and 0010. It seems to be a problem with the fxsave and fxrestore fast
extended FPU save and restore feature used in kernel-2.2.14-5.0's
linux-2.2.12-PIII.patch: if one rebuilds the kernel with CONFIG_X86_FX
unconfigured the problems disappear.

Cheers
  Cord
Comment 1 Doug Ledford 2000-04-22 03:19:59 EDT
The errata kernel doesn't include the PIII patches and as such solves this
problem.  It's also beleived to be fixed internally in the latest version of the
PIII patches.

Note You need to log in before you can comment on or make changes to this bug.