Bug 10400 - Bug in the double sqrt routine. Result should be corr. rounded according to the IEEE standard.
Bug in the double sqrt routine. Result should be corr. rounded according to t...
Product: Red Hat Linux
Classification: Retired
Component: glibc (Show other bugs)
i386 Linux
medium Severity high
: ---
: ---
Assigned To: Cristian Gafton
Depends On:
  Show dependency treegraph
Reported: 2000-03-28 17:59 EST by shane.story
Modified: 2008-05-01 11:37 EDT (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2000-03-28 18:32:32 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description shane.story 2000-03-28 17:59:35 EST
#include <stdio.h>
#include <float.h>
#include <math.h>

double sqrt(double);

double x, y, r;

*((int*)&x+1) = 0x3d2ffffe;
*(int*)&x = 0x7fffffff;

printf("x                %08X%08X %.16e \n\n", *(((int*)&x) + 1),

y = sqrt(x);

printf("result           %08X%08X %.16e \n", *(((int*)&y) + 1),

*((int*)&r+1) = 0x3e8fffff;
*(int*)&r = 0x3ffffdbf;
printf("result should be %08X%08X %.16e \n", *(((int*)&r) + 1),


It reports the following
x                      3D2FFFFE7FFFFFFF 5.6843378203226540e-14

result                3E8FFFFF3FFFFDC0 2.3841849383641896e-07
result should be 3E8FFFFF3FFFFDBF 2.3841849383641894e-07

The machine is running:

Red Hat Release 6.0 (hedwig)
Kernel 2.2.5-15smp on an i686
Comment 1 Cristian Gafton 2000-03-28 18:32:59 EST
The representation is outside of the bounds of representation. 15 precision
digits is the most you can do on a double (well, that's what is guaranteed)

Note You need to log in before you can comment on or make changes to this bug.