Bug 2164840

+++ This bug was initially created as a clone of Bug #2061557 +++

User has software with floating point exceptions enabled, and when using
libmagic internals, get a floating point exception when comparing a NaN with
a number.

This is a very special case, but would force user to disable/enable exceptions
when calling libmagic functions. Or, user suggests libmagic could
disable/(re)enable exceptions.

  Sample of what happens:

"""
#include <stdio.h>
#include <math.h>
#include <fenv.h>

int feenableexcept(int);
int fedisableexcept(int);

int
main(int argc, char *argv[])
{
  union {
    int   i;
    float f;
  } u[2];
  fedisableexcept(FE_ALL_EXCEPT);
  feenableexcept(FE_INVALID);
  u[0].i = -1;  // NaN
  u[1].f = 1;
  printf("%d\n", u[0].f < u[1].f);
  //printf("%d\n", isless(u[0].f, u[1].f));
  return 0;
}
"""

$ gcc test.c -lm
$ ./a.out
Floating point exception (core dumped)

  User also sets other exceptions, but libmagic should only trigger
FE_INVALID.

  A possible pseudo patch would be to change file/softmagic.c with pseudo patch:

...
#include <math.h>
...
		case '!':
-			matched = fv != fl;
+			matched = isunordered(fv, fl) ? 1 : fv != fl;
			break;

		case '=':
-			matched = fv == fl;
+			matched = isunordered(fv, fl) ? 0 : fv == fl;
			break;

		case '>':
-			matched = fv > fl;
+			matched = isgreater(fv, fl);
			break;

		case '<':
-			matched = fv < fl;
+			matched = isless(fv, fl);
			break;
...

and as appropriate for double values.

  There should be no floating operations on infinities or division
by zero, so, the above should resolve the issue for the user. There would
be a small extra overhead, when using the macros, that resolve to
__builtin_is{less,greater,unordered}.

  The problem happens with a customer user file. User is afraid other data
files might trigger the problem in libmagic.