Bug 1104802

The problem function is this one:

static int
ell_over_Fq(GEN E)
{
  long t = ell_get_type(E);
  return t==t_ELL_Fp || t==t_ELL_Fq;
}

The value of t is indeed 0x10 when the crash occurs, so somehow t is treated as a pointer and dereferenced.

I reduced this to the following nonsense code, which exhibits the same behavior:

#include <stdio.h>
#include <stdlib.h>

inline static long ell_get_type(long *e) { return ((long **)e)[14][1]; }

static int
ell_over_Fq(long *E)
{
  long t = ell_get_type(E);
  return t==3 || t==4;
}

long *
ellmul(long *e)
{
  if (ell_over_Fq(e))
    {
      if ((((unsigned long)(e[0])) >> (__WORDSIZE - 7)) == 5U)
	return &e[1];
      else
	return e;
    }
  return e;
}

int
main ()
{
  long gen_0[2] = { 0x0200000000000002, 2L };
  long e[2] = { 0x2200000000000002, (long) gen_0 };

  ellmul(e);
  puts ("Successful completion");
  return EXIT_SUCCESS;
}

But this is bogus, because e, to which ell_get_type() is applied, doesn't have 14 elements.  It has 2.  The real problem is that we're walking off the end of an array, which somehow manifests as the strange error where the long int t is somehow treated as a pointer.  That may be a gcc bug, but nevertheless, there is a pari bug here, too.  In the case of the input given above, the real pari ell_get_type() function is called on an array of length 6.

Note that if 14 is changed to 1 in this example, the program completes successfully.

Upstream says:

 "The correct syntax for both versions is
  E = ellinit([0,1,0,2,-15]);
  ellpow(E,[2,1],5)

  Of course, it should report an error instead of crashing."

Upstream fix that will be in pari 2.7.2 is included in pari-2.7.1-4.fc21, resolving this issue.

Code relying on use of this syntax will need to be fixed to use the preferred format from Comment #3 if it will need to work with pari 2.8 onwards, which will treat the original syntax as an error.