Bug 821191 - RFE: Update to pari 2.5.1
Summary: RFE: Update to pari 2.5.1
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: pari
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Paul Howarth
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 822896 822897 822898 837004
Blocks: 821195 823083 836816 836823
TreeView+ depends on / blocked
 
Reported: 2012-05-12 22:55 UTC by Paulo Andrade
Modified: 2012-07-02 20:21 UTC (History)
3 users (show)

Fixed In Version: pari-2.5.1-1.fc18
Clone Of:
Environment:
Last Closed: 2012-07-02 20:21:53 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
proposed pari patch (2.44 KB, patch)
2012-05-18 00:24 UTC, Paulo Andrade
no flags Details | Diff

Description Paulo Andrade 2012-05-12 22:55:06 UTC
Pari 2.5 is required by my work in progress fedora sagemath package.

  I packaged locally pari 2.5.1 and major change I noticed is that now
the emacs interface is no longer in the main tarball, so I updated it
to use the external upstream pariemacs-3.03.tar.gz tarball.

  Initially reported http://lists.fedoraproject.org/pipermail/scitech/2012-April/000087.html

Spec URL: http://fedorapeople.org/~pcpa/pari.spec
SRPM URL: http://fedorapeople.org/~pcpa/pari-2.5.1-1.fc16.src.rpm

Comment 1 Paul Howarth 2012-05-12 23:34:59 UTC
I'm working on this at the moment with Jerry James. Complications include:

* the test suite segfaults if the package is built with optimization higher than -O0 with gcc 4.7 (src/basemath/bibli2.c appears to be where the problem is but I haven't narrowed it any further yet)

* the updated version is incompatible with some downstream users of pari, such as perl-Math-Pari, so I also need to introduce a libpari235 package to avoid breakage there

I think it would be best if pariemacs was packaged separately given that upstream does it that way.

Comment 2 Paulo Andrade 2012-05-17 02:22:21 UTC
I preferred to not make a full description of the new patch in the
spec to avoid visual polution and just point to this bugzilla.
Following valgrind chain it was easy to figure out what was the
most likely cause, and it did indeed fix it:

#==22995== Invalid read of size 8
#==22995==    at 0x54116E0: veccmp (bibli2.c:1158)
#==22995==    by 0x5412B03: gen_sortspec (bibli2.c:1243)
#==22995==    by 0x5412885: gen_sortspec (bibli2.c:1258)
#==22995==    by 0x541AFFE: gen_sort (bibli2.c:1326)
#==22995==    by 0x541BB84: vecsort0 (bibli2.c:1454)

I did not work on fixing some rpmlint issues to avoid making changes
harder to follow.

I did also not package separately pariemacs, but for my needs
in sagemath, it could just be removed. I already have too many
packages needing review of enhancement requests :-)

Spec URL: http://fedorapeople.org/~pcpa/pari.spec
SRPM URL: http://fedorapeople.org/~pcpa/pari-2.5.1-2.fc18.src.rpm

Comment 3 Paul Howarth 2012-05-17 17:46:56 UTC
I found the vecsort0 issue too - see Bug #821918.

Has there been any upstream discussion of pari-2.5.1-gmp.patch?

Comment 4 Paulo Andrade 2012-05-17 19:40:45 UTC
I just reported it
http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1317

and will try to work on what I suggested upstream, that is to
only call mp_set_memory_functions from the standalone gp
application, but not from the library.
(In reply to comment #3)
> I found the vecsort0 issue too - see Bug #821918.

  Nice :-) It was one of the first things I did want to debug
after upgrading to rawhide, but was unaware of the other bug report.
Good we come out to the same conclusion. (After reading the pari
bug report) Unfortunately I do not believe any compiler can track
this problem; maybe some static analysis tool could notice it. BTW,
for the sake of testing, I did build the package, without applying
the patch with clang, it does not show the problem, but works... but
that is undefined behavior, and clang should be just statically
allocating space in the stack, while gcc should be increasing/decreasing
it when entering/leaving the block (would need to check disassembly to
confirm that...), what causes valgrind to detect it.

> Has there been any upstream discussion of pari-2.5.1-gmp.patch?

  I just reported it
http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1317

and will try to work on what I suggested upstream, that is to
only call mp_set_memory_functions from the standalone gp
application, but not from the library.

Comment 5 Paul Howarth 2012-05-17 20:16:56 UTC
(In reply to comment #4)
> > Has there been any upstream discussion of pari-2.5.1-gmp.patch?
> 
>   I just reported it
> http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1317
> 
> and will try to work on what I suggested upstream, that is to
> only call mp_set_memory_functions from the standalone gp
> application, but not from the library.

OK, let's see how that goes.

I'm going to run the full test suite in the 2.5.1 build ("make test-all") but that requires the elldata.tgz, galdata.tgz and seadata.tgz files from upstream, two of which are rather large and I don't think they're likely to change much, so although I've currently got them as additional sources in my pari package, I think they'd be better as one or more separate packages pulled in as buildrequires by pari. debian/Ubuntu seem to package these separately. If I put together some packages for these, would you review them?

Comment 6 Paulo Andrade 2012-05-17 20:46:59 UTC
Sure I would review them. BTW, I have these statically in Mandriva pari:

Source1: http://pari.math.u-bordeaux.fr/pub/pari/packages/elldata.tgz
Source2: http://pari.math.u-bordeaux.fr/pub/pari/packages/galdata.tgz
Source3: http://pari.math.u-bordeaux.fr/pub/pari/packages/seadata.tgz
Source4: http://pari.math.u-bordeaux.fr/pub/pari/packages/nftables.tgz
Source5: http://pari.math.u-bordeaux.fr/pub/pari/packages/galpol.tgz

%package data
Group: Sciences/Mathematics
Summary: Optional pari data packages
Requires: %{name} > 2.2.10

%description data
elldata is PARI/GP version of J. E. Cremona Elliptic Curve Data,
needed by ellsearch and ellidentify.
galdata is needed by polgalois to compute Galois group in degrees
8 through 11.
seadata is needed by ellap for large primes.
nftables is a repackaging of the historical megrez number field
tables (errors fixed, 1/10th the size, easier to use).

But installed disk usage is not small, 154M.

I should update the description to tell about galpol...
http://pari.math.u-bordeaux.fr/pub/pari/packages/README

AFAIR, sagemath also needs pari with at least elldata.

Comment 7 Paulo Andrade 2012-05-18 00:22:51 UTC
(In reply to comment #3)
> I found the vecsort0 issue too - see Bug #821918.
> 
> Has there been any upstream discussion of pari-2.5.1-gmp.patch?

With my limited pari internals and usages, I figured 2 possible
options, all of which rely on code calling pari_init_opts()
with INIT_SIGm not set (what sagemath does), that is, to tell
pari to not handle signals, what should be the logical approach.

Solution 1:
Add an extra argument to pari_kernel_init() that is the init_opts
argument, and then pari_kernel_init only wraps gmp memory functions
if INIT_SIGm is set, otherwise the code to defer SIGINT is a noop.

Solution 2:
Only call pari_kernel_init if INIT_SIGm is not set.

Solution 1 should be the best approach, because no code should
be calling pari_kernel_init() directly, as that function does
not have an "if (initialized)" guard anyway, and is declared
in a private header, but it is installed by pari-devel:

$ grep -r pari_kernel_init /usr/include/pari/
/usr/include/pari/paripriv.h:int   pari_kernel_init(void);

Comment 8 Paulo Andrade 2012-05-18 00:24:52 UTC
Created attachment 585319 [details]
proposed pari patch

Comment 9 Paul Howarth 2012-05-18 10:44:05 UTC
(In reply to comment #8)
> Created attachment 585319 [details]
> proposed pari patch

Not sent it upstream yet?

Comment 10 Paul Howarth 2012-05-18 13:03:52 UTC
(In reply to comment #6)
> Sure I would review them. BTW, I have these statically in Mandriva pari:
> 
> Source1: http://pari.math.u-bordeaux.fr/pub/pari/packages/elldata.tgz
> Source2: http://pari.math.u-bordeaux.fr/pub/pari/packages/galdata.tgz
> Source3: http://pari.math.u-bordeaux.fr/pub/pari/packages/seadata.tgz
> Source4: http://pari.math.u-bordeaux.fr/pub/pari/packages/nftables.tgz
> Source5: http://pari.math.u-bordeaux.fr/pub/pari/packages/galpol.tgz

I've done the elldata (Bug #822896), galdata (Bug #822897) and seadata (Bug #822898) packages that are needed for the test suite; others can be added later as required. I prefer to keep them separate to minimize the churn on the mirrors when one of the packages needs to be updated.

Current work-in-progress pari package is:
http://www.city-fan.org/~paul/extras/pari/pari-2.5.1-1.fc18.src.rpm

It doesn't have the malloc/init patch yet; I'm interested to see what upstream has to say about it.

Comment 11 Paulo Andrade 2012-05-18 13:54:57 UTC
(In reply to comment #9)
> (In reply to comment #8)
> > Created attachment 585319 [details]
> > proposed pari patch
> 
> Not sent it upstream yet?

Actually I was interested in knowing if you or anybody else
had any comments :-) I added the bugzilla link to the upstream
report, but will also forward the patch to the upstream bug
report.

Comment 12 Paulo Andrade 2012-05-18 14:11:05 UTC
Just forward the proposed patch, for the possible solutions what
I considered the best, to the upstream bug report. See
http://pari.math.u-bordeaux.fr/cgi-bin/bugreport.cgi?bug=1317

Comment 13 Paulo Andrade 2012-06-09 14:32:00 UTC
Any news about update of pari?
As per discussion of still open report for patches to avoid calling mp_set_memory_functions when pari is a library, what I should do is to save callback pointers before any code path that calls pari_kernel_init, if pari built with gmp; usually by pari_init_opts.
After returning from such code, it should restore the callbacks calling again mp_set_memory_functions. Personally, I do not like this as it should really be patched in pari, but pari upstream does not want the patch and sagemath just nukes the call to mp_set_memory_functions in the pari spkg.
After the rant above, I still want to see pari upgraded :-)

Comment 14 Paul Howarth 2012-06-14 15:56:43 UTC
I've been really busy this week at work and also with the perl 5.16 rebuild in Rawhide so I've not been able to work on it much.

My plan is to make a "libpari23" package that I can build perl-Math-Pari against, and also hopefully anything else in Fedora that won't build against pari 2.5. Before updating Rawhide to 2.5.x I need to get the "libpari23" package done and reviewed, and also to check for problems building other users of pari in Fedora against 2.5.x. Where problems occur, those packages will need patching either to fix the 2.5.x compatibility problem (it might be fixed by a release to a new upstream version in some cases) or to build against libpari23 instead.

If you would like to help with this exercise (Jerry James volunteered to do this too), you could try building those packages with the pari-2.5.1 package I mentioned in Comment 10.

I don't think the list of dependent packages is very long; I could only find these:

L-function
Macaulay2
eclib
genus2reduction

And of course there'll be your sagemath package to come later.

Comment 15 Paul Howarth 2012-06-19 14:37:44 UTC
I now have a libpari23 package and a perl-Math-Pari package that builds against it. The libpari23 and libpari23-devel packages are parallel-installable with their pari 2.5.1 equivalents. To build against libpari23, set CPPFLAGS (or CFLAGS if CPPFLAGS isn't supported) to $(pkg-config --cflags-only-I libpari23) and LDFLAGS to $(pkg-config --libs-only-L libpari23).

All packages available for review here:

http://www.city-fan.org/~paul/extras/pari/

Comment 16 Paulo Andrade 2012-07-01 15:25:16 UTC
(In reply to comment #14)

[...]

> I don't think the list of dependent packages is very long; I could only find
> these:
> 
> L-function

  Ops on this one. I will try to merge https://bugzilla.redhat.com/show_bug.cgi?id=821195 where I called it just L, and already patched for pari-2.5*

> Macaulay2

  Need to check, but AFAIR it did build fine in Mandriva with pari 2.5.

> eclib

  This one I know builds.

> genus2reduction

  I will also check this one. Currently in my sagemath package it builds the bundled version in the sagemath tarball.

> And of course there'll be your sagemath package to come later.

Comment 17 Paulo Andrade 2012-07-02 13:10:07 UTC
BTW, I am really in a hurry of pari being updated :-) It will temporarily only break perl-Math-Pari and other packages are just a matter of a rebuild, so, it should be ok to build it for rawhide.
Can you please do it ? :-) please :-)
Reason is that then I can proceed with a lot of other builds that are waiting for pari update (heck, it is uncommon to have a booting system after a rawhide yum update, so, leaving perl-Math-Pari broken a few days is not a big issue I guess)

Comment 18 Paul Howarth 2012-07-02 13:25:01 UTC
(In reply to comment #17)
> BTW, I am really in a hurry of pari being updated :-) It will temporarily
> only break perl-Math-Pari and other packages are just a matter of a rebuild,
> so, it should be ok to build it for rawhide.
> Can you please do it ? :-) please :-)
> Reason is that then I can proceed with a lot of other builds that are
> waiting for pari update (heck, it is uncommon to have a booting system after
> a rawhide yum update, so, leaving perl-Math-Pari broken a few days is not a
> big issue I guess)

Actually it is, as there's a mass rebuild of perl packages going on for Perl 5.16 in Rawhide at the moment, and a broken Math::Pari would mean that any package trying to pull that into its buildroot would be unbuildable.

I'll have a review request for libpar23 ready soon. Once that's reviewed it can be imported into Rawhide probably within a few hours and then I can update pari, which is ready to go.

Comment 19 Paul Howarth 2012-07-02 13:48:40 UTC
libpari23 review request submitted:
https://bugzilla.redhat.com/show_bug.cgi?id=837004

Comment 20 Paul Howarth 2012-07-02 20:21:36 UTC
pari 2.5.1 is now available to build against in Rawhide.

I just need to write an announcement to fedora-devel and the maintainers of the affected packages about the soname bump. I'll suggest that anyone having problems report them here.

Please apply for co-maintainership of pari in pkgdb - my main interest in this package is for perl-Math-Pari, which I think is a much weaker interest than you have!


Note You need to log in before you can comment on or make changes to this bug.