Bug 1411070 - Broken on ppc64
Summary: Broken on ppc64
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: python-matplotlib
Version: rawhide
Hardware: ppc64
OS: Unspecified
unspecified
urgent
Target Milestone: ---
Assignee: Thomas Spura
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-08 06:20 UTC by Adam Williamson
Modified: 2017-01-09 16:46 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-01-09 16:46:35 UTC
Type: Bug


Attachments (Terms of Use)

Description Adam Williamson 2017-01-08 06:20:52 UTC
There seems to be some weird problem with matplotlib and fonts on ppc64. The test suite fails, with some *gigantic* recursive tracebacks that repeat this zillions of times:

  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph
    return self._get_glyph('rm', font_class, sym, fontsize)
  File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 861, in _get_glyph

and ultimately wind up with:

File "/builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py", line 87, in get_unicode_index
    if symbol == '-':
RuntimeError: maximum recursion depth exceeded in cmp

this is not just a test suite problem, because it's breaking a build I'm trying to do. My build of python-deap for Rawhide failed because the noarch -doc package was not identical on all arch builds; that was because of this bug. In the ppc64 build, this bug occurred:

https://kojipkgs.fedoraproject.org//work/tasks/6214/17206214/build.log

and broke the generation of one of the image files in the documentation, meaning it was missing from the final doc package, which was the cause of the build failure (because the image was included in the doc package for other arches).

I can't see how to get around this problem and get a python-deap build done aside from disabling the build for ppc64...

I tried building matplotlib 2.0.0rc2, but if I enable the tests for ppc64, the same error occurs during the test run.

Comment 1 Adam Williamson 2017-01-08 08:14:39 UTC
so matplotlib's UnicodeFonts._get_glyph() has this code, which obviously has the risk of recursing:

        if not found_symbol:
            if self.cm_fallback:
...
            else:
                if fontname in ('it', 'regular') and isinstance(self, StixFonts):
                    return self._get_glyph('rm', font_class, sym, fontsize)

since it's calling itself, that obviously runs the risk of a recursion loop; it's assuming that call will always go down some other path and not just recurse. I don't know, yet, why we're hitting a recursion loop there on ppc64, but not on other arches. I don't know where this 'rm' font is coming from, even.

Comment 2 Adam Williamson 2017-01-08 17:04:00 UTC
<ignatenkobrain> adamw: which original issue do you have?
<adamw> ignacio: well, i started at https://bugzilla.redhat.com/show_bug.cgi?id=1411070
<adamw> grr
<adamw> ignatenkobrain: ^^
<adamw> but the recursion is, i think, only happening because we're often failing to find a character on ppc64 where we don't on le arches
<adamw> it just so happens that there's one code path where an unexpected failure to find a character causes an infinite recursion; it's easy enough to cut off the recursion, but that's only fixing one symptom of a deeper problem
<ignatenkobrain> hm
<adamw> the ppc64 build has a bunch of these warnings:
<adamw>  /builddir/build/BUILDROOT/python-matplotlib-2.0.0-0.5.rc2.fc26.ppc64/usr/lib64/python2.7/site-packages/matplotlib/mathtext.py:867: MathTextWarning: Font 'rm' does not have a glyph for '5' [U+1d7e7]
<adamw> there are *none* of those in an x86_64 build - grepping the log for "does not have a glyph" gets you nothing
<adamw> so i traced it to https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/mathtext.py#L801
<adamw> the UnicodeFonts _get_glyph() function
<adamw> and i'm fairly sure what's going wrong that shouldn't go wrong is this call to get_char_index:
<adamw> https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/mathtext.py#L839
<adamw> in our ppc64 build, that's often returning 0, so found_symbol is False, so we then hit the `if not found_symbol:` branch which can either print that warning or recurse infinitely via https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/mathtext.py#L857
<adamw> FUN STUFF
<adamw> what i don't know is why we're frequently getting 0s back from get_char_index() on ppc64, but not on the other arches. obviously an endian problem *somewhere*, but i'm damned if i can figure out where
<adamw> ppc64le doesn't have the problem either, which makes it 99.99% sure a be problem i guess
<ignatenkobrain> adamw: can't it be freetype bug?
<adamw> it could be!
<adamw> and i was halfway through writing a freetype test program
<adamw> but now i have to go take a shower and go for dim sum...

Comment 3 Adam Williamson 2017-01-09 01:29:43 UTC
So after some more digging, I'm pretty sure I've figured this out:

https://github.com/matplotlib/matplotlib/pull/7768

will try and get an rc2 build done.

Comment 4 Dominik 'Rathann' Mierzejewski 2017-01-09 08:14:34 UTC
Great work, Adam, thanks a lot.

Comment 5 Charalampos Stratakis 2017-01-09 13:38:36 UTC
It seems the build is successful in rawhide. Should the bug be closed?

Comment 6 Adam Williamson 2017-01-09 16:46:35 UTC
Well, the specific bug I ran into does appear to be fixed. matplotlib is still quite badly broken on ppc64 - the test suite is disabled in the spec, if you enable it and run it it hits several hundred failures, and they're not all freetype rendering differences, there are all sorts of classic endian errors like divide-by-zeros. But that's probably better addressed upstream than here, so sure.


Note You need to log in before you can comment on or make changes to this bug.