Bug 928326

Summary: fontconfig problem with python3-matplotlib
Product: [Fedora] Fedora Reporter: Thomas Spura <tomspur>
Component: python-matplotlibAssignee: Thomas Spura <tomspur>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: asaha, gwync, haliyo, jonathan.underwood, jspaleta, paulo.cesar.pereira.de.andrade, thibault.north, tomspur
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-matplotlib-1.2.0-14.fc19 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-04-12 22:25:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 784947    

Description Thomas Spura 2013-03-27 12:37:43 UTC
Description of problem:
matplotlib is build with the experimental fontconfig-based backend and it's currently failing with:

/usr/lib64/python3.3/site-packages/matplotlib/axis.py in draw(self, renderer, *args, **kwargs)
   1049         ticks_to_draw = self._update_ticks(renderer)
   1050         ticklabelBoxes, ticklabelBoxes2 = self._get_tick_bboxes(ticks_to_draw,
-> 1051                                                                 renderer)
   1052 
   1053         for tick in ticks_to_draw:
/usr/lib64/python3.3/site-packages/matplotlib/axis.py in _get_tick_bboxes(self, ticks, renderer)
    998         for tick in ticks:
    999             if tick.label1On and tick.label1.get_visible():
-> 1000                 extent = tick.label1.get_window_extent(renderer)
   1001                 ticklabelBoxes.append(extent)
   1002             if tick.label2On and tick.label2.get_visible():
/usr/lib64/python3.3/site-packages/matplotlib/text.py in get_window_extent(self, renderer, dpi)
    751             raise RuntimeError('Cannot get window extent w/o renderer')
    752 
--> 753         bbox, info = self._get_layout(self._renderer)
    754         x, y = self.get_position()
    755         x, y = self.get_transform().transform_point((x, y))
/usr/lib64/python3.3/site-packages/matplotlib/text.py in _get_layout(self, renderer)
    303         tmp, lp_h, lp_bl = get_text_width_height_descent('lp',
    304                                                          self._fontproperties,
--> 305                                                          ismath=False)
    306         offsety = lp_h * self._linespacing
    307 
/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py in get_text_width_height_descent(self, s, prop, ismath)
    206 
    207         flags = get_hinting_flag()
--> 208         font = self._get_agg_font(prop)
    209         font.set_text(s, 0.0, flags=flags)  # the width and height of unrotated string
    210         w, h = font.get_width_height()
/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py in _get_agg_font(self, prop)
    244 
    245         if font is None:
--> 246             fname = findfont(prop)
    247             font = RendererAgg._fontd.get(fname)
    248             if font is None:
/usr/lib64/python3.3/site-packages/matplotlib/font_manager.py in findfont(prop, fontext)
   1297             return cached
   1298 
-> 1299         result = fc_match(prop, fontext)
   1300         if result is None:
   1301             result = fc_match(':', fontext)
/usr/lib64/python3.3/site-packages/matplotlib/font_manager.py in fc_match(pattern, fontext)
   1281             return None
   1282         if pipe.returncode == 0:
-> 1283             for match in _fc_match_regex.finditer(output):
   1284                 file = match.group(1)
   1285                 if os.path.splitext(file)[1][1:] in fontexts:
TypeError: can't use a string pattern on a bytes-like object


Version-Release number of selected component (if applicable):
python3-matplotlib-1.2.0-8.fc18.x86_64

How reproducible:
Run ipython tests in mock and tests "test_px_pylab" and "test_magic_pylab" will be failing with the TypeError from above. It seems to work fine when not using mock though...

The TypeError hints, that the regex needs to be created with a bytes-like object too:
diff --git a/lib/matplotlib/font_manager.py b/lib/matplotlib/font_manager.py
index 28ac734..91bd294 100644
--- a/lib/matplotlib/font_manager.py
+++ b/lib/matplotlib/font_manager.py
@@ -1293,7 +1293,7 @@ if USE_FONTCONFIG and sys.platform != 'win32':
                     return file
         return None
 
-    _fc_match_regex = re.compile(r'\sfile:\s+"([^"]*)"')
+    _fc_match_regex = re.compile(b'\sfile:\s+"([^"]*)"')
     _fc_match_cache = {}
 
     def findfont(prop, fontext='ttf'):

Doing that, the build is failing as it finds the font file "None", which is not existent...

I know too less about font config to know, what really is going on here...

Comment 1 Paulo Andrade 2013-03-28 18:44:02 UTC
I made an update for this problem but did not submit it
28th february because it did not correct all problems
with python3 and matplotlib, but I guess better to remake
it and submit, for an "evolutionary" update of the package,
as I did not work on that anymore.

There is a nasty issue with USE_FONTCONFIG, it is not used
by default by upstream and breaks a lot of stuff with
python3+matplotlib, but it is required to avoid a lot of
issues with stix-fonts changes (and afaik works correctly
with python2). Upstream does not have this issue because
they use the fonts/data in the mpl_data subdir, what
is agains't fedora policies to bundle fonts.

I will make an update to correct the exact problems
reported and/or add the patches suggested.

Comment 2 Fedora Update System 2013-03-28 20:31:14 UTC
python-matplotlib-1.2.0-10.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-10.fc18

Comment 3 Thomas Spura 2013-04-02 11:26:06 UTC
Now there is this crash from http://koji.fedoraproject.org/koji/taskinfo?taskID=5197830:

ERROR: %pylab works on engines
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/testing/decorators.py", line 229, in skipper_func
    return f(*args, **kwargs)
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/parallel/tests/test_magics.py", line 329, in test_px_pylab
    ip.magic("px plot(rand(100))")
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/core/interactiveshell.py", line 2137, in magic
    return self.run_line_magic(magic_name, magic_arg_s)
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/core/interactiveshell.py", line 2063, in run_line_magic
    result = fn(*args)
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/parallel/client/magics.py", line 234, in px
    return self.parallel_execute(line)
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/parallel/client/magics.py", line 259, in parallel_execute
    result.get()
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/parallel/client/asyncresult.py", line 125, in get
    raise self._exception
  File "/builddir/build/BUILDROOT/ipython-0.13.1-4.fc19.noarch/usr/lib/python3.3/site-packages/IPython/parallel/client/asyncresult.py", line 150, in wait
    raise r
IPython.parallel.error.RemoteError: RuntimeError(Could not open facefile None; Cannot_Open_Resource

It seems facefile is not found...

Comment 4 Paulo Andrade 2013-04-02 14:52:51 UTC
I just tested a plain rpmbuild in rawhide, and only got
this failure:

ERROR: test_send (IPython.zmq.tests.test_session.TestSession)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.3/site-packages/zmq/sugar/attrsettr.py", line 35, in __setattr__
    opt = getattr(constants, upper_key)
AttributeError: 'module' object has no attribute 'DATA'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pcpa/rpmbuild/BUILDROOT/ipython-0.13.1-4.fc20.x86_64/usr/lib/python3.3/site-packages/IPython/zmq/tests/test_session.py", line 76, in test_send
    socket = MockSocket(zmq.Context.instance(),zmq.PAIR)
  File "/home/pcpa/rpmbuild/BUILDROOT/ipython-0.13.1-4.fc20.x86_64/usr/lib/python3.3/site-packages/IPython/zmq/tests/test_session.py", line 34, in __init__
    self.data = []
  File "/usr/lib64/python3.3/site-packages/zmq/sugar/attrsettr.py", line 38, in __setattr__
    self.__class__.__name__, upper_key)
AttributeError: MockSocket has no such option: DATA

I am almost sure adding
%if 0%{?fedora} >= 18
BuildRequires:	stix-math-fonts
%else
BuildRequires:	stix-fonts
%endif

May also need to run the tests under xvfb-run.

Trying it in a f19 mock build...

Comment 5 Paulo Andrade 2013-04-02 16:19:26 UTC
The mock build did find the same error adding stix-fonts
to build requires.

I also just made a fool proof test of a python-matplotlib
update to actually have the fonts as requires, so that
the problem should not happen again in other circumstances.

Comment 6 Fedora Update System 2013-04-02 17:48:06 UTC
python-matplotlib-1.2.0-11.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-11.fc19

Comment 7 Fedora Update System 2013-04-02 17:49:01 UTC
python-matplotlib-1.2.0-11.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-11.fc18

Comment 8 Fedora Update System 2013-04-02 19:17:44 UTC
Package python-matplotlib-1.2.0-11.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-matplotlib-1.2.0-11.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-4658/python-matplotlib-1.2.0-11.fc19
then log in and leave karma (feedback).

Comment 9 Thomas Spura 2013-04-03 00:21:20 UTC
Sorry, the patch from comment #0 was wrong and I have it working differently locally. I'll change the patch and build it in fc18+ and ship the updates.

Unfortunately the ipython issue is not fully resolved, as there is another bug in pyparsing, see bug #947655.

I just saw, that bug #912843 is actually the same issue as this one (or the other way around...).

The upstream pull request is at:
https://github.com/matplotlib/matplotlib/pull/1879

Comment 10 Fedora Update System 2013-04-03 01:20:20 UTC
python-matplotlib-1.2.0-12.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-12.fc18

Comment 11 Fedora Update System 2013-04-03 01:20:45 UTC
python-matplotlib-1.2.0-12.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-12.fc19

Comment 12 Thomas Spura 2013-04-10 13:09:55 UTC
*** Bug 915722 has been marked as a duplicate of this bug. ***

Comment 13 Thomas Spura 2013-04-10 13:18:07 UTC
*** Bug 929383 has been marked as a duplicate of this bug. ***

Comment 14 Fedora Update System 2013-04-12 22:25:52 UTC
python-matplotlib-1.2.0-12.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2013-04-20 19:09:19 UTC
python-matplotlib-1.2.0-12.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2013-07-23 14:01:43 UTC
python-matplotlib-1.2.0-14.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-14.fc19

Comment 17 Fedora Update System 2013-07-23 14:02:49 UTC
python-matplotlib-1.2.0-14.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-14.fc18

Comment 18 Fedora Update System 2013-07-25 00:33:33 UTC
python-matplotlib-1.2.0-14.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2013-07-25 00:49:48 UTC
python-matplotlib-1.2.0-14.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.