Bug 912843

Summary:	run time Type error while calling show() to plot any graph, due to type mismatch in font_manager.py
Product:	[Fedora] Fedora	Reporter:	fedvasu <vadie.developer>
Component:	python-matplotlib	Assignee:	Paulo Andrade <paulo.cesar.pereira.de.andrade>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	18	CC:	gwync, jonathan.underwood, jspaleta, paulo.cesar.pereira.de.andrade, thibault.north, tomspur
Target Milestone:	---
Target Release:	---
Hardware:	noarch
OS:	Linux
Whiteboard:
Fixed In Version:	python-matplotlib-1.2.0-14.fc19	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-04-12 22:26:14 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description fedvasu 2013-02-19 20:06:19 UTC

Description of problem:
There is a runtime error, in font_manager.py file of python3-matplotlib-tk backend

This problem doesn't even let us plot anything.(can't show our plot)

typical path:
/usr/lib/python3.3/site-packages/matplotlib/font_manager.py

Version-Release number of selected component (if applicable):


How reproducible:
Try this in (any) python3 shell 

Steps to Reproduce:
1.import numpy as np
2.import matplotlib.pyplot as plt
3.a = random.rand(5,5)
4.plt.plot(a)
5.plt.show() # runtime error
  
Actual results:
run time type error at line number 1283 of the aforementioned file at step 5.

Expected results:
should plot just fine

Additional info:

Upstream bug-report with trivial fix (good enough for me)
https://github.com/matplotlib/matplotlib/issues/1767

replace at 1283 of font_manager.py, output by output.decode("utf-8"),

for match in _fc_match_regex.finditer(output): #old-broken
for match in _fc_match_regex.finditer(output.decode("utf-8")):#replacement-works

i.e. converting bytes to string solves the regex matching issue.

I am not aware of any upstream fixes/patches,if and when upstream provides patch for this problem use it, untill then use my fix/tell other users about this workaround.

IMPORTANT : Bugzilla asked me to give the component, I gave python-matplotlib,
as srpm for pyhton3-matplotlib was revealed as python-matplotlib, as did 
rpm -qi python3-matplotlib.

I have tested this thing with python3-matplotlib ONLY, I don't know this issue is also with python-matplotlib or not.

Comment 1 Paulo Andrade 2013-02-20 17:39:27 UTC

Yes, the component is python-matplotlib as the python3 build
is done in the same step.

Out of curiosity, do you have any idea about
http://pcpa.fedorapeople.org/matplotlib-3.3.txt

also reported when asking for upstream help on another issue at
http://sourceforge.net/mailarchive/message.php?msg_id=30360217

that is, run "nosetests-3.3 -v matplotlib.tests", but I
confess I did not look much at, as currently I only care
about python2.7 and python-matplotlib :-(

There is a simple correction pending for
https://bugzilla.redhat.com/show_bug.cgi?id=908717
so this should be worth a python-matplotlib update
for that bug and the problem you report here.

Comment 2 fedvasu 2013-02-21 12:15:06 UTC

(In reply to comment #1)
> Yes, the component is python-matplotlib as the python3 build
> is done in the same step.
> 
> Out of curiosity, do you have any idea about
> http://pcpa.fedorapeople.org/matplotlib-3.3.txt

No, I don't know about it, changing dict size while iterating, seems nasty to me :).

> also reported when asking for upstream help on another issue at
> http://sourceforge.net/mailarchive/message.php?msg_id=30360217
> 
> that is, run "nosetests-3.3 -v matplotlib.tests", but I
> confess I did not look much at, as currently I only care
> about python2.7 and python-matplotlib :-(
> 
> There is a simple correction pending for
> https://bugzilla.redhat.com/show_bug.cgi?id=908717
> so this should be worth a python-matplotlib update
> for that bug and the problem you report here.

Here is the upstream patch link for my issue.

https://github.com/mdboom/matplotlib/commit/991b06079ef2548e5b8699fb31174b0c804365f3

Basically, they made regex matcher take byte string instead of string. It should do the trick.

Comment 3 Paulo Andrade 2013-02-25 02:35:53 UTC

Sorry for the delay... while making tests for a possible
update, I noticed I could not get it to work with
python3. Only way to make it work is to patch
/usr/lib64/python3.3/site-packages/matplotlib/font_manager.py
and set "USE_FONTCONFIG = False"

Pyhton2 apparently works the same way with or without
your suggested patch.

Can you provide a simple script that shows the problem?
The original script needs some minor syntax changes, and
I may not have done it right.

Comment 4 fedvasu 2013-02-28 17:29:50 UTC

(In reply to comment #3)
> Sorry for the delay... while making tests for a possible
> update, I noticed I could not get it to work with
> python3. Only way to make it work is to patch
> /usr/lib64/python3.3/site-packages/matplotlib/font_manager.py
> and set "USE_FONTCONFIG = False"
> 
> Pyhton2 apparently works the same way with or without
> your suggested patch.
> 
> Can you provide a simple script that shows the problem?
> The original script needs some minor syntax changes, and
> I may not have done it right.

why don't you, communicate this to upstream,a you know what tests are failing because of the suggested upstream patch.For me my workaround/fix works.

It would be greate you bundle fixes of  all the issues you mentioned, including this one, by taking official upstream patches.

Any thoughts?

Comment 5 Paulo Andrade 2013-02-28 18:07:14 UTC

Using this small test:

---%<---
import numpy as np
import matplotlib.pyplot as plt
a = [1,4,3,2]
plt.plot(a)
plt.show()
---%<---

I see this in f18 and rawhide:
---%<---
$ python3
Python 3.3.0 (default, Sep 29 2012, 22:07:38) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> a = [1,4,3,2]
>>> plt.plot(a)
[<matplotlib.lines.Line2D object at 0x7f6ab14ace50>]
>>> plt.show()
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib64/python3.3/tkinter/__init__.py", line 1442, in __call__
    return self.func(*args)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_tkagg.py", line 276, in resize
    self.show()
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_tkagg.py", line 348, in draw
    FigureCanvasAgg.draw(self)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 439, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/figure.py", line 1000, in draw
    func(*args)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axes.py", line 2088, in draw
    a.draw(renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axis.py", line 1051, in draw
    renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axis.py", line 1000, in _get_tick_bboxes
    extent = tick.label1.get_window_extent(renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/text.py", line 753, in get_window_extent
    bbox, info = self._get_layout(self._renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/text.py", line 305, in _get_layout
    ismath=False)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 208, in get_text_width_height_descent
    font = self._get_agg_font(prop)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 246, in _get_agg_font
    fname = findfont(prop)
  File "/usr/lib64/python3.3/site-packages/matplotlib/font_manager.py", line 1299, in findfont
    result = fc_match(prop, fontext)
  File "/usr/lib64/python3.3/site-packages/matplotlib/font_manager.py", line 1283, in fc_match
    for match in _fc_match_regex.finditer(output):
TypeError: can't use a string pattern on a bytes-like object
---%<---

So, I see the point of your suggested patch, after applying it, I see:
---%<---
$ python3
Python 3.3.0 (default, Sep 29 2012, 22:07:38) 
[GCC 4.7.2 20120921 (Red Hat 4.7.2-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import matplotlib.pyplot as plt
>>> a = [1,4,3,2]
>>> plt.plot(a)
[<matplotlib.lines.Line2D object at 0x7fa6e1886290>]
>>> plt.show()
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib64/python3.3/tkinter/__init__.py", line 1442, in __call__
    return self.func(*args)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_tkagg.py", line 276, in resize
    self.show()
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_tkagg.py", line 348, in draw
    FigureCanvasAgg.draw(self)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 439, in draw
    self.figure.draw(self.renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/figure.py", line 1000, in draw
    func(*args)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axes.py", line 2088, in draw
    a.draw(renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/artist.py", line 55, in draw_wrapper
    draw(artist, renderer, *args, **kwargs)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axis.py", line 1051, in draw
    renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/axis.py", line 1000, in _get_tick_bboxes
    extent = tick.label1.get_window_extent(renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/text.py", line 753, in get_window_extent
    bbox, info = self._get_layout(self._renderer)
  File "/usr/lib64/python3.3/site-packages/matplotlib/text.py", line 305, in _get_layout
    ismath=False)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 208, in get_text_width_height_descent
    font = self._get_agg_font(prop)
  File "/usr/lib64/python3.3/site-packages/matplotlib/backends/backend_agg.py", line 251, in _get_agg_font
    hinting_factor=rcParams['text.hinting_factor'])
RuntimeError: Could not open facefile None; Cannot_Open_Resource
---%<---

But, if I change /usr/lib64/python3.3/site-packages/matplotlib/font_manager.py
to also use the pseudo patch:

-USE_FONTCONFIG = True
+USE_FONTCONFIG = False

it works, with or without your suggested patch. Also, with python2 it works
with or without your patch, so, I am trying to understand the problem, and
thus asked you for a small code snippet to reproduce what you are
experiencing, because I am assuming you are not using USE_FONTCONFIG = False

After that, that is, understanding the problem, I will bother upstream about it :-)

Comment 6 fedvasu 2013-03-01 18:36:03 UTC

yes, I get it that upstream patch breaks,but did you try this

#the following didn't give me any error/exception.

+for match in _fc_match_regex.finditer(output.decode("utf-8"))
-for match in _fc_match_regex.finditer(output)


I am  not sure, but as much as inline doc of font_manager.py as I read it, It says font_manager.py is new backend font configurator(something like that).(if you disable it, it doesn't use it, the "pseudo patch")

the way i see it, there is a type mismatch at the call site of a function.
#_fc_match_regex.finditer # <-- this function

Comment 7 fedvasu 2013-03-01 18:45:21 UTC

>  Also, with python2 it works
> with or without your patch, 
what?!! How can python2 cast 'output'(a byte array) to string?
or whenever output is created  it is created as string python2 and bytearray in python3?(!!) # I strongly suspect this.

>so, I am trying to understand the problem, and
> thus asked you for a small code snippet to reproduce what you are
> experiencing, because I am assuming you are not using USE_FONTCONFIG = False
> After that, that is, understanding the problem, I will bother upstream about
> it :-)


Cool, I totally agree. Yes I am not using USE_FONTCONFIG = False 
#I didn't change default configuration.

Comment 8 Fedora Update System 2013-03-28 20:30:57 UTC

python-matplotlib-1.2.0-10.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-10.fc18

Comment 9 Fedora Update System 2013-04-02 17:49:14 UTC

python-matplotlib-1.2.0-11.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-11.fc18

Comment 10 Fedora Update System 2013-04-03 01:20:33 UTC

python-matplotlib-1.2.0-12.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-12.fc18

Comment 11 Fedora Update System 2013-04-03 01:21:00 UTC

python-matplotlib-1.2.0-12.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-12.fc19

Comment 12 Fedora Update System 2013-04-03 04:55:07 UTC

Package python-matplotlib-1.2.0-11.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-matplotlib-1.2.0-11.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-4741/python-matplotlib-1.2.0-11.fc18
then log in and leave karma (feedback).

Comment 13 Fedora Update System 2013-04-12 22:26:16 UTC

python-matplotlib-1.2.0-12.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2013-04-20 19:09:42 UTC

python-matplotlib-1.2.0-12.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2013-07-23 14:02:22 UTC

python-matplotlib-1.2.0-14.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-14.fc19

Comment 16 Fedora Update System 2013-07-23 14:03:09 UTC

python-matplotlib-1.2.0-14.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/python-matplotlib-1.2.0-14.fc18

Comment 17 Fedora Update System 2013-07-25 00:33:55 UTC

python-matplotlib-1.2.0-14.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2013-07-25 00:49:59 UTC

python-matplotlib-1.2.0-14.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.