Bug 562421

Created attachment 389290 [details]
matplotlib plot sing F12 demonstrating the wrong latex fonts

Created attachment 389291 [details]
plot from F10 using the attached script that shows the proper fonts.

This problem persists with the latest version, python-matplotlib-0.99.1.2-1.fc12.i686

I'll test your script this evening. My gut tells me this is a change due to font handling you just need to install some additional font packages to get the behavior your expect.  I'll let you know after I test the script.

Sorry I didnt catch this sooner. I was in Antarctica until mid Feb and this fell off the end of my todo list compared to other things.

-jef

Okay... I was able to reproduce the problem  on F12

I was able to fix it by adding the following lines to your script.

from matplotlib import rc

rc('text',usetex=True)

I also tried the scripts at http://matplotlib.sourceforge.net/users/usetex.html
which use the rc() statements and they work just fine.

Can you try the tex_demo.py from that url and follow the troubleshooting steps they suggest there?

-jef

Thanks Jef, that works once I also install the dvipng package.  I guess matplotlib changed the behavior of latex fonts from the version in Fedora 10, as I didn't need that line there.  I'll update my scripts on my F12 machine.

I appreciate your help.

I've the same problem with fedora 13, and I've dvipng installed

Ruggero,

Did you had the rc('text',usetex=True) line as per Comment 5 and/or test
the examples at http://matplotlib.sourceforge.net/users/usetex.html?


I can not reproduce the issue with examples which correctly configure matplotlib to use tex for rendering.     Please make sure you are configuring your script or your matplotlibrc file to explicitly turn tex rendering on. It appears the matplotlib system defaults as shipped with upstream might have changed between versions. Explicitly configuring matplotlib config to usetex=True  worksforme and for zingale.

-jef

(In reply to comment #8)
> Ruggero,
> 
> Did you had the rc('text',usetex=True) line as per Comment 5 and/or test
> the examples at http://matplotlib.sourceforge.net/users/usetex.html?
> 
> 
> I can not reproduce the issue with examples which correctly configure
> matplotlib to use tex for rendering.     Please make sure you are configuring
> your script or your matplotlibrc file to explicitly turn tex rendering on. It
> appears the matplotlib system defaults as shipped with upstream might have
> changed between versions. Explicitly configuring matplotlib config to
> usetex=True  worksforme and for zingale.
> 
> -jef    

Yes, you're right, now it works, but now I'm not able to save in svg

an other comment: rc('text',usetex=True) is a solution, but it change every label, every ticks. I want to use latex only when it's needed, only between $ $, and not ever.

an other comment: rc('text',usetex=True) is a solution, but it change every label, every ticks. I want to use latex only when it's needed, only between $ $, and not ever.

So... latex formatting works.  The question is why isn't $$ syntax working.

Can you give me a very simple script that I can use to reproduce the problem. I suspect this is a bug in the upstream matplotlib codebase and we'll need to have a discussion with upstream. But I need to do some testing to make sure we haven't introduced the problem in our packaging somehow.

-jef

So just for the record.... the specific feature is referred to as "mathtext"
in the documentation. And I can confirm its not working.  I can also confirm a configuration fix.

in your .matplotlibrc file change mathtext.fontset to
mathtext.fontset : stix

and make sure you have the stix-fonts package installed.

What is happening is that mathtext is using the cm "computer symbols" font by default and you are getting the wrong symbols rendered.

This looks like a mistake in the upstream code it should probably default to stix.
And my packages should probably require stix-fonts package.


-jef

The problem is that if I declare rc('text', usetex=True) the effect is that every text, label, axis label, ... is rendered using latex. For example I have to change the xlabel from 'n [%]' to r'n [\%]' because in latex you need to escape %. I want to use latex only when it is needed, only for formula between $ $, not for everythig. On opensuse I have this behaviour, on fedora I have this problem.

Created attachment 426531 [details]
output from this example: http://matplotlib.sourceforge.net/examples/pylab_examples/stix_fonts_demo.html

output from this example: http://matplotlib.sourceforge.net/examples/pylab_examples/stix_fonts_demo.html

(In reply to comment #14)
See comment 13

install stix-fonts  edit your matplotlibrc to instruct matplotlib to use stix fonts for mathext.

This will give you the behaviour you are expecting.

-jef

I also get this behavior on Fedora 13. Using the workaround Jef suggests makes the symbols appear ok. However, I think the default (cm) Computer Modern font, should also work. I tried this in Debian testing with a default installation of matplotlib and it works (without creating a .matplotlibrc). They are using matplotlib 0.99.3-1 and texlive-binaries version 2009-6. Perhaps the fix to this issue is in the new version of matplotlib or the newer version of texlive they're using? My Fedora 13 is still using texlive 2007...

Cassio:

Perhaps the cm font just isnt installed? We did change the font handling. It could simply be the case that I need to add the correct font package dep to the matplotlib packaging to ensure the cm font is on the system.  Can you do me a favor and continue the testing and confirm that you have the cm font on the system when the problem happens.

The fact that I can install the stix font and reconfigure and things work says the underlying font handling is working just fine and points to the cm font not being on the system.  That I can fix easily in packaging if you can tell me which package installs the cm font.

-jef

(In reply to comment #18)
> Cassio:
> 
> Perhaps the cm font just isnt installed? We did change the font handling. It
> could simply be the case that I need to add the correct font package dep to the
> matplotlib packaging to ensure the cm font is on the system.  Can you do me a
> favor and continue the testing and confirm that you have the cm font on the
> system when the problem happens.
> 
> The fact that I can install the stix font and reconfigure and things work says
> the underlying font handling is working just fine and points to the cm font not
> being on the system.  That I can fix easily in packaging if you can tell me
> which package installs the cm font.
> 
> -jef    

Jef, thanks for looking into this. 
I believe the CM font is installed in my system, as I have the latex packages. Just to test this, I created a file test.tex with the following:

-------- cut here ----------------
\documentclass[11pt,a4paper]{minimal}
\begin{document}
this is just a test of Computer Modern font.
\end{document}
-------- cut here ----------------

Compiling it with pdflatex test.tex produces the output:

Document Class: minimal 2001/05/25 Standard LaTeX minimal class
) (./test.aux) [1{/usr/share/texmf/fonts/map/pdftex/updmap/pdftex.map}]
(./test.aux) )</usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb>
Output written on test.pdf (1 page, 7312 bytes).

running ls shows that the file is there: 

[cassio@fedora ~]$ ls -l /usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb 
-rw-r--r--. 1 root root 26882 Jan  9  2006 /usr/share/texmf/fonts/type1/bluesky/cm/cmr10.pfb

Openning the pdf in evince > properties > fonts, shows that the document has CMR10 embedded.

The font should be there. Perhaps matplotlib is expecting the font in a different path?

hmm but installing the stix font package explicitly and reconfiguring matplotlib to use it works.  It could very well be a weird fontpath gotcha and I'm just not seeing down /usr/share/texmf for fonts or something.


stix fonts are installed in /usr/share/fonts/stix

hmmm

Can you try a local edit for me in your matplotlib.

/usr/lib64/python2.6/site-packages/matplotlib/font_manager.py
or
/usr/lib/python2.6/site-packages/matplotlib/font_manager.py

depending on your cpu arch.

Add /usr/share/texmf/fonts into the list X11FontDirectories

and see if that solves the problem. I bet it does.  And if it does. I can fix this in the packaging easily.

-jef

(In reply to comment #20)

Jef,

I tried what you recommended, but it didn't work.
Actually, of all the entries in the config file, only /usr/share/fonts contains fonts that can actually be used (the other directories are either empty or non-existent). 

I checked the Debian testing installation I have on a VM and the entries for the X11FontDirectories are exactly the same as Fedora's in font_manager.py. Debian also has actual fonts only in /usr/share/fonts. There are three folders there, X11, type1 and truetype. 

I tried copying the whole three folders to my Fedora machine in /usr/share/fonts and it also didn't work (math symbols are still messed up using $$ syntax). I think this must be some other configuration problem. If it was a font/no-font problem then copying all the files should have solved it.

cheers,
Cassio.

This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Jef,

Could you please bump this to F14?
It is still an issue for me.
Running:
1) plot()
2) xlabel("$x_1$")

now returns the following:

In [3]: /usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeOneSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeThreeSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeFourSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeFiveSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeTwoSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXNonUnicode'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=italic:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=bold:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmb10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmtt10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmmi10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmex10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmsy10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmr10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmss10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))

I tested these exactly same commands on Ubuntu 10.10 and it works as expected.
My guess is that this is an issue with fonts configuration in Fedora. I had yum install stix-* and tried adding mathtext.fontset : stix to matplotlibrc but it still does not work:

In [3]: /usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeOneSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeThreeSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeFourSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeFiveSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXSizeTwoSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['STIXNonUnicode'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=italic:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=bold:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)


The way to make this work without stix fonts and without spitting errors in F14 is to run:

1) rc('text', usetex=1)
2) plot()
3) xlabel("$x_1$")

While this works around the issue, the fonts in both axes become the ones used in latex, which can be avoided if the order of the commands is changed to:

1) plot()
2) rc('text', usetex = 1)
3) xlabel("$x_1$")

I think that while it is possible to make these workarounds, the $$ syntax should work without the need to issue the rc() command, as is done in other distros.

Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

I'm still not convinced this is a bug.

the matplotlib rc() commands work as part the documented instructions from upstream.

The question becomes what should the default for usetex actually be? If there is a good argument for us to set the default to something other than what upstream provides.. then its an argument to get the default changed in the upstream sources.

Why isn't usetex=1 the default in the upstream sources?

-jef

And another point.
I don't understand why the stix fonts work differently than the cm fonts with regards to $$ usage.    Are the cm font related files we ship malformed in some way?

-jef

Cassio,

Can you try doing a local patch for me.

In:  /whatever/site-packages/matplotlib/font_manager.py

Can you edit the call to fc-list  in get_fontconfig_fonts()

so that it reads like this:

status, output = commands.getstatusoutput("fc-list : file")


Does this fix the problem?


-jef

Jef, 

About this not being a bug: I agree with you that the rc() command is documented and "solves" this issue. However, as I explained in comment #21 in Debian this kind of behavior works perfectly without issuing the rc() command (and usetex=1 is not the default). I even tried copying the fonts to the fedora box but it didn't work. So I'm assuming this is some configuration problem in Fedora. If the rc() command should really be the only correct way to use Latex fonts, then I think that using the $$ command (without rc() before) should give an indication to run rc() first, not display incorrect fonts and give a bunch of warnings of fonts not being found. What do you think?

About the patch:

I don't have a commands symbol on my font_manager.py, so I changed it in this way:

substituted:
pipe = subprocess.Popen(['fc-list', '', 'file'], stdout=subprocess.PIPE)
for: 
pipe = subprocess.Popen(['fc-list : file'], stdout=subprocess.PIPE)

When I try to:

In [1]: plot()
Out[1]: []

In [2]: xlabel("$x_1$")
Out[2]: <matplotlib.text.Text object at 0xa507d2c>

I get:


In [3]: /usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXSizeOneSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1253: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXSizeThreeSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXSizeFourSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXSizeFiveSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXSizeTwoSym'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['STIXNonUnicode'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1253: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=italic:variant=normal:weight=normal:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1253: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=bold:stretch=normal:size=12.0. Returning /usr/share/fonts/thai-scalable/Waree-Oblique.ttf
  UserWarning)
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmb10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmtt10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmmi10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmex10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmsy10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmr10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib/python2.7/site-packages/matplotlib/font_manager.py:1243: UserWarning: findfont: Font family ['cmss10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))

even though:

[cassio@fedora ~]$ rpm -qa | grep stix
stix-fonts-1.0.0-1.fc14.noarch

Without this patch I get the same error.

No there is a bug in matplotlib associated with calling fclist and parsing the installed fonts to build a list of available fonts that matplotlib can then use. I am very sure of it. My patch fixes this on the matplotlib I have installed on my f13 system but my patch doesn't apply to the matplotlib version you are runnng. Completely different behaviour in the font manager stuff supplied by matplotlib when I make that adjustment. It could be our fclist that we ship is different in terms of argument handling than debian's.

What I need from you right now is for you to confirm for me which matplotlib you have installed and go give me as simple as possible non-interactive script that works as you expect on your debian based system. A non-interactive script I can add additional diagnostics python calls to so we can start seeing how matplotlib is parsing fonts in more detail ahead of when the tracebacks are being thrown.

The tracebacks aren't really helping at this point. The tracebacks are being produced by code that is expecting a certain font and are happening well after the font manager stuff calls attempts to build the list of available fonts. I'm pretty sure its the finding of fonts that is broken. 

-jef

Cassio,

I've confirmed that my patch for python-matplotlib-0.99.1.2-4.fc13 on two different F13 machines fixes the issue using a completely default set of config.  The call to fc-list as used in matplotlib is just busted of F13 and presumably F12.  My proposed patch fixes this on F13 and gets matplotlib parsing the fc-list output correctly.

Additionally on F-13 you need to install the lyx-fonts package to get the the crm fonts.  With my patch and that package the $$  syntax works using the test script in comment #1 as expected.  

Now F14 is a different story. It doesn't appear that F14 needs my fc-list patch at all. The subprocess call appears to be doing its just just fine. Revert your attempted patch. And in ipython run:

import matplotlib.font_manager
matplotlib.font_manager.get_fontconfig_fonts()

You'll see that the fontconfig stuff is working just fine.

This will return all the ttf fonts that fontconfig knows about.

So what is happening in F14 then? Maybe the get_fontconfig_fonts function isn't called at any point.


-jef

Cassio,

Got it!

Okay the deal is that matplotlib caches some fontconfig related stuff locally in your .matplotlib directory and those cached files can lie about the full state of the available font collection.

Note also that matplotlib prefers to uses its cached font files instead of calling get_fontconfig_fonts(). On F13 this function is busted. On F14 this function works just fin. Matplotlib only calls this to build a font list if the user's font config caches are missing.

Also note that just copying fonts into the system wide font directories may not be good enough to have fontconfig and therefore fc-list see them. You may need to run fc-scan with the correct arguments as root.  Font packages provided by fedora do this as part of their install routine to register newly installed fonts with the fontconfig system.  This is why your copy over from Debian didn't really work as you expected.


So after all that. Here is the F14 workaround that I confirmed on an F-14 system. 

On you F-14 system try this
install lyx-fonts  and make sure you are using an unmodified python-matplotlib F14 package.

delete or move your .fontconfig  and .matplotlib directories in your user account if they exist. 

Then try the test script from comment #1   
You'll see some of the warning messages but the cmr10 font from lyx-fonts package should be picked up and used to create the x/(1-x) y label correctly as part of the font family cascading to find a usable font.  


I will update the python-matplotlib packaging to require the lyx-font or stix-font package. but I also need to look into making a new fonts package that includes any additional fonts that matplotlib expects by default that appear in that warning list that we aren't already providing.  If I were to require one font for latex purposes to be installed for matplotlib $$ operation is the cmr10 from lyx-fonts good enough default?  stix will work fine too. And in F14 it looks like stix is preferred in the warning cascade.


-jef

Jef,

Superb job. Installing lyx-fonts and deleting the caches really solves the problem. The warning messages continue though, as you have pointed out.

As you stated, I think lyx-fonts should be a dependency too then.

About the font selection issue, I think that Computer Modern is the more "standard" latex font, so I'd personally prefer that.

Thanks for the hard work.

Cassio,


The works not done.  All we've done is identify the problem.

I could still use your help identifying the specific font files that you feel should be installed by default.  If you could figure out on your debian system exactly what file location of the font you are using there. That would help.

I might make a matplotlib-fonts subpackage and house those for now. Then get together with other tex related maintainers and figure out if we can dump a common set of files in to a common package.  

And more importantly. What I really need to figure out is how to teach matplotlib to test to see if system fonts have been updated and then regenerate its own caches accordingly and drive that logic into the upstream codebase.  Thoughts?

Simplest thing to do is to would be to force matplotlib to re-fetch the fontconfig stuff every run. But that's an expensive operation. What I really need is something I can timestamp check on the system and fire off the get_fontconfig_fonts()  function again only when that timestamp is updated because the system fontconfig caches have been updated.

Jef,

Ok. This is going to take some time. I'll post back when I have results.

But on a side note: I haven't looked at the code, but I don't think Debian has patched matplotlib to do that "dynamic discovery" of fonts you're talking about. Perhaps they're just installing the tex fonts in a place matplotlib already looks for, which solves this problem, but would not be a solution if you were trying to use a different, non-standard font.

About the timestamp check: Can't you just look at the modification date of .fontconfig and .matplotlib? I just did a echo "foo" > .fontconfig/bar and the date gets updated on the directory...

cheers.

Cassio,

Indeed having a standard collection of expected fonts on disk when the package is installed would hide the issue with caching from users. Doesn't mean its not an issue.  I'd really like to get to the point where I can have just a set of defaults installed and then users can install additional fonts via the packaging system and have it all just work...instead of lumping many fonts in as a hard require.

it's the systemwide fontconfig caches in particular that I need to watch... not the directories in the home user's space.  So for now I'm going to see if checking the mtime on /var/cache/fontconfig can be used to trigger a re-caching.

-jef

Hi Jef, I am back to working on the paper that originally prompted me to report this bug.

Here is a summary of where things stand for me:

if I use the 

mathtext.fontset : stix

fix in my mathplotlibrc, I get the mathtext to look right, but, as reported above, it now seems to use it everywhere.  

The trouble is that I can only generate PNG figures with this change.  If I try an EPS, it says:


Traceback (most recent call last):
  File "mach_plot.py", line 140, in <module>
    do_diagnostics()
  File "mach_plot.py", line 135, in do_diagnostics
    pylab.savefig("mach_384_rotation.eps")
  File "/usr/lib64/python2.6/site-packages/matplotlib/pyplot.py", line 356, in savefig
    return fig.savefig(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/figure.py", line 1032, in savefig
    self.canvas.print_figure(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/backend_bases.py", line 1476, in print_figure
    **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/backend_bases.py", line 1329, in print_eps
    return ps.print_eps(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/backends/backend_ps.py", line 855, in print_eps
    return self._print_ps(outfile, 'eps', *args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/backends/backend_ps.py", line 888, in _print_ps
    **kwargs)
  File "/usr/lib64/python2.6/site-packages/matplotlib/backends/backend_ps.py", line 1025, in _print_figure
    raise RuntimeError("OpenType CFF fonts can not be saved using the internal Postscript backend at this time.\nConsider using the Cairo backend.")
RuntimeError: OpenType CFF fonts can not be saved using the internal Postscript backend at this time.
Consider using the Cairo backend.


If I switch to the Cairo backend, and make a PS instead of EPS (since Cairo doesn't support EPS), I get an error tha Cairo doesn't support more than 18980 points.  I have atleast an order of magnitude more points to plot.

I've tried this with Fedora 12 and Fedora 14.  Only Fedora 10 works as desired.

Sadly, this is the main reason I am forced to keep my Fedora 10 box around, since EPS files were produced without error and mathtext just worked.  I'd love to know if you have any ideas on how to get around the crash with the normal backend and PS/EPS output, since that's what I need to a paper.  And I want to use the new features of matplotlib that are not in Fedora 10.

Thanks for the help.

As usual, I may have spoken too soon.  When I install lyx-fonts and remove the mathtext.fontset : stix from my mathplotlibrc, things work with the default backend on my F14 machine.  I'm not going to worry about Cairo if my plots look ok, and I also will not worry about F12 at all.

zingale,

Make sure you nuke both the .fontconfig caches and caches in your users .matplotlib directory any time you add fonts via the package system.  

One of the problems you can run into here is even if the fonts are correctly registered with fontconfig (as is the case with font packages in the fedora repositories)  matplotlib builds its own cache of font info using the fontconfig tools.  But matplotlib doesn't rebuild its local cache on every run.

I need to patch matplotlib's font handling to check the mtime of the system wide fontconfig cache and if that has been updated to rebuild the cache in your user's .matplotlib directory.

I just haven't found the cycles to put the full solution together.

For now install lyx-fonts manually and nuke .fontconfig caches and cached in .matplotlib in your home directory.

-jef

yes, this worked.  Thanks again for all your help on this.  F14 matplotlib now behaves as I would like/expect.

*** Bug 659891 has been marked as a duplicate of this bug. ***

I've been hit by this in Fedora 17 when plotting with logaxes. I have installed stix* fonts and lyx fonts and I'm still getting these warningd any time I plot:

findfont: Font family ['cmb10'] not found. Falling back to Bitstream Vera Sans
findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=12. Returning /usr/share/fonts/un-core/UnDotum.ttf
findfont: Font family ['cmtt10'] not found. Falling back to Bitstream Vera Sans
findfont: Font family ['cmb10'] not found. Falling back to Bitstream Vera Sans
findfont: Font family ['cmtt10'] not found. Falling back to Bitstream Vera Sans

Looking in my system, these fonts appear in the matplotlib source and in 

/usr/share/texmf/fonts/tfm/public/cm

but matplotlib is not looking for fonts in this directory. The output of  matplotlib.font_manager.get_fontconfig_fonts()

lists my $HOME/.fonts dir and /usr/share/fonts

matplotlib has this nasty habit of caching font list information in a users
.matplotlib/  resource directory.

If a cached list is found matplotlib reuses that list and does not rescan for new fonts. The scanning is somewhat expensive..hence the cache...but it seems matplotlib doesn't look at filesystem timestamps or anything like that to determine if a rescan is required.

examine your .matplotlib/   and remove the fontList.cache file and any other caches you find.  This will force matplotlib to rescan for fonts and recache.

-jef

 I have installed lyx-fonts and stix*, removed .matplotlib and .fontconfig, but when running the sample program

plot()
xlabel("$x_1$")

I get 

 /usr/lib64/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmb10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py:1252: UserWarning: findfont: Could not match :family=Bitstream Vera Sans:style=normal:variant=normal:weight=normal:stretch=normal:size=12. Returning /usr/share/fonts/lyx/cmr10.ttf
  UserWarning)
/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmtt10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))
/usr/lib64/python2.7/site-packages/matplotlib/font_manager.py:1242: UserWarning: findfont: Font family ['cmss10'] not found. Falling back to Bitstream Vera Sans
  (prop.get_family(), self.defaultFamily[fontext]))

and the thing is these fonts aren't in any rpm I'm aware of. If these fonts are not available in Fedora, why not patch matplotlib to avoid trying to load them?

Avoid trying to load fonts entirely because they are generating non-critical warnings? No...im not willing to carry that patch.

If those fonts do become available... and they should be available... cmb10 for example is part of the texlive-texmf-fonts I just don't know why matplotlib's font manager isn't detecting them on my system. Any chance you can figure out why matplotlib doesn't see those fonts?

So no.. I'm not patching out the warnings. I will accept patches which:

1) fix matplotlibs packaging that tex rendering works without installing additional fonts even if it continues to generate warnings.

2) fix matplotlibs font manager to see the texlive-texmf-fonts to avoid generating some warnings. 

3) reorders matplotlibs preferred font list to avoid generating the warnings at all.

Or if not patches... and effort to package missing fonts int the correct systemwide location...so they can be packaged according to the Fedora project font packaging guidelines (ie not part of the matplotlib payload.)  I'll take that and push it through the package review process and maintain that font package.

-jef

Perhaps matplotlib doesn't support reading font information from texlive-texmf-fonts...

The doc page of font_manager says

"The findfont() function returns the best TrueType (TTF) font file in the local or system font path that matches the specified FontProperties instance. The FontManager also handles Adobe Font Metrics (AFM) font files for use by the PostScript backend."

The fonts in texlive-texmf-fonts aren't TTF neither AFPM. We have 

cmb10.tfm: TeX font metric data (TeX text)
cmb10.pfb: PostScript Type 1 font program data

What's the difference between these different formats? No idea, I don't know a word a fonts...

By the way, in the same doc page there is this:

"Experimental support is included for using fontconfig on Unix variant platforms (Linux, OS X, Solaris). To enable it, set the constant USE_FONTCONFIG in this file to True. "

This variable is set to False currently. I have changed it to True in may local package but I don't see any difference. But perhaps it is worth to have a look what is doing exactly

This message is a notice that Fedora 14 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 14. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained.  At this time, all open bugs with a Fedora 'version'
of '14' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this 
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen 
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we were unable to fix it before Fedora 14 reached end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" (top right of this page) and open it against that 
version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping