Bug 1806272 - DejaVu has been repackaged in F32+ using new fonts packaging macros; those macros use an srpm-independent file layout and remove srpm-level sharing like the previous -common subpackage
Summary: DejaVu has been repackaged in F32+ using new fonts packaging macros; those ma...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: dejavu-fonts
Version: 33
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nicolas Mailhot
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1807317 (view as bug list)
Depends On: 1823360 1835134 1835195 1835486 1835489 1835491 1835493 1835495 1835498 1835499 1835502 1835503 1835504 1835506 1835507 1835509 1835510 1839164
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-23 13:27 UTC by Fabio Valentini
Modified: 2021-11-22 06:36 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-22 06:36:12 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Fabio Valentini 2020-02-23 13:27:18 UTC
With the transition to new forge-based fonts macros, the -common subpackage was dropped, but some packages depend on that. They are now not installable on fedora 32+ because that package is gone (only Obsoleted, not Provided).

This affects at least python3-weasyprint.

Additionally, the sdljava-demo package now has broken dependencies as well:

- /usr/share/fonts/dejavu/DejaVuSans-Bold.ttf
- /usr/share/fonts/dejavu/DejaVuSans-BoldOblique.ttf
- /usr/share/fonts/dejavu/DejaVuSans-Oblique.ttf
- /usr/share/fonts/dejavu/DejaVuSans.ttf

Probably those files were renamed with the forge macro transition.

Comment 1 Nicolas Mailhot 2020-02-23 14:15:23 UTC
Thanks for the report!

It is very weird to have anything depending on the common package, it only contained some common documentation files and a shared directory.

I suppose some packages depended on it for the directory structure, which, as you noted, is gone. It’s no use providing it, it does not exist anymore.

The new macros completely break the link between srpm name and directory structure, everything is based on the rpm name. Some font upstreams can’t make up their mind on how they release fonts, so the same font package may be generated from different srpms as time passes. Sometimes they flip flop between release styles depending on how releases upstream (sick, I know, complain to upstreams, not to me).

DejaVu Sans font files now live in /usr/share/fonts/dejavu-sans-fonts/ DejaVu Sans Mono in /usr/share/dejavu-sans-mono-fonts/ and so on, and there is no shared structure between the subpackages anymore, you don’t need to depend on anything except the exact font package you want.


No package should care about this except software which has not finished its migration to fontconfig yet (fontconfig: 2003). fontconfig hides file path changes for apps.


For major fonts like dejavu I try as packager to keep the paths stable within a release. So F32 is now set (just before freeze, the whole review process took a long time) and won’t change, and the new paths won’t be backported to F31. non-fontconfig software can symlink or configure them wherever they want in their private app space.

For other font families it’s way harder because upstreams can change file names every minor release (and some do change very often). Software is supposed to use solutions like fontconfig to avoid depending on exact file paths.


I’m a bit surprised about sdljava-demo because Java is supposed to use fontconfig to resolve fonts nowadays. I suppose it’s a leftover from SUN times

Comment 2 Fabio Valentini 2020-02-23 14:52:16 UTC
^ sorry about that, wrong tab.

Comment 3 Fabio Valentini 2020-02-23 14:58:23 UTC
@nim, thanks for the explanation!

I've added the maintainers of sdljava and and weasyprint to CC so that they are aware of these changes, and the rationale behind them.

Comment 4 Nicolas Mailhot 2020-02-26 07:26:32 UTC
*** Bug 1807317 has been marked as a duplicate of this bug. ***

Comment 5 Hans de Goede 2020-03-06 15:41:04 UTC
I have prepared a new sdljava-demo version using  Requires: font(dejavusans) instead of path Requires and I've adjusted the path based font lookup for the new layout. Note these are demos for using the SDL_ttf bindings in sdljava, so they do not use the java font APIs, instead they use a direct path to the font-file as games often do.

Comment 6 Hans de Goede 2020-03-06 20:30:39 UTC
I've just created an update for the fixed sdljava build in bodhi, so the sdljava side of this bug is fixed now:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-eab1dd33b5

I guess that the remaining python weasyprint issue really is an issue with that package, so maybe this bug should be reassigned to that package?

Comment 7 Nicolas Mailhot 2020-03-07 07:31:43 UTC
Thanks, I’ll reassign now, maybe that will get some traction weasyprint side

Comment 8 Hans de Goede 2020-03-07 14:59:32 UTC
Ugh, I just noticed that widelands (another game) is hit by the Dejavu font-file path changes in F32+ too. I'm fixing this now, but this made my wonder if others are affected to, a dnf repoquery --whatrequires dejavu-*-fonts results in the following list:

0ad-data-0:0.0.23b-4.fc32.noarch
dejavu-fonts-all-0:2.37-5.fc32.noarch
devedeng-0:4.16.0-2.fc32.noarch
e16-0:1.0.19-4.fc32.x86_64
e16-docs-0:0.16.8.0.2-18.fc32.noarch
emacs-1:26.3-2.fc32.x86_64
ember-media-0:0.7.2.1-11.fc32.noarch
enigma-0:1.21-17.20160222git0027b3b8e694.fc32.x86_64
fbida-0:2.14-8.fc32.x86_64
feh-0:3.2.1-2.fc32.x86_64
foobillard-0:3.0a-33.fc32.x86_64
gambas3-gb-sdl-0:3.14.3-2.fc32.x86_64
glob2-0:0.9.4.4-49.fc32.x86_64
gnubg-1:1.06.001-9.fc32.x86_64
gnuplot-0:5.2.8-2.fc32.x86_64
gravity-beams-and-evaporating-stars-0:1.0-4.fc32.x86_64
hedgewars-0:1.0.0-4.fc32.x86_64
htmldoc-0:1.9.7-2.fc32.x86_64
hyperrogue-0:10.0-9.d.fc32.x86_64
impressive-0:0.13.0-0.2.20190902svn275.fc32.noarch
kodi-common-0:18.5-2.fc32.x86_64
kosmtik-0:0.0.17-6.fc32.noarch
langpacks-core-font-af-0:3.0-2.fc32.noarch
<snip many more langpacks-core-font-*>
libprojectM-0:3.1.1-0.9.rc7.fc32.x86_64
libreoffice-core-1:6.4.1.2-1.fc32.x86_64
lincity-ng-data-0:2.9-0.20.20160605git7f266b1.fc32.noarch
mapnik-0:3.0.23-1.fc32.x86_64
mapserver-0:7.2.2-3.git7fe9b2b.fc31.x86_64
marsshooter-data-0:0.7.6-14.fc32.noarch
munin-0:2.0.54-3.fc32.noarch
mygui-0:3.2.2-13.fc32.x86_64
neverball-neverball-0:1.6.0-20.fc32.x86_64
neverball-neverputt-0:1.6.0-20.fc32.x86_64
perl-PDF-API2-0:2.037-1.fc32.noarch
php-tcpdf-dejavu-sans-fonts-0:6.3.5-1.fc32.noarch
php-tcpdf-dejavu-sans-mono-fonts-0:6.3.5-1.fc32.noarch
php-tcpdf-dejavu-serif-fonts-0:6.3.5-1.fc32.noarch
pioneer-data-0:20200203-1.fc32.noarch
pokerth-0:1.1.2-9.fc32.x86_64
python3-iep-0:3.7-16.fc32.noarch
python3-matplotlib-0:3.1.2-1.fc32.1.x86_64
python3-reportlab-0:3.5.34-3.fc32.x86_64
python3-tilestache-0:1.51.14-3.fc32.noarch
python3-weasyprint-0:51-2.fc32.noarch
python3-weasyprint-0:51-3.fc32.noarch
qcad-0:3.24.2.6-2.fc32.x86_64
retroarch-assets-0:1.8.4-4.fc32.noarch
retroarch-freeworld-assets-0:1.8.4-3.fc32.noarch
rrdtool-0:1.7.2-7.fc32.x86_64
scorched3d-0:44-21.fc32.x86_64
simspark-0:0.3.0-16.fc32.x86_64
sumwars-data-0:0.5.8-19.fc32.noarch
teeworlds-data-0:0.7.4-3.fc32.noarch
trackballs-0:1.1.4-36.fc32.x86_64
vdr-burn-0:0.3.0-16.fc32.x86_64
vdr-graphlcd-0:1.0.1-7.fc32.x86_64
vlc-1:3.0.9-33.fc32.x86_64
vodovod-0:1.10r22-15.fc32.x86_64
wesnoth-data-0:1.15.2-2.fc32.noarch
widelands-0:0-0.73.build20.fc32.x86_64
xblast-0:2.10.4-30.fc32.x86_64
xfce4-terminal-0:0.8.9.1-2.fc32.x86_64
xmoto-0:0.5.11-19.fc32.x86_64
xpilot-ng-data-0:4.7.3-20.fc32.noarch
zabbix-web-1:4.0.16-2.fc32.noarch

And I'm pretty sure various of these (at least most of the games in there) will likely have hardcoded paths, typically the game data includes a copy of some Dejavu*.ttf file and we replace it with a symlink which are now all broken.

Looking at this entire list, I really believe that we need to fix this at the dejavu packaging level, fixing all other packages is going to be a lot of work, esp. since it seems that there is no guarantee that this will not happen again in the future! So I'm changing the component back to dejavu-fonts for now.

One possible solution would be to do something like the following in %build of the fonts packages (example for the dejavu-sans-fonts pkg):

mkdir -p $RPM_BUILD_ROOT%{_datadir}/fonts/dejavu
for j in $(ls $RPM_BUILD_ROOT%{_datadir}/fonts/dejavu-sans-fonts); do
  ln -s /usr/share/fonts/dejavu-sans-fonts/$i $RPM_BUILD_ROOT%{_datadir}/fonts/dejavu
done

So that we get symlinks with the old filepath-s to the new locations, that seems like a much saner solution then patching all consumers of the old file-paths.

Comment 9 Nicolas Mailhot 2020-03-07 17:22:14 UTC
Hi Hans,

Quite a lot of those are perfectly healthy and do not need any change (libreoffice*, langpack*, dejavu-fonts-all*, probably e16* and xfce4-*).

They’re only using the Requires to:
– force the installation of the font family (Requires: font(dejavusans) would do the same with clearer intent), 
– or Fedora default fonts, we used to hide those in in comps groups, and users forgot to select in on custom installations: a Requires: font(:lang=en) would do the same today if the requirement is on default English fonts
– i18n should probably add a generic langpack-core-fonts-any or font(:any) Provides to all the langpack*font* packages so distro packages can depend on the system having been installed with some fonts (it sucks that the langpack* were not named with font plural using the existing Fedora naming convention) 

The only packages in trouble are those that depend on a specific hardcoded paths. Those paths will change every once in a while for Fedora or upstream reasons (font tech permits merging and splitting font files and font makers are making use of this possibility). And the solution to avoid those problems is to use fontconfig (that works for every need, except web fonts).

From a migration POW compat symlinking is useless because the affected package list will be the same in F34 when the compat package is retired. So, it’s a waste of time (if someone insists on it I will do it but it will be gone in F34).

If someone wants those symlinks maintained forever, just do so in another package, there’s no need to burden the general Fedora user with them (and I’m not interested as a packager). That will fail hard when upstream makes use of new OpenType features. OpenType feature changes usually result in a new font file split.

Comment 10 Hans de Goede 2020-03-07 18:17:53 UTC
Nicolas,

The dejavu font packaging changes are IMHO a really unnecessary / gratuitous change. The upstream files were not changed yet for "packaging" reasons and for "packaging" reasons only the directories in which the fonts are stored were stored causing a LOT, really really a LOT, of work for other packagers.

If this were a kernel level change Torvald's would be explaining to you how in the kernel it is not allowed to go around and break other people's stuff (break userspace) and you would likely be told to go and fix things ASAP.

I do not see how this is really different a change which you (or the font packaging group) made is causing a lot of breakage left and right. Since you broke it, *you *should also be the one to fix it.

At a minimum you should go over the above list, see which one use direct file-paths to the fonts (usually that is done straight in the spec file) and go and file bug against affected packages, so that we can track this and make sure that we get everything fixed before Fedora 32 ships. I suggest that you make all the bugs you file for this block this bug, so that we can use this bug as tracking bug. FWIW I've already fixed xblast, sdljava-demo and widelands, but that seems to be just the tip of the iceberg.

Also in the future please refrain from this kind of packaging changes for such a core font as Dejavu. If upstream changes the filenames then that is something which we will have to deal with, but this whole Fedora 32 change feels a lot like changing things around purely for the sake of changing things around, causing me and other packagers a lot of extra work and that makes me really unhappy!

Regards,

Hans

Comment 11 Hans de Goede 2020-03-07 19:01:44 UTC
Note that even with the following list removed from the list I posted earlier: "libreoffice*, langpack*, dejavu-fonts-all*, e16* and xfce4-*), there are still 59 pkgs on this list.

Taking this into account, I believe that this change really should have been following the changes-process, arguably even as a System-Wide change, but there is no change page for this at all; and clearly there has been no coordination with maintainers of packages which are affected by this change.

At this point in the cycle I believe that it is best to just revert the dejavu packaging changes done for Fedora 32 in Fedora 32 and in rawhide, with the exception of the dropping of the 'Provides: font(:lang=...)' provides.

And then have a proper discussion on how to move forward with this for Fedora, also involving the maintainers of other packages affected by this change in that discussion.

Comment 12 Nicolas Mailhot 2020-03-07 21:26:46 UTC
Hans,

Those are just symlinks. No different from a so bump. You don’t write change pages for so bumps.

The new packaging spent half a year in public review, first in copr, then FPC-side (4 months). It was announced multiple times on the devel list. 

It was (and still is) an insane amount of work. Reading arcane OpenType and CSS specification pages, reviewing and fixing hundreds of font files, almost an hundred specs, tens of thousands of fontconfig lines, pages of badly mangled asciidoc, code in rpm, shell, lua and python, appstream spec, etc.

You’re bikeshedding here because the amount of side-effects seen from your side are trivial enough you feel the whole thing is unnecessary / gratuitous.

Comment 13 Felix Schwarz 2020-03-07 21:55:12 UTC
(In reply to Nicolas Mailhot from comment #7)
> Thanks, I’ll reassign now, maybe that will get some traction weasyprint side

Weasyprint co-maintainer here: I believe I fixed the problem in weasyprint-51-3.fc32 by dropping the dependency on "dejavu-fonts-common" as part of bug 1810150.

weasyprint.spec has this commented right next to the requirement of dejavu-fonts-…: "Weasyprint will fail if no fonts are installed. There's no way to know what fonts the user would actually want, but require a few common ones that might be useful"
As the package also requires dejavu-sans-fonts, dejavu-sans-mono-fonts, dejavu-serif-fonts I think dropping -common is right choice.

Weasyprint uses fontconfig at runtime anyway. However I'm not familiar with Fedora's font packaging and I'm only maintaining the weasyprint package for a few months so I'd be glad to know if the current weasyprint spec file needs changes. :-)

Comment 14 Nicolas Mailhot 2020-03-07 22:17:43 UTC
(In reply to Felix Schwarz from comment #13)

> weasyprint.spec has this commented right next to the requirement of
> dejavu-fonts-…: "Weasyprint will fail if no fonts are installed. There's no
> way to know what fonts the user would actually want, but require a few
> common ones that might be useful"
> As the package also requires dejavu-sans-fonts, dejavu-sans-mono-fonts,
> dejavu-serif-fonts I think dropping -common is right choice. 
> Weasyprint uses fontconfig at runtime anyway. 

Then the -common dep was never needed in this spec (-common was a dep of the others when it existed :()

> However I'm not familiar with
> Fedora's font packaging and I'm only maintaining the weasyprint package for
> a few months so I'd be glad to know if the current weasyprint spec file
> needs changes. :-)

The “will fail if no fonts are installed” case could not be handled gracefully in the past because default font groups existed solely in comps. Now that we have proper default metapackages I believe depending on 'font(:lang=en)' would work better for you in current Fedora releases. Though I don’t like much hardcoding a specific script. langpack-core-fonts-any or font(:any) would be even better if those were available

@parag: how would i18n prefer to handle this case?

Comment 15 Parag Nemade 2020-03-08 03:25:21 UTC
Sorry if I confused you but let me clear the confusion on my usage of email addresses. For only Fedora package review work, I use my @gmail email id. For all other kind of bugs/issues I can be addressed using @redhat email id. I will read this bug and will reply soon.

Comment 16 Nicolas Mailhot 2020-03-08 06:51:58 UTC
@Parag no problem, thanks for the clarification, the confusion only existed because you do an awful lot of work in Fedora (which is very nice, thank you)

Comment 17 Hans de Goede 2020-03-08 10:10:09 UTC
(In reply to Nicolas Mailhot from comment #12)
> Those are just symlinks. No different from a so bump. You don’t write change
> pages for so bumps.

so-bumps are automatically picked up by dependent packages with just a rebuild, that is not the case here. Also in case of so-bumps the person doing the so-bump typically also does the rebuilds of dependent packages.

I seem to be spending most of my working hours lately on fixing breakage caused by others and I'm totally fed up with having to keep fixing things which other people broke!

My vision here is really clear: you broke it, you fix it!

I've proposed 3 different ways how *you* can fix the things which *you* broke:

1. Check which packages are affected and file bugs against them, IOW do the minimum amount of coordination so that other people at least know that there is breakage which needs to be fixed

2. Provide compat symlinks with the old file-paths to the files

3. Revert the part of the changes where the directory in which the font files are stored changes. I'm fine with all the other changes, the one causing the problem is that the directory in which the files are stored has changed.

And I'll add a 4th option:

4. By far most packages which Require dejavu-*-fonts seem to Require the dejavu-sans-fonts package, so how about adding an /usr/share/fonts/dejavu symlink to /usr/share/fonts/dejavu-sans-fonts, this will fix the issues for apps relying on the file-path to the fonts in most cases. This reduces the list of apps which needs to be checked and potentially fixed because they rely on the serif or mono version to 24 pkgs, which is a lot better then 59.

So let me repeat myself: you broke it, you fix it, so please select one of the options from above (I suggest combining 1 + 4) and let me know how you plan to proceed with fixing this.

Comment 18 Parag Nemade 2020-03-09 05:19:19 UTC
(In reply to Nicolas Mailhot from comment #14)
> 
> > However I'm not familiar with
> > Fedora's font packaging and I'm only maintaining the weasyprint package for
> > a few months so I'd be glad to know if the current weasyprint spec file
> > needs changes. :-)
> 
> The “will fail if no fonts are installed” case could not be handled
> gracefully in the past because default font groups existed solely in comps.
> Now that we have proper default metapackages I believe depending on
> 'font(:lang=en)' would work better for you in current Fedora releases.
> Though I don’t like much hardcoding a specific script.
> langpack-core-fonts-any or font(:any) would be even better if those were
> available
> 
> @parag: how would i18n prefer to handle this case?


we have langpacks-core-fonts-<langcode> packages to use. They provide pre-defined default font for that <langcode>. If packages want to use particular font package they need to pull it explicitly. But if there is no such particular font demand then default langpacks-core-font-<langocode> can be used.

I am still exploring how much new guidelines are compatible with older packaging. (I was on vacation last few days) I see fonts-rpm-macros package already provided older macros as well so old packages can be built using new fonts-rpm-macros package. Yesterday I first tried to use new guidelines and convert some fonts packages and there I found about change in directory structure and fontconfig file format. Converting existing fonts packages to new packaging guidelines will definitely take some time due to these changes. Right now I am trying to understand how to write fontconfig xml files for new packaging guidelines.

If I missed to address any question here please ask again.

Comment 19 Nicolas Mailhot 2020-03-09 08:16:50 UTC
(In reply to Parag Nemade from comment #18)

> we have langpacks-core-fonts-<langcode> packages to use. They provide
> pre-defined default font for that <langcode>. If packages want to use
> particular font package they need to pull it explicitly. But if there is no
> such particular font demand then default langpacks-core-font-<langocode> can
> be used.

Sure, but here we have the case where <langcode> is not defined (the package wants some default fonts installed, it does not care which ones, as long as some default is installed).
What dep should the package use in this case? I can add the generic answer to this question to guidelines, but I need an answer i18n, as maintainer of the distribution default font lists, is comfortable with

> Yesterday I first tried to use new guidelines and
> convert some fonts packages and there I found about change in directory
> structure and fontconfig file format. 

The new fontconfig format is not part of the guidelines change, you can use the usual format, and the usual format is the one documented in guidelines and default templates. Just set %fontconfs not %fontconfsngs.

The new format exists in my own packages as a stopgap demonstrator because Akira wanted proof the current fontconfig format does not scale to current needs. Also, there was no way in hell I could write and check and proof and maintain by hand the multi-hundred/multi-thousand lines of configuration required by some of the packaged fonts. Lastly, the current syntax does not work at all unless you have perfect knowledge of the styles that will be declared by your font files, so it needs brittle workarounds in %install to generate a conf file that will only work with the exact files installed.

Simple font packages can be handled with the legacy syntax (though it is overly verbose and painful). Complex (rich fonts) like Plex Sans, Merriweather, Droid, Noto, etc are quite impossible to maintain without something like it.

Also, I don’t intend to maintain the new syntax generator long term. Fontconfig syntax fixes belong in fontconfig. The generator is already way more complex than I imagined at first, because font makers are behaving way worse than, making current fontconfig syntax that more inadequate and harder to workaround outside the engine itself.

Comment 20 Hans de Goede 2020-03-09 08:42:20 UTC
Can someone please reply to my comment 17 ?   I would like to resolve this amicably and not start a long thread about this on the fedora-devel list...

Comment 21 Jens Petersen 2020-03-09 12:09:49 UTC
(In reply to Hans de Goede from comment #17)
> 4. By far most packages which Require dejavu-*-fonts seem to Require the
> dejavu-sans-fonts package, so how about adding an /usr/share/fonts/dejavu
> symlink to /usr/share/fonts/dejavu-sans-fonts, this will fix the issues for
> apps relying on the file-path to the fonts in most cases. This reduces the
> list of apps which needs to be checked and potentially fixed because they
> rely on the serif or mono version to 24 pkgs, which is a lot better then 59.

Particularly for F32, considering how late this "Change" landed,
adding compat symlinks for dejavu at least seems a reasonable request to me.


(In reply to Nicolas Mailhot from comment #19)
> Sure, but here we have the case where <langcode> is not defined (the package
> wants some default fonts installed, it does not care which ones, as long as
> some default is installed).
> What dep should the package use in this case? I can add the generic answer
> to this question to guidelines, but I need an answer i18n, as maintainer of
> the distribution default font lists, is comfortable with

Well as of F32 'font(:lang=en)' is provided by langpacks-core-font-en, which will pull in dejavu-sans-fonts.
I think currently that is the best option/answer (not that we want to privilege 'en' particularly...
basically for all major European languages/scripts I believe we default to dejavu).
We can think about a better/more correct answer for F33 perhaps, but it is tricky.
In terms of orthography English is really the lowest common denominator I think.
Better take this to a separate discussion: feel free to open a langpacks bug to discuss this more.

Comment 22 Hans de Goede 2020-03-09 14:08:10 UTC
p.s. One important thing which I would like to add here: Despite all my grumbling about the file-path changes I do really appreciate all the hard work which the font SIG is doing for Fedora.

Comment 24 Hans de Goede 2020-03-09 20:26:07 UTC
Thank you for adding the compat packages.

I must be honest though, this is not entirely what I was expecting. I was expecting the compat symlinks to be part of the main dejavu-sans-fonts, dejavu-serif-fonts, etc. packages. That way they will automatically just be there, without packages depending on say dejavu-sans-fonts needing to adjust their Requires. If someone needs to go my specfile changes to adjust the Requires (and do a build and create an update in bodhi) they might just as well fix the paths and be done with it.

I guess the compat packages are still useful though as a quick fix/workaround which users can install themselves in case of pkgs where no one has gotten around to fixing them.

Talking about fixing packages. I've spend some time thinking about how to fix things. Most affected packages are games, typically using SDL[2]_ttf which requires a direct file-path to a ttf file. Typically the fonts are bundled together with other game-assets, living under /usr/share/<game-name>/font.ttf .  The Fedora pkgs replace font.ttf with a symlink, either because the font was removed because it was non-free, or to use system-fonts instead of shipping a private copy.

For these games my plan is to add e.g.:

BuildRequires:  font(dejavusans) fontconfig

To the packages and then generate the name the symlink points to using commands like these:

  fc-match -f "%{file}" "sans"
  fc-match -f "%{file}" "sans:italic"
  fc-match -f "%{file}" "sans:bold"
  fc-match -f "%{file}" "sans:bold:italic"
  fc-match -f "%{file}" "serif"
  fc-match -f "%{file}" "monospace"

The idea here is to make things more future proof, so that we can deal with any future font-file-path changes with just a rebuild.

Before I spend a chunk of time on fixing various packages, I would appreciate your feedback on this approach. Do you think that this is a good idea? Any suggestions wrt the fc-match commands?

Comment 25 Hans de Goede 2020-03-09 20:35:26 UTC
I've gone over the list which I posted in comment 8. From this I've come up with 2 lists which I plan to take action on once I've some feedback on the approach which I suggested in comment 24.

I plan to file bugs against these packages, asking them to check if they need to adjust their package for the file-path changes and adding some example specfile bits for how this can be done in a future proof way using fc-match:

xpilot-ng-data-0:4.7.3-20.fc32.noarch
0ad-data-0:0.0.23b-4.fc32.noarch
wesnoth-data-0:1.15.2-2.fc32.noarch
retroarch-assets-0:1.8.4-4.fc32.noarch
retroarch-freeworld-assets-0:1.8.4-3.fc32.noarch
neverball-neverball-0:1.6.0-20.fc32.x86_64
neverball-neverputt-0:1.6.0-20.fc32.x86_64
teeworlds-data-0:0.7.4-3.fc32.noarch
foobillard-0:3.0a-33.fc32.x86_64
hedgewars-0:1.0.0-4.fc32.x86_64
gravity-beams-and-evaporating-stars-0:1.0-4.fc32.x86_64
glob2-0:0.9.4.4-49.fc32.x86_64
lincity-ng-data-0:2.9-0.20.20160605git7f266b1.fc32.noarch
pioneer-data-0:20200203-1.fc32.noarch

And since I maintain them (or there does not seem to be an active maintainer) I will try to take care of fixing these myself:

marsshooter-data-0:0.7.6-14.fc32.noarch
trackballs-0:1.1.4-36.fc32.x86_64
vodovod-0:1.10r22-15.fc32.x86_64
sumwars-data-0:0.5.8-19.fc32.noarch
scorched3d-0:44-21.fc32.x86_64
enigma-0:1.21-17.20160222git0027b3b8e694.fc32.x86_64
hyperrogue-0:10.0-9.d.fc32.x86_64

Note these are all games and such these are likely all cases using direct file-paths (usually through symlinks) so these have a high likelyhood of really needing to be fixed. If you look at the list from comment 8, then you will see that there are a bunch of other packages there. If you suspect that any of the other packages need to be fixed too, then please file a bug against them for this.

Comment 26 Nicolas Mailhot 2020-03-09 20:59:25 UTC
(In reply to Hans de Goede from comment #24)
> Thank you for adding the compat packages.
> 
> I must be honest though, this is not entirely what I was expecting. I was
> expecting the compat symlinks to be part of the main dejavu-sans-fonts,
> dejavu-serif-fonts, etc. packages. That way they will automatically just be
> there, without packages depending on say dejavu-sans-fonts needing to adjust
> their Requires.

https://src.fedoraproject.org/rpms/dejavu-fonts/blob/f32/f/dejavu-fonts.spec#_151

> For these games my plan is to add e.g.:
> 
> BuildRequires:  font(dejavusans) fontconfig
> 
> To the packages and then generate the name the symlink points to using
> commands like these:
> 
>   fc-match -f "%{file}" "sans"
>   fc-match -f "%{file}" "sans:italic"
>   fc-match -f "%{file}" "sans:bold"
>   fc-match -f "%{file}" "sans:bold:italic"
>   fc-match -f "%{file}" "serif"
>   fc-match -f "%{file}" "monospace"
> 
> The idea here is to make things more future proof, so that we can deal with
> any future font-file-path changes with just a rebuild.

It will be a little more future-proof, but not as much as you think. Fonts are evolving outside the one style = one file model. Mostly, due to the influence of web fonts, because CSS uses a stacking model, identical to the one in fontconfig. Styles can now be spread over multiple files (that is already true for the Math extension of DejaVu Serif) or, on the contrary, be merged into a single file (variable fonts).

And I’ve no idea when exactly the legacy model used before fontconfig will become totally incompatible with the distribution default font choices, but the movement has been accelerating, thanks to the huge amount of money Google pumped in web fonts, which is breaking the Windows fonts deadlock right now. Companies now require their design offices to create web sites and mobile apps, which forces those to drop legacy proprietary fonts for OFL fonts. You have more chance to bump into Fortune500 designers in github’s font issue trackers nowadays, than to bump into someone who knows what fontforge or Fedora are.

Comment 27 Hans de Goede 2020-03-10 09:17:32 UTC
(In reply to Nicolas Mailhot from comment #26)
> (In reply to Hans de Goede from comment #24)
> > Thank you for adding the compat packages.
> > 
> > I must be honest though, this is not entirely what I was expecting. I was
> > expecting the compat symlinks to be part of the main dejavu-sans-fonts,
> > dejavu-serif-fonts, etc. packages. That way they will automatically just be
> > there, without packages depending on say dejavu-sans-fonts needing to adjust
> > their Requires.
> 
> https://src.fedoraproject.org/rpms/dejavu-fonts/blob/f32/f/dejavu-fonts.
> spec#_151

Ah I looked over the Supplements line, cool. Thank you this will give us a whole cycle to deal with the fallout of this, much appreciated.

> > The idea here is to make things more future proof, so that we can deal with
> > any future font-file-path changes with just a rebuild.
> 
> It will be a little more future-proof, but not as much as you think. Fonts
> are evolving outside the one style = one file model. Mostly, due to the
> influence of web fonts, because CSS uses a stacking model, identical to the
> one in fontconfig. Styles can now be spread over multiple files (that is
> already true for the Math extension of DejaVu Serif) or, on the contrary, be
> merged into a single file (variable fonts).

Ok I understand, if:

BuildRequires:  font(dejavusans) fontconfig

+

fc-match -f "%{file}" "sans"

Ever results in the second command suggesting a font which is not suitable as a default font for "legacy" ttf users, then I guess we will need to come up with a different way of handling this at that point in time. Since we do not know what the font situation will exactly look like then, I don't think that speculating on this now is all that useful. So for now I'll just go with the 'ln -s $(fc-match -f "%{file}" "sans") $RPM_BUILD_ROOT%{_datadir}/%{name}/font.ttf' approach.

Comment 28 Miro Hrončok 2020-03-19 11:17:31 UTC
Suplemenets are not guaranteed to be there and are not pulled in with BuildRequires.

Can we just please make the symbolic links part of the main packages? Than we can coordinate their removal with all dependent packages.

Comment 29 Solomon Peachy 2020-03-19 11:35:57 UTC
While doing an upgrade from F31 to F32 beta, I ran into a problem:

[...]
Running transaction test
The downloaded packages were saved in cache until the next successful
transaction.
You can remove cached packages by executing 'dnf clean packages'.
Error: Transaction test error:
  file /usr/share/widelands/i18n/fonts/DejaVu from install of
widelands-0-0.76.build20.fc32.x86_64 conflicts with file from package widelands-0-0.72.build20.fc31.x86_64
  file /usr/share/fonts/dejavu/DejaVuSans-Bold.ttf conflicts between attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and widelands-0-0.76.build20.fc32.x86_64
  file /usr/share/fonts/dejavu/DejaVuSans-BoldOblique.ttf conflicts between attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and widelands-0-0.76.build20.fc32.x86_64
  file /usr/share/fonts/dejavu/DejaVuSans-ExtraLight.ttf conflicts between attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and widelands-0-0.76.build20.fc32.x86_64
  [...and many many more font conflicts]

On fedora-devel I was told this is a packaging bug with dejavu-fonts (not widelands) and to speak up in this ticket..

Comment 30 Hans de Goede 2020-03-19 12:24:55 UTC
(In reply to Solomon Peachy from comment #29)
> While doing an upgrade from F31 to F32 beta, I ran into a problem:
> 
> [...]
> Running transaction test
> The downloaded packages were saved in cache until the next successful
> transaction.
> You can remove cached packages by executing 'dnf clean packages'.
> Error: Transaction test error:
>   file /usr/share/widelands/i18n/fonts/DejaVu from install of
> widelands-0-0.76.build20.fc32.x86_64 conflicts with file from package
> widelands-0-0.72.build20.fc31.x86_64
>   file /usr/share/fonts/dejavu/DejaVuSans-Bold.ttf conflicts between
> attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and
> widelands-0-0.76.build20.fc32.x86_64
>   file /usr/share/fonts/dejavu/DejaVuSans-BoldOblique.ttf conflicts between
> attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and
> widelands-0-0.76.build20.fc32.x86_64
>   file /usr/share/fonts/dejavu/DejaVuSans-ExtraLight.ttf conflicts between
> attempted installs of compat-f32-dejavu-sans-fonts-2.37-7.fc32.noarch and
> widelands-0-0.76.build20.fc32.x86_64
>   [...and many many more font conflicts]
> 
> On fedora-devel I was told this is a packaging bug with dejavu-fonts (not
> widelands) and to speak up in this ticket..

The problem here seems to be replacing the Dejavu symlink to the dejavu directory with a Dejavu directory with symlinks to the individual font files. So we are hitting: https://docs.fedoraproject.org/en-US/packaging-guidelines/Directory_Replacement/
here and according to that page, this is a widelands problem and the fix is to remove the symlink in a %pretrans script. I'm working on a fixed package for this now.

Comment 31 Hans de Goede 2020-03-19 19:41:04 UTC
(In reply to Hans de Goede from comment #30)
> The problem here seems to be replacing the Dejavu symlink to the dejavu
> directory with a Dejavu directory with symlinks to the individual font
> files. So we are hitting:
> https://docs.fedoraproject.org/en-US/packaging-guidelines/
> Directory_Replacement/
> here and according to that page, this is a widelands problem and the fix is
> to remove the symlink in a %pretrans script. I'm working on a fixed package
> for this now.

Ok, so the tricks from:

https://docs.fedoraproject.org/en-US/packaging-guidelines/Directory_Replacement/

Do not work. Instead I've kept the symlinks and made them point to a new dir under /usr/share/widelands/i18n/fonts, e.g. I made the "DejaVu" symlink point be a relative link to "dejavu-fonts" which is a new dir under /usr/share/widelands/i18n/fonts and this new dir then contains links to all the DejaVuSerif  DejaVuSans and  DejaVuSansMono ttf files.

I've started a new widelands build with these changes includes, once this hits the stable updates repo, the upgrade problem should be fixed (upgrading works for me using a mockbuild I did to test).

Comment 32 Hans de Goede 2020-03-19 19:43:58 UTC
(In reply to Miro Hrončok from comment #28)
> Suplemenets are not guaranteed to be there and are not pulled in with
> BuildRequires.
> 
> Can we just please make the symbolic links part of the main packages? Than
> we can coordinate their removal with all dependent packages.

Actually not having these automatically added to buildroots is a feature IMHO. I'm switching all my packages which use direct file-path accesses to fonts (e.g. games using SDL_ttf) to use fc-match to generate the filename during the build and then replace the bundled fonts with a symlink generated this way. If we get the compat symlinks added then fc-match will prefer the compat-path and print that, causing potential troubles / need for a rebuild if we remove the compat symlinks later.

Comment 33 Hans de Goede 2020-03-19 20:37:50 UTC
Here is the update with the fixed widelands build:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-324e223a8d

Comment 34 George R. Goffe 2020-03-21 04:46:47 UTC
I just got this when I did a dnf upgrade...

Error: Transaction test error:
file /usr/share/widelands/i18n/fonts/DejaVu from install of widelands-0-0.77.build20.fc33.x86_64 conflicts with file from package widelands-0-0.76.build20.fc33.x86_64

Comment 35 Hans de Goede 2020-03-21 15:13:48 UTC
(In reply to George R. Goffe from comment #34)
> I just got this when I did a dnf upgrade...
> 
> Error: Transaction test error:
> file /usr/share/widelands/i18n/fonts/DejaVu from install of
> widelands-0-0.77.build20.fc33.x86_64 conflicts with file from package
> widelands-0-0.76.build20.fc33.x86_64

Ah yes, that is the result of the changes I made to fix the upgrade path from F31 to F32, that will break the upgrade path for people who were running the old unfixed F32 version, see comment 31 for details.

Unfortunately I cannot fix both upgrade paths and since F32 is not yet released I believe that the upgrade path from F31 -> F32 is the important one to get right.

To workaround this on F32 beta do:

sudo rpm -e widelands
sudo dnf install widelands

Comment 36 George R. Goffe 2020-03-22 04:58:19 UTC
Hans,

Cool. Thanks for responding.

George...

Comment 37 Ben Cotton 2020-08-11 13:09:25 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 33 development cycle.
Changing version to 33.

Comment 38 Ben Cotton 2021-11-04 17:40:55 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 39 Parag Nemade 2021-11-22 06:36:12 UTC
I believe all the reported issues here are already fixed in Fedora.


Note You need to log in before you can comment on or make changes to this bug.