Bug 1377367 - fontconfig cache in /var incompatible with ostree model; unable to `rpm-ostree install emacs`
Summary: fontconfig cache in /var incompatible with ostree model; unable to `rpm-ostre...
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: fontconfig
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Akira TAGOH
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1456278 (view as bug list)
Depends On:
Blocks: 1352154 1377369
TreeView+ depends on / blocked
 
Reported: 2016-09-19 13:38 UTC by Micah Abbott
Modified: 2017-05-29 14:00 UTC (History)
14 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1377369 (view as bug list)
Environment:
Last Closed: 2017-02-24 01:46:32 UTC


Attachments (Terms of Use)

Description Micah Abbott 2016-09-19 13:38:27 UTC
# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/24/x86_64/docker-host
       Version: 24.45 (2016-09-18 01:58:21)
        Commit: b1aaee15d913332740bb3badd493fe3ac9e2cc22914833481e2efca5cc48580b
        OSName: fedora-atomic

  fedora-atomic:fedora-atomic/24/x86_64/docker-host
       Version: 24.43 (2016-09-15 22:18:06)
        Commit: b46e283bd4defb10757c4343ff4627f50ba10766a33b8f8c0a95b2f8824cdcb5
        OSName: fedora-atomic

# rpm-ostree pkg-add emacs

Downloading metadata: [========================] 100%
Resolving dependencies... done
Will download: 115 packages (107.9 MB)

  Downloading from updates: [===========================] 100%

  Downloading from fedora: [=========================] 100%

Importing: [==============================] 100%
Checking out tree b1aaee1... done
Overlaying... done
Running %post for gdk-pixbuf2...... done
Running %post for fontconfig...... error: Running %post for fontconfig: Executing bwrap: Child process exited with code 5

Comment 1 Colin Walters 2016-10-28 14:57:59 UTC
For this we need to move the fontconfig cache to /usr.  See https://git.gnome.org/browse/gnome-continuous/tree/manifest.json?id=6d10d44c7f656a2ded923e8c61c4b0c4465a3d10#n170

Comment 2 Matt Hirsch 2016-12-12 21:07:15 UTC
I notice that this is still a problem. Also in F25 Atomic. I want to install the nvidia binary driver from rpmfusion, and fontconfig is a dependency. Is there a testing version of this package available somewhere built with the cache directory under /usr?

Comment 3 Akira TAGOH 2016-12-13 03:56:36 UTC
One concern is, that proposed directory, /usr/lib/fontconfig/cache is writable from the POV of SELinux? and possible slowness on updating caches for people who mount /usr through nfs?

Comment 4 Colin Walters 2016-12-13 15:12:04 UTC
RPM scripts are fully privileged and can write to /usr.  And we don't support NFS /usr anymore: https://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken/

Comment 5 Akira TAGOH 2016-12-19 05:56:55 UTC
Sure. maybe I should open a change proposal for f26 and get more attentions on it.

Comment 6 Jonathan Underwood 2016-12-21 10:25:50 UTC
Unfortunately this proposal isn't compatible with the FHS requirements of /usr, which is supposed to be read only.

Comment 7 Jonathan Underwood 2016-12-21 10:27:14 UTC
.. by "read only" I really mean static rather than dynamically generated. /var/cache really is the right location for this sort of thing.

Comment 8 Vít Ondruch 2016-12-21 11:48:02 UTC
(In reply to Jonathan Underwood from comment #6)
> Unfortunately this proposal isn't compatible with the FHS requirements

Agree with this. Can't see how /usr could be the right choice.

Comment 9 Colin Walters 2016-12-21 14:09:11 UTC
There's many other derived caches in /usr.  gtk-update-icon-cache is one offhand.
GNU info pages (`install-info`).  kernel modules (depmod -a).

Remember that in the ostree model:
https://ostree.readthedocs.io/en/latest/manual/adapting-existing/
Things like the RPM database are also moved to /usr/share/rpm.

OSTree also *enforces* that /usr is read-only at runtime - but with rpm-ostree, we construct a new tree on the client side, and run %posts there with it mutable, then it's sealed back up at runtime.

An important thing to keep in mind with the FHS is that they tried to push for having /usr be NFS mountable, which Fedora (and all systemd systems), don't support.  It's just not relevant - it's easier to NFS mount /, and do something like ostree on the server to dedup the /usr content between different hosts.

Basically, the FHS requirements are from a previous mindset - the https://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken/ merge pushed the semantics for /usr in a different direction, and OSTree goes much farther with it.

The main concern I have here about breaking things is more around upgrades for people who are using yum/dnf on the host.  I could probably change rpm-ostree to include an e.g. `OSTREE_SEMANTICS=1` environment variable, and the fontconfig %post could key off that.

Comment 10 Colin Walters 2016-12-21 14:12:11 UTC
Rather than thinking of the above things as "caches", they're more like "indexes" or "lookup tables".  The distinction is that they're *only* written or generated during a software installation.

A good example of another one of these to move to /usr would be /etc/ld.so.cache.

Comment 11 Matthew Miller 2016-12-21 17:17:14 UTC
Rather than moving the cache around -- or, I guess, in addition to it, but obviating an concern about FHS or dyanamic, unmanaged files in /usr, could we just pregenerate the cache files at _build_ time rather than install time? They appear to be arch specifc (well, word size and order specific) but not really host-specific.

Bonus: faster system install (when not using ostree; I guess faster ostree generation too).

Comment 12 Matthew Miller 2016-12-21 21:54:29 UTC
I agree with Colin's assessment that the FHS doesn't really cover this situation well. From the section on /var:

   /var is specified here in order to make it possible to mount /usr
   read-only. Everything that once went into /usr that is written to
   during system operation (as opposed to installation and software
   maintenance) must be in /var.

But here, we're talking about stuff that is written specifically and only during "installation and software maintenance". That's just not well-accounted-for in the spec. Generate-at-build-time as I suggest above might (I hope) work for fontconfig, but it won't for /etc/ld.so.cache -- and I guess that living in /etc may be a reflection of the above problem.

Comment 13 Jonathan Underwood 2016-12-21 22:57:03 UTC
OK, I mostly buy that. But, I think you're arguing that this is data for fontconfig (and applications using it), so it might more naturally fit under /usr/share/fontconfig than /usr/lib/fontconfig. Right?

Comment 14 Akira TAGOH 2016-12-22 01:39:01 UTC
(In reply to Matthew Miller from comment #11)
> Rather than moving the cache around -- or, I guess, in addition to it, but
> obviating an concern about FHS or dyanamic, unmanaged files in /usr, could
> we just pregenerate the cache files at _build_ time rather than install
> time? They appear to be arch specifc (well, word size and order specific)
> but not really host-specific.

At this moment, unfortunately the updates at the install time are required to update the directory caches. there are two types of the caches in fontconfig. one is to contain a font meta data to reduce the costs to iterate a font, which could be generated at the build time (for Fedora at least. generally speaking, maybe not, because the unit of the cache are per-directories not per-files). one is to contain a directory meta data to reduce the costs to iterate files in a directory and detect the updates, which is likely to be changed at the install time.

Comment 15 Jens Petersen 2016-12-26 02:15:00 UTC
(BTW how about using emacs-nox instead or is this an Atomic Workstation?)

Comment 16 Jens Petersen 2016-12-26 02:23:42 UTC
Matthew: do you mean Atomic image build-time?

Comment 17 Matthew Miller 2016-12-26 22:39:03 UTC
(In reply to Jens Petersen from comment #16)
> Matthew: do you mean Atomic image build-time?

No, I meant at package build time, but that's no help if it's per-directory rather than per font.

Comment 18 Nicolas Mailhot 2016-12-27 10:18:59 UTC
Well, Fedora font package are one directory per srpm, it would probably not be too hard to modify the macro to be one package by rpm (the hard part being filesystem cration which is not automated today). And then rebuild all font packages.

Though, of course, this may harm fontconfig performance. Not sure it it like too many directories. And it would transform noarch packages to arch packages which would be expensive for mirrors, given how bulky unicode fonts are.

On the plus side, installing noto would not kill systems performance for minutes anymore.

Comment 19 Akira TAGOH 2016-12-27 11:15:14 UTC
There were some discussion in upstream about the cache updates at the runtime these days though, disabling the runtime cache updating may be one of solution for this and build the cache at the build time. the caches won't be updated unless the system administrators runs fc-cache with their privileges.

Comment 20 Colin Walters 2017-02-03 14:47:58 UTC
One thing that makes this more urgent is that libvirt -> qemu indirectly pulls in fontconfig (and tons of desktop libraries), because qemu links to gtk for bad reasons.

Comment 21 Colin Walters 2017-02-03 14:48:48 UTC
(Urgency/priority here meaning for Fedora Atomic Host, I'd like to support installing libvirt.  The workstation is another thing)

Comment 22 Matthew Miller 2017-02-03 14:49:50 UTC
(In reply to Colin Walters from comment #20)
> One thing that makes this more urgent is that libvirt -> qemu indirectly
> pulls in fontconfig (and tons of desktop libraries), because qemu links to
> gtk for bad reasons.

Bad and hard to fix, or bad and fixable reasons?

Comment 23 Daniel Berrangé 2017-02-03 15:15:32 UTC
(In reply to Matthew Miller from comment #22)
> (In reply to Colin Walters from comment #20)
> > One thing that makes this more urgent is that libvirt -> qemu indirectly
> > pulls in fontconfig (and tons of desktop libraries), because qemu links to
> > gtk for bad reasons.
> 
> Bad and hard to fix, or bad and fixable reasons?

Bad and hard to fix I'm afraid :-(

QEMU has multiple different frontends it builds support for, SDL, GTK, VNC and SPICE. While users who run VMs via libvirt will always use VNC or SPICE, users who run VMs without libvirt will typically use the GTK or SDL frontend. While we encourage people to use libvirt, we can't simply ignore people who don't use libvirt as they have valid reasons for their choices.

So we have to enable GTK or SDL or both in Fedora. We could probably make a case for dropping SDL support in QEMU in Fedora, as GTK frontend is better these days, but that's going to have negligible impact on the deps pulled in.

There is a desire in QEMU upstream to modularize the codebase so that things can be split off into separately dlopen()able modules, with the intention that it will allow distros to ship all features without having to pull in the whole world at once. This is non-trivial work, especially for UI frontends to QEMU, so I'm not expecting it to be done any time soon.

Comment 24 Akira TAGOH 2017-02-24 01:46:32 UTC
fixed in fontconfig-2.12.1-4

Comment 25 Jonathan Lebon 2017-05-29 14:00:01 UTC
*** Bug 1456278 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.