Bug 1048115 - texlive is split into an insane amount of packages
Summary: texlive is split into an insane amount of packages
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: texlive
Version: rawhide
Hardware: All
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Tom "spot" Callaway
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-03 08:41 UTC by Ferry Huberts
Modified: 2021-05-13 13:21 UTC (History)
16 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-13 13:21:19 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ferry Huberts 2014-01-03 08:41:16 UTC
Description of problem:
I did an update of my F20 machine, receive 4742 texlive package.
You read that right: nearly 5000 packages!

What insane decision lies at the base of this insane split?

This has been present for way too long, time to end this insanity!

Consider a texlive user: how the heck is that user going to decide which packages he needs for his use-case? Trail-and-error of way too many install cycles it seems.... (and this is only 1 use-case)

texlive packaging is completely hostile to a user

Comment 1 Ralph Loader 2014-04-18 21:55:29 UTC
There is also the deleterious effect on rpm of having so many packages.

rpm's database handling is pretty slow; 'rpm -qa' with cold disk caches on a system with a spinning-rust disk typically takes 30 seconds to read the packege list, on a system with ~ 1000 packages.

I haven't measured it, but inflating 1000 to 1000+5000=6000 is not going to help here.

Comment 2 Ferry Huberts 2015-01-13 18:25:48 UTC
ping?

any ETA on fixing this insanity?

Comment 3 Lukas Middendorf 2015-05-13 11:42:24 UTC
What makes this even worse is the fact that the split is not utilized to update only affected subpackages for small (security) updates.

There have been two updates to texlive for F21 in a little bit more than one month (FEDORA-2015-4872 and FEDORA-2015-7292) that require updating/ reinstalling all packages while changing only the content of one of them.
The update process should be changed to only push the affected subpackages and not also all of the unchanged packages.

Comment 4 Fedora Admin XMLRPC Client 2015-08-11 05:32:02 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 5 Tom "spot" Callaway 2015-09-17 20:02:40 UTC
As the new maintainer, I feel your pain. I really do. That said, I don't see anyone (including myself) volunteering to package/review/maintain all of the components within the texlive distribution, nor do most end users want _all_ of texlive.

This is the best compromise I can see at the moment. rpm has gotten notably faster in the year or so since this was filed.

Comment 6 Ferry Huberts 2016-02-22 11:19:30 UTC
See https://plus.google.com/+RichardHughes/posts/7MyQWzu5RAJ

Comment 7 Richard Hughes 2016-02-22 11:26:49 UTC
Trying to update 4000 subpackages in the gnome-software GUI looks really bad. I'm thinking it's best to just blacklist texlive-* in gnome-software until it is fixed.

Comment 8 Lukas Middendorf 2016-02-23 15:22:28 UTC
The current upgrade process is completely unsuited for the huge load of texlive packages.
The main problem is not that texlive is split into so many subpackages but that the complete load of unchanged packages is pushed with a version bump when one (or very few) of the subpackages changes.
If it is not possible to push only single subpackages as update this issue should likely be raised with FESCo.

Comment 9 Tom "spot" Callaway 2016-06-15 14:30:39 UTC
Okay. Let's lay out the scenarios I can think of. Feel free to chime in if you can think of another one.

Upstream:
TeXLive does a "year" release, where the thousands of TeX components are collected and marked as "stable" for that release. 

Option A:
One SRPM, each TeX component in its own subpackage. (This is what we have now.)
Pros: 
 * Users only install the TeX components they need. 
 * Packages can depend on just the specific TeX components they need.
 * TeX component packages are small.
 * Matches upstream release model
 * Easy to update and build
Cons:
 * If any TeX component needs a change, all of the component packages get updated, causing significant and unnecessary update churn for users.
 * GNOME Software GUI looks bad when there are too many packages in an update transaction. (This is arguably a design flaw in the GNOME Software tool, but I'll leave it here.)

Option B:
Each TeX component in its own SRPM
Pros: 
  * Users only install the TeX components they need.
  * Packages can depend on just the specific TeX components they need.
  * TeX component packages are small.
  * Mostly matches upstream release model
  * Components can be updated independently.
Cons:
  * Terribly unmaintainable without a small army (4000 new packages)
  * Build dependencies make a tangled mess
  * Functionally impossible to review this quantity of packages
  * Each new TeXLive release brings with it a noteworthy number of new or 
    renamed TeX components
  * GNOME Software GUI looks bad when there are too many packages in an update 
    transaction.

Option C:
One giant TeXLive package (binary and SRPM)
Pros:
  * Maintainable.
  * Easy to update and build.
  * Matches upstream release model
  * Packaging is simple (put bits in bag)
  * GNOME Software GUI looks clean (1 vs 4000)  
Cons:
  * Users have to install all or nothing.
  * Packages have to depend on large package.
  * Package is large, takes up unnecessary space in Fedora (significantly 
    increasing every single Fedora user's install footprint)
  * Any change to any TeX component results in new TeXLive package update (lots 
    of update churn, with a large package)

Option D:
TeXLive-noarch and TeXLive-arch SRPMs, each TeX component in its own subpackage
Pros:
  * Minimizes update churn a bit (sort of, most packages are noarch in TeXLive)
  * Packages can depend on just the specific TeX components they need.
  * TeX component packages are small.
  * Mostly matches upstream release model
  * Maintainable
  * Easy to update and build
Cons:
  * Packaging is more complicated (spec files have to be essentially rewritten for each TeXLive release).
  * Doesn't really minimize update churn (most updates are noarch already)
  * GNOME Software GUI looks bad when there are too many packages in an update transaction.

Option E:
TeXLive-noarch and TeXLive-arch SRPMs and binary RPMs.
Pros:
  * Maintainable.
  * Easy to build.
  * TeXLive-arch package probably doesn't update very often.
  * GNOME Software GUI looks cleaner (2 vs 4000)
Cons:
  * Doesn't match upstream release model.
  * Packaging is more complicated (spec files have to be essentially rewritten 
    for each TeXLive release).
  * Users have to install most or nothing. Practically, they have to install 
    both.
  * Packages have to depend on two large packages.
  * Packages are large, takes up unnecessary space in Fedora (basically, 
    increasing every single Fedora user's install footprint)
  * Any change to majority of TeX components results in new TeXLive-noarch 
    package update (lots of update churn, with a large package)

Option F:
TeXLive alpha split SRPMs (texlive-a, texlive-b, texlive-c, texlive-numbers), individual component binary packages
Pros:
  * Users only install the TeX components they need.
  * Packages can depend on just the specific TeX components they need.
  * TeX component packages are small.
  * Mostly matches upstream release model
  * Minimizes unnecessary update pushes (only if another component in the same 
    letter group needs an update)
Cons:
  * Slightly harder to build (27 SRPMs vs 1)
  * Packaging is more complicated (though, probably scriptable... spec files 
    would have to be essentially rewritten for each TeXLive release)
  * Still results in some components being pushed out as updates without changes
  * GNOME Software GUI looks bad when there are too many packages in an update 
    transaction.

Option G:
TeXLive alpha split SRPMs and binary RPMs
Pros:
  * Alpha subset packages are possibly small
  * Minimizes unnecessary update pushes (only if another component in the same 
    letter group needs an update)
  * GNOME Software GUI looks cleaner (27 vs 4000)
Cons:
  * Alpha subset packages are potentially large.
  * Doesn't match upstream release model
  * Packages pull in entire alpha subset to get component they need.
  * Users have to install some components they don't need within the alpha 
    package grouping (i need armadillo, but not antelope or anteater)
  * Increases the install footprint for all Fedora users slightly
  * Slightly harder to build (27 SRPMs vs 1)
  * Does not match upstream release model
  * Packaging is more complicated (though, probably scriptable... spec files 
    would have to be essentially rewritten for each TeXLive release)
  * Still results in some components being pushed out as updates without changes

Option H:
Nuke TeXLive from orbit.
Pros:
  * TeXLive is gone. The people cheer.
  * TeXLive never again appears in update transactions.
  * Fedora mass rebuilds take at least a day less.
Cons:
  * Fedora users no longer have TeX support.
  * TeXLive upstream now radioactive.
  * Fedora packages can no longer generate documentation.

*******

The previous TeXLive maintainer looked at all of this, and came to the conclusion that Option A was the best-bad option. There is no good option.

Comment 10 Ferry Huberts 2016-06-15 14:56:36 UTC
For me personally options C and E look the sanest and very acceptable.

Comment 11 Yu Watanabe 2016-06-26 17:27:01 UTC
+1 for option A.

Comment 12 Lukas Middendorf 2016-06-27 14:56:13 UTC
I definitely would prefer Option D over the currently implemented Option A.

From my past experience with texlive updates in Fedora, updates that happen to a released Fedora version occur because of bugs (possibly security related) in an executable. This means that with a separation of the arch and noarch packages, only the arch part has to be updated, leaving all noarch packages untouched.

Comment 13 Tom "spot" Callaway 2016-07-05 15:54:36 UTC
(In reply to Lukas Middendorf from comment #12)
> I definitely would prefer Option D over the currently implemented Option A.
> 
> From my past experience with texlive updates in Fedora, updates that happen
> to a released Fedora version occur because of bugs (possibly security
> related) in an executable. This means that with a separation of the arch and
> noarch packages, only the arch part has to be updated, leaving all noarch
> packages untouched.

This has not been true for at least the last year (while I've been maintaining it). I think we may have had one arch specific change, all of the rest were noarch.

Comment 14 Fedora End Of Life 2016-11-24 11:05:37 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 15 Christian Stadelmann 2016-12-03 13:59:03 UTC
Issue is still present, although I doubt it will ever be fixed in a way that reduces the number of packages. Texlive has so many packages it could have its own package manager, which is NOT what I suggest to do.

Anyway, isn't the issue somewhere else? I don't think the issue is the fact that texlive has so many subpackages, but that updating one of them will force an update to all of them.

(In reply to Tom "spot" Callaway from comment #9)
> Okay. Let's lay out the scenarios I can think of. Feel free to chime in if
> you can think of another one.
> 
> Option A:
> One SRPM, each TeX component in its own subpackage. (This is what we have
> now.)
[…]
> Cons:
>  * If any TeX component needs a change, all of the component packages get
> updated, causing significant and unnecessary update churn for users.

Why is that? Is this a limitation of the fedora buildsys? If yes, shouldn't it be fixed there?
It would be a lot easier if only the subpackages with changed sources are updated. This could easily be automated through diff or parsing CTAN update announcements.

Comment 16 Fedora End Of Life 2016-12-20 12:44:13 UTC
Fedora 23 changed to end-of-life (EOL) status on 2016-12-20. Fedora 23 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 17 pav 2018-10-08 22:43:15 UTC
Obviously, the issue is still present in Fedora 28 and 29. Wouldn't matter except that rpm/dnf spends hours on upgrades when the gates open and the texlive package horde comes rolling in...

Comment 18 Denis Leroy 2018-11-19 09:53:31 UTC
Reopening since still relevant. Texlive packages are currently eating a whopping 10% of the package list space.

Comment 19 Tom "spot" Callaway 2019-06-26 21:04:45 UTC
Just as an FYI to those coming to this bug... I do not have any plans to change how texlive is managed. TL upstream is intentionally designed around this many components, so having a "lot of texlive packages" is a reflection of that upstream decision. It is also not practical to switch to thousands of independent TL source packages, and neither RPM nor koji have the capacity to only generate some subpackages vs all subpackages.

I'm open to suggestions and patches, but I've already split off texlive-base (things that are part of the compiled core) and left texlive as entirely noarch, to minimize the amount of packages that get updated.

Comment 20 Garrett Mitchener 2020-01-21 14:44:18 UTC
Is there a way to use DRPMs to cut down on the size of these updates? I installed 3.8 GB of TeXLive updates on Friday. Now it's Tuesday and I have to install 1.5 GB more, many of them repeats.

Comment 21 Ben Cotton 2020-04-30 20:32:15 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 23 gwelch925 2021-03-27 04:29:40 UTC
I apologize in advance because I realize this is an old issue. I was about
comment with my annoyance that nothing is being done about this "issue" until I
learned that there are more nuanced package options available.

My primary use case for texlive is keeping my resume up-to-date, so I'm hardly a
LaTeX power user. Whenever I have needed to make updates and recompile my resume
(which is rare thankfully), I invariably just search for "fedora install texlive"
to remind myself of the package name.

And, every time that I can recall, the suggestion has been to do either:

    sudo (yum|dnf) install texlive-*                # or
    sudo (yum|dnf) install texlive-scheme-full


It wasn't until moments ago that I came across another source the NeuroFedora
Latex Documentation (https://docs.fedoraproject.org/en-US/neurofedora/latex/)
that I became aware that I had other options available.

Ex.
    texlive-scheme-basic   # 368 packages, 173M Total, 84M Download; or
    texlive-scheme-medium  # 1503 ",       768M,       383M

As well as the ability to install specific styles; ex:

    tex(beamer.cls)     # A specific class, or
    tex(hyperref.sty)   # A specific style


Then there's the official Fedora wiki article (https://fedoraproject.org/wiki/Features/TeXLive)
which lists even more options.  If you really look at all of the different
available collections, then you might start to realize how insane it is to
suggest `texlive-scheme-full` (or `texlive-*`) to 99% of users.


I would suggest that this "issue" isn't an software "issue" nor a "bug", but it
is an opportunity for someone (me in this case) to tell the Fedora community to
STOP SUGGESTING `texlive-scheme-full` TO PEOPLE.

I'm glad that we have a fairly large and helpful community, but it's just as
easy to suggest any of the smaller sets `-minimal`, `-basic`, .-small`, or
`-medium`.


The only possible suggestion that I would have for improving
`texlive-scheme-full` would be to warn the user about what they are doing,
propose the alternative package options, and then force them to type a
longer message than `Y`; i.e. "I, <name>, of sound mind, willingly and
voluntarily want to install texlive-scheme-full".

Comment 24 Fedora Program Management 2021-04-29 16:47:15 UTC
This message is a reminder that Fedora 32 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 32 on 2021-05-25.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '32'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 32 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 25 Christian Stadelmann 2021-05-13 13:13:41 UTC
Still present in rawhide: https://koji.fedoraproject.org/koji/buildinfo?buildID=1720735

Comment 26 Tom "spot" Callaway 2021-05-13 13:21:19 UTC
Ugh. As described (repeatedly), this is not a bug, this is how TeXLive is. Please don't reopen it unless you have a patch in hand.


Note You need to log in before you can comment on or make changes to this bug.