Bug 1424762 - Install size is too big
Summary: Install size is too big
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mupdf
Version: 25
Hardware: Unspecified
OS: Unspecified
low
medium
Target Milestone: ---
Assignee: Pavel Zhukov
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-19 09:46 UTC by Till Hofmann
Modified: 2017-03-23 17:21 UTC (History)
5 users (show)

Fixed In Version: mupdf-1.10a-4.fc25 mupdf-1.10a-4.fc24
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-03-07 13:21:46 UTC


Attachments (Terms of Use)

Description Till Hofmann 2017-02-19 09:46:10 UTC
Description of problem:
The size of the package changed from 39MB (1.8-2) to 214MB (1.10a-1).
In particular, the size of /usr/bin/mupdf-x11 changed from 9.3MB to 34MB.

Version-Release number of selected component (if applicable):
1.10a-1

Comment 1 Kevin Kofler 2017-02-21 18:47:08 UTC
Indeed, this has:
%global debug_package %{nil}
which is NOT valid for a package with compiled C code.

This was added by the latest commit:
http://pkgs.fedoraproject.org/cgit/rpms/mupdf.git/commit/mupdf.spec?id=c923519b02ec8fd5e2a3054b193630c041b5a8c4
with no explanation whatsoever. It MUST be removed.

Comment 2 Pavel Zhukov 2017-02-23 07:07:55 UTC
Thank you for raising this. I'm aware of this issue.

The reason of this is empty debug file list:

So I decided to fix CVE asap by disabling debuginfo for now and forgot to open a bug


[1] 
https://kojipkgs.fedoraproject.org//work/tasks/7988/18007988/build.log
[2]
Processing files: mupdf-debuginfo-1.10a-2.fc24.x86_64
RPM build errors:
error: Empty %files file /builddir/build/BUILD/mupdf-1.10a-source/debugfiles.list
    Empty %files file /builddir/build/BUILD/mupdf-1.10a-source/debugfiles.list
Child return code was: 1

Comment 3 Kevin Kofler 2017-02-24 14:04:46 UTC
But disabling the debuginfo is not a solution, it is not allowed.

Comment 4 Kevin Kofler 2017-02-24 14:07:55 UTC
There is also the issue that the verbose build output is missing. You are using "make %{?_smp_mflags} verbose=1"
but that is not the correct syntax, because we still see only "CC build/release/fontdump.o" and not the actual compiler invocation.

Comment 5 Kevin Kofler 2017-02-24 14:12:32 UTC
Looking at the (handwritten) makefile, it should be verbose=yes, not verbose=1.

Comment 6 Pavel Zhukov 2017-02-24 16:45:15 UTC
(In reply to Kevin Kofler from comment #3)
> it is not allowed.
Not true.

Comment 7 Pavel Zhukov 2017-02-24 16:46:00 UTC
(In reply to Kevin Kofler from comment #5)
> Looking at the (handwritten) makefile, it should be verbose=yes, not
> verbose=1.

Feel free to open a bug upstream

Comment 8 Kevin Kofler 2017-02-24 17:21:06 UTC
(In reply to Pavel Zhukov from comment #6)
> (In reply to Kevin Kofler from comment #3)
> > it is not allowed.
> Not true.

It is true.


https://fedoraproject.org/wiki/Packaging:Guidelines#Debuginfo_packages

> Packages should produce useful -debuginfo packages, or explicitly disable them
> when it is not possible to generate a useful one but rpmbuild would do it
> anyway.

So you should explicitly disable -debuginfo IF rpmbuild is producing a useless/empty one, this is not the case here.

You can argue that "it is not possible to generate a useful one" because it errors, but "it is not possible to generate a useful one but rpmbuild would do it anyway" is still not true because "rpmbuild would do it anyway" is false (rpmbuild does NOT "do it anyway", it errors). So the condition for explicitly disabling -debuginfo is not satisfied.

You are instead hitting a bug somewhere which needs to be addressed. I suspect it is an issue with how the executables are built, which is why I need to see the build output, which is why I raised the issue about the missing verbose output (also something most reviewers would treat as a blocker, because it does not allow verifying complaince with the packaging guidelines for compiler flags).

> Whenever a -debuginfo package is explicitly disabled, an explanation why it 
> was done is required in the specfile.

This is also missing in your specfile.


https://fedoraproject.org/wiki/Packaging:Debuginfo

> Missing debuginfo packages
>
> It is normal for noarch package builds to not produce a debuginfo package. If
> it's missing in other cases (where it has not been explicitly disabled),
> something's wrong.

That's the case we have here, something's clearly wrong.

Comment 9 Kevin Kofler 2017-02-24 17:21:51 UTC
(In reply to Pavel Zhukov from comment #7)
> (In reply to Kevin Kofler from comment #5)
> > Looking at the (handwritten) makefile, it should be verbose=yes, not
> > verbose=1.
> 
> Feel free to open a bug upstream

This is not an upstream bug, it is a bug in your specfile.

You need to change:
make %{?_smp_mflags} verbose=1
to:
make %{?_smp_mflags} verbose=yes

Comment 10 Pavel Zhukov 2017-02-24 17:29:20 UTC
(In reply to Kevin Kofler from comment #9)
> (In reply to Pavel Zhukov from comment #7)
> > (In reply to Kevin Kofler from comment #5)
> > > Looking at the (handwritten) makefile, it should be verbose=yes, not
> > > verbose=1.
> > 
> > Feel free to open a bug upstream
> 
> This is not an upstream bug, it is a bug in your specfile.
> 
> You need to change:
> make %{?_smp_mflags} verbose=1
> to:
> make %{?_smp_mflags} verbose=yes

It's not related to this bug anyway nor solved the big size problem. This happens because upstream project change it's structure too often and in very unpredictable way. And it's not "lightweight pdf viewer" as it was but rather set of xxxxx for pdf processing.
As I don't use huge number of buggy "features" of the project (like curl support, embedded js, converters etc) I don't care about them too much.

Comment 11 Pavel Zhukov 2017-02-24 17:30:43 UTC
(In reply to Kevin Kofler from comment #8)

> So you should explicitly disable -debuginfo IF rpmbuild is producing a
> useless/empty one, this is not the case here.
Not true again., It's *the* case here. rpmbuild produces  empty debuginfo. That's it.

Comment 12 Kevin Kofler 2017-02-24 19:02:32 UTC
(In reply to Pavel Zhukov from comment #10)
> (In reply to Kevin Kofler from comment #9)
> > (In reply to Pavel Zhukov from comment #7)
> > > (In reply to Kevin Kofler from comment #5)
> > > > Looking at the (handwritten) makefile, it should be verbose=yes, not
> > > > verbose=1.
> > > 
> > > Feel free to open a bug upstream
> > 
> > This is not an upstream bug, it is a bug in your specfile.
> > 
> > You need to change:
> > make %{?_smp_mflags} verbose=1
> > to:
> > make %{?_smp_mflags} verbose=yes
> 
> It's not related to this bug anyway nor solved the big size problem.

The verbose build output is needed to debug the build issue that causes the debuginfo issue. It may give us a hint as to what is going wrong with the build. Right now, I have no idea what the makefile is doing behind the scenes.

> This happens because upstream project change it's structure too often and in 
> very unpredictable way. And it's not "lightweight pdf viewer" as it was but
> rather set of xxxxx for pdf processing.
> As I don't use huge number of buggy "features" of the project (like curl
> support, embedded js, converters etc) I don't care about them too much.

Then I think the package needs a new maintainer or comaintainer.

Comment 13 Michael J Gruber 2017-02-28 18:35:54 UTC
Let me also mention that the size is due to changed linkage in mupdf; in particular, statically linking against bundled libraries.

Comment 14 Kevin Kofler 2017-02-28 22:57:43 UTC
The verbose command lines show that your linking command lines contain "-Wl,-s", e.g.:
cc -Wl,--gc-sections -Wl,-s -o build/release/fontdump build/release/fontdump.o -lpthread  -lm -lfreetype -lharfbuzz -ljbig2dec -ljpeg     -lopenjp2 -lz

That is what breaks the debuginfo package generation. You need to remove that flag (patch or sed it out of the makefile(s), or override the makefile variable that sets it on the make command line) and reenable debug_package.

Comment 15 Rex Dieter 2017-03-01 00:02:05 UTC
Simply dropping the explicit strip here doesn't seem to help make the main package much smaller, but does fix the no -debuginfo problem.

Doing a test with local builds I still end up with:

$ ls -sh1 x86_64/
total 173M
109M mupdf-1.10a-2.fc25.x86_64.rpm
 39M mupdf-debuginfo-1.10a-2.fc25.x86_64.rpm
 26M mupdf-devel-1.10a-2.fc25.x86_64.rpm

Something else is going on here, I suspect it may have to do with embedding resources (like fonts) into the executable.

Comment 16 Kevin Kofler 2017-03-01 02:26:30 UTC
It indeed embeds the fonts, which is a very silly thing to do, why does it not use the (system) TTF files? Yet another project trying to apply Windows development practices to *nix? It looks like it has always done that, but CJK fonts were added in June 2016:
http://git.ghostscript.com/?p=mupdf.git;a=commit;h=954e3bb173c3302bf458875e86c49c712f0789d4
which are probably the reason for the size increase. (Previously, only Noto Sans Fallback was provided, which is a lot smaller than the full-blown CJK font Source Han Sans they added now.)

Comment 17 Kevin Kofler 2017-03-01 02:32:41 UTC
I also think it would be useful to do what Debian does, and put mutool into a mupdf-tools subpackage. The embedded fonts are only used in the GUI viewer, and there are also dependencies on GUI libraries there. And you can really use mutool without the mupdf application, and also the other way round.

Comment 18 Pavel Zhukov 2017-03-01 08:49:14 UTC
(In reply to Kevin Kofler from comment #16)
> It indeed embeds the fonts, which is a very silly thing to do, why does it
> not use the (system) TTF files? Yet another project trying to apply Windows
> development practices to *nix?
Kevin, feel free to open BZ upstream. I tried to do so few times and all requests were closed with "I don't care" even if patch was attached. As far as I don't use any features but mupdf-x11 the best solution for me would be forking of the project.

According to -s option. I have the patch but it'll land rawhide most likely as it doesn't solve the big size issue and require patching of upstream files.

Comment 19 Michael J Gruber 2017-03-01 09:27:43 UTC
Yes, there were a lot of changes between mupdf 1.8 and 1.10a, which is why incremental updates could have helped.

Since I don't quite see where this is going here, I'm trying to develop the spec step-by-step at

https://github.com/mjg/mupdf.spec

so that at each step, a koji scratch build (at least) for f25 succeeds and one can see which step does what. I don't consider our testing repo a testbed for non-finalized specs, let alone the stable repo.

In -2, the maintainer already reverted the removal of "rm -rf thirdparty", good.

-3 corrects the wrong BR xulrunner-devel.

-4 reenables debug builds. After that, we should be in compliance with packaging guidelines again. (Upstream changed from default debug builds to default release builds between 1.8 and 1.10a.)

Stripping gets the size of the executables down from 40MB to 32MB or so. The debuginfo script complains about the size of the elf parts but "succeeds".

Upstream has a policy of bundling everything - that's not easy on packagers, but we do the unbundling for other packages, too.

Next step is the embedded font issue. Other distros create a .so that all executables link to dynamically, so that that space is wasted once only.

Also, as mentioned already, a changed subpackage structure may be in order.

I'd be happy to take pull requests there or hints here, of course. I have a personal interest in keeping mupdf working and alive in Fedora.

Comment 20 Kevin Kofler 2017-03-01 10:47:19 UTC
(In reply to Pavel Zhukov from comment #18)
> According to -s option. I have the patch but it'll land rawhide most likely
> as it doesn't solve the big size issue and require patching of upstream
> files.

Well, -4 would make a lot more sense to push as an update than -2. The only potentially user-visible change in -2 (compared to -1) is the fixed unbundling, which you did not even mention in the changelog and the update notes. Adding a comment to the specfile and enabling verbose build outputs are things that can make your package more compliant to the packaging guidelines, but what is the point of pushing a Bodhi update for that? On the other hand, -4 enables the -debuginfo subpackage again, which is user-visible, even if the size issue is still there.

Comment 21 Pavel Zhukov 2017-03-01 21:14:38 UTC
(In reply to Michael J Gruber from comment #19)
> Yes, there were a lot of changes between mupdf 1.8 and 1.10a, which is why
> incremental updates could have helped.
> 
> Since I don't quite see where this is going here, I'm trying to develop the
> spec step-by-step at
> 
> https://github.com/mjg/mupdf.spec
> 
> so that at each step, a koji scratch build (at least) for f25 succeeds and
> one can see which step does what. I don't consider our testing repo a
> testbed for non-finalized specs, let alone the stable repo.
> 
> In -2, the maintainer already reverted the removal of "rm -rf thirdparty",
> good.
> 
> -3 corrects the wrong BR xulrunner-devel.

Well...
It's not about karma. mupdf has 4 (I can be mistaken but quite many) CVE's assigned so first and most important part was to patch it against security issues as I had bug opened to update the packages I decided to go with update instead of patching. I think this is understandable.  That's why I've disabled debuginfo. I don't remember the reason of removing of rm -rf thirdparty libraries directory maybe it's was too late and I wanted to go to bed :) 
As we're on branching time right now I'm trying to fix all FTBFS bugs assigned to me to keep packages in the release. GCC7 brought many cakes for me and I have to eat them all. 
I really appreciate your comments and help to make the package better and I use mupdf on daily basis. But please don't forget about f26 branching and related load.
Patches and co-maintainers are more than welcomed :)

Comment 22 Joachim Frieben 2017-03-02 05:38:22 UTC
The latest Fedora 25 RPM mupdf-1.10a-1.fc25.x86_64 has a size of 133 MB. From the package description: "A lightweight PDF viewer and toolkit" ;o)

Comment 23 Michael J Gruber 2017-03-02 11:51:17 UTC
So, I can get the size down to roughly 4M per binary now by avoiding the font embedding. PDFs seem to display fine. I'm wondering about a few things, though:

- Do we need the embedded fons for something?
- Why is this even smaller than the 1.8 binaries?
- What to do with library?

mupdf-devel shipped a static library, which is not quite what we should do. There's a Gentoo/LFS patch that changes this to a shared library, for which we would need to introduce versioning and maybe a different package split (main, devel, libs).

Maybe do the down-sizing now (leaving the static devel as is) and worry about the library later?

Comment 24 Michael J Gruber 2017-03-02 13:38:54 UTC
So, spec and sources (for -5) as in

https://github.com/mjg/mupdf.spec/tree/a735ad2b424987d966d2a4736c33283db01cc278

result in a small rpm (9M) and small binaries. They seem to work for me and build on f25, f26 and rawhide:

https://koji.fedoraproject.org/koji/taskinfo?taskID=18137343
https://koji.fedoraproject.org/koji/taskinfo?taskID=18137438
https://koji.fedoraproject.org/koji/taskinfo?taskID=18137504

The library issue is treated exactly the same way as in 1.8, but it's 1.10a with the necessary patches for debug info and package size.

Feel free to use that. If you prefer me to apply for comaintainership and push those changes, that's fine too.

A new package split will require more planning and crosschecks with users of mupdf/tool in Fedora land, I think,

Comment 25 Kevin Kofler 2017-03-02 15:51:53 UTC
> - Do we need the embedded fons for something?

If you checked that the system font loading already works, then no.

You will probably want to Require or at least Recommend the expected fonts though. At least urw-fonts.

> - Why is this even smaller than the 1.8 binaries?

Because 1.8 already embedded plenty of fonts, just not the full-size CJK ones (only the lightweight Noto Sans Fallback for CJK).

> - What to do with library?

I'd just build it as an unversioned shared library. rpmlint will complain about it, but who cares? As long as no other package builds against it, versioning it is pointless.

Comment 26 Fedora Update System 2017-03-02 20:59:40 UTC
mupdf-1.10a-4.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6

Comment 27 Pavel Zhukov 2017-03-02 21:02:37 UTC
Thank you very much for testing and patches.
Please test
https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da and https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6 this bugfix and fix for https://bugzilla.redhat.com/show_bug.cgi?id=1425338 should be there.

Comment 28 Fedora Update System 2017-03-02 21:29:57 UTC
mupdf-1.10a-4.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da

Comment 29 Fedora Update System 2017-03-02 21:30:08 UTC
mupdf-1.10a-4.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da

Comment 30 Fedora Update System 2017-03-03 04:54:13 UTC
mupdf-1.10a-4.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6

Comment 31 Fedora Update System 2017-03-03 05:24:54 UTC
mupdf-1.10a-4.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da

Comment 32 Fedora Update System 2017-03-07 13:21:46 UTC
mupdf-1.10a-4.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 33 Fedora Update System 2017-03-23 17:21:05 UTC
mupdf-1.10a-4.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.