Description of problem: The size of the package changed from 39MB (1.8-2) to 214MB (1.10a-1). In particular, the size of /usr/bin/mupdf-x11 changed from 9.3MB to 34MB. Version-Release number of selected component (if applicable): 1.10a-1
Indeed, this has: %global debug_package %{nil} which is NOT valid for a package with compiled C code. This was added by the latest commit: http://pkgs.fedoraproject.org/cgit/rpms/mupdf.git/commit/mupdf.spec?id=c923519b02ec8fd5e2a3054b193630c041b5a8c4 with no explanation whatsoever. It MUST be removed.
Thank you for raising this. I'm aware of this issue. The reason of this is empty debug file list: So I decided to fix CVE asap by disabling debuginfo for now and forgot to open a bug [1] https://kojipkgs.fedoraproject.org//work/tasks/7988/18007988/build.log [2] Processing files: mupdf-debuginfo-1.10a-2.fc24.x86_64 RPM build errors: error: Empty %files file /builddir/build/BUILD/mupdf-1.10a-source/debugfiles.list Empty %files file /builddir/build/BUILD/mupdf-1.10a-source/debugfiles.list Child return code was: 1
But disabling the debuginfo is not a solution, it is not allowed.
There is also the issue that the verbose build output is missing. You are using "make %{?_smp_mflags} verbose=1" but that is not the correct syntax, because we still see only "CC build/release/fontdump.o" and not the actual compiler invocation.
Looking at the (handwritten) makefile, it should be verbose=yes, not verbose=1.
(In reply to Kevin Kofler from comment #3) > it is not allowed. Not true.
(In reply to Kevin Kofler from comment #5) > Looking at the (handwritten) makefile, it should be verbose=yes, not > verbose=1. Feel free to open a bug upstream
(In reply to Pavel Zhukov from comment #6) > (In reply to Kevin Kofler from comment #3) > > it is not allowed. > Not true. It is true. https://fedoraproject.org/wiki/Packaging:Guidelines#Debuginfo_packages > Packages should produce useful -debuginfo packages, or explicitly disable them > when it is not possible to generate a useful one but rpmbuild would do it > anyway. So you should explicitly disable -debuginfo IF rpmbuild is producing a useless/empty one, this is not the case here. You can argue that "it is not possible to generate a useful one" because it errors, but "it is not possible to generate a useful one but rpmbuild would do it anyway" is still not true because "rpmbuild would do it anyway" is false (rpmbuild does NOT "do it anyway", it errors). So the condition for explicitly disabling -debuginfo is not satisfied. You are instead hitting a bug somewhere which needs to be addressed. I suspect it is an issue with how the executables are built, which is why I need to see the build output, which is why I raised the issue about the missing verbose output (also something most reviewers would treat as a blocker, because it does not allow verifying complaince with the packaging guidelines for compiler flags). > Whenever a -debuginfo package is explicitly disabled, an explanation why it > was done is required in the specfile. This is also missing in your specfile. https://fedoraproject.org/wiki/Packaging:Debuginfo > Missing debuginfo packages > > It is normal for noarch package builds to not produce a debuginfo package. If > it's missing in other cases (where it has not been explicitly disabled), > something's wrong. That's the case we have here, something's clearly wrong.
(In reply to Pavel Zhukov from comment #7) > (In reply to Kevin Kofler from comment #5) > > Looking at the (handwritten) makefile, it should be verbose=yes, not > > verbose=1. > > Feel free to open a bug upstream This is not an upstream bug, it is a bug in your specfile. You need to change: make %{?_smp_mflags} verbose=1 to: make %{?_smp_mflags} verbose=yes
(In reply to Kevin Kofler from comment #9) > (In reply to Pavel Zhukov from comment #7) > > (In reply to Kevin Kofler from comment #5) > > > Looking at the (handwritten) makefile, it should be verbose=yes, not > > > verbose=1. > > > > Feel free to open a bug upstream > > This is not an upstream bug, it is a bug in your specfile. > > You need to change: > make %{?_smp_mflags} verbose=1 > to: > make %{?_smp_mflags} verbose=yes It's not related to this bug anyway nor solved the big size problem. This happens because upstream project change it's structure too often and in very unpredictable way. And it's not "lightweight pdf viewer" as it was but rather set of xxxxx for pdf processing. As I don't use huge number of buggy "features" of the project (like curl support, embedded js, converters etc) I don't care about them too much.
(In reply to Kevin Kofler from comment #8) > So you should explicitly disable -debuginfo IF rpmbuild is producing a > useless/empty one, this is not the case here. Not true again., It's *the* case here. rpmbuild produces empty debuginfo. That's it.
(In reply to Pavel Zhukov from comment #10) > (In reply to Kevin Kofler from comment #9) > > (In reply to Pavel Zhukov from comment #7) > > > (In reply to Kevin Kofler from comment #5) > > > > Looking at the (handwritten) makefile, it should be verbose=yes, not > > > > verbose=1. > > > > > > Feel free to open a bug upstream > > > > This is not an upstream bug, it is a bug in your specfile. > > > > You need to change: > > make %{?_smp_mflags} verbose=1 > > to: > > make %{?_smp_mflags} verbose=yes > > It's not related to this bug anyway nor solved the big size problem. The verbose build output is needed to debug the build issue that causes the debuginfo issue. It may give us a hint as to what is going wrong with the build. Right now, I have no idea what the makefile is doing behind the scenes. > This happens because upstream project change it's structure too often and in > very unpredictable way. And it's not "lightweight pdf viewer" as it was but > rather set of xxxxx for pdf processing. > As I don't use huge number of buggy "features" of the project (like curl > support, embedded js, converters etc) I don't care about them too much. Then I think the package needs a new maintainer or comaintainer.
Let me also mention that the size is due to changed linkage in mupdf; in particular, statically linking against bundled libraries.
The verbose command lines show that your linking command lines contain "-Wl,-s", e.g.: cc -Wl,--gc-sections -Wl,-s -o build/release/fontdump build/release/fontdump.o -lpthread -lm -lfreetype -lharfbuzz -ljbig2dec -ljpeg -lopenjp2 -lz That is what breaks the debuginfo package generation. You need to remove that flag (patch or sed it out of the makefile(s), or override the makefile variable that sets it on the make command line) and reenable debug_package.
Simply dropping the explicit strip here doesn't seem to help make the main package much smaller, but does fix the no -debuginfo problem. Doing a test with local builds I still end up with: $ ls -sh1 x86_64/ total 173M 109M mupdf-1.10a-2.fc25.x86_64.rpm 39M mupdf-debuginfo-1.10a-2.fc25.x86_64.rpm 26M mupdf-devel-1.10a-2.fc25.x86_64.rpm Something else is going on here, I suspect it may have to do with embedding resources (like fonts) into the executable.
It indeed embeds the fonts, which is a very silly thing to do, why does it not use the (system) TTF files? Yet another project trying to apply Windows development practices to *nix? It looks like it has always done that, but CJK fonts were added in June 2016: http://git.ghostscript.com/?p=mupdf.git;a=commit;h=954e3bb173c3302bf458875e86c49c712f0789d4 which are probably the reason for the size increase. (Previously, only Noto Sans Fallback was provided, which is a lot smaller than the full-blown CJK font Source Han Sans they added now.)
I also think it would be useful to do what Debian does, and put mutool into a mupdf-tools subpackage. The embedded fonts are only used in the GUI viewer, and there are also dependencies on GUI libraries there. And you can really use mutool without the mupdf application, and also the other way round.
(In reply to Kevin Kofler from comment #16) > It indeed embeds the fonts, which is a very silly thing to do, why does it > not use the (system) TTF files? Yet another project trying to apply Windows > development practices to *nix? Kevin, feel free to open BZ upstream. I tried to do so few times and all requests were closed with "I don't care" even if patch was attached. As far as I don't use any features but mupdf-x11 the best solution for me would be forking of the project. According to -s option. I have the patch but it'll land rawhide most likely as it doesn't solve the big size issue and require patching of upstream files.
Yes, there were a lot of changes between mupdf 1.8 and 1.10a, which is why incremental updates could have helped. Since I don't quite see where this is going here, I'm trying to develop the spec step-by-step at https://github.com/mjg/mupdf.spec so that at each step, a koji scratch build (at least) for f25 succeeds and one can see which step does what. I don't consider our testing repo a testbed for non-finalized specs, let alone the stable repo. In -2, the maintainer already reverted the removal of "rm -rf thirdparty", good. -3 corrects the wrong BR xulrunner-devel. -4 reenables debug builds. After that, we should be in compliance with packaging guidelines again. (Upstream changed from default debug builds to default release builds between 1.8 and 1.10a.) Stripping gets the size of the executables down from 40MB to 32MB or so. The debuginfo script complains about the size of the elf parts but "succeeds". Upstream has a policy of bundling everything - that's not easy on packagers, but we do the unbundling for other packages, too. Next step is the embedded font issue. Other distros create a .so that all executables link to dynamically, so that that space is wasted once only. Also, as mentioned already, a changed subpackage structure may be in order. I'd be happy to take pull requests there or hints here, of course. I have a personal interest in keeping mupdf working and alive in Fedora.
(In reply to Pavel Zhukov from comment #18) > According to -s option. I have the patch but it'll land rawhide most likely > as it doesn't solve the big size issue and require patching of upstream > files. Well, -4 would make a lot more sense to push as an update than -2. The only potentially user-visible change in -2 (compared to -1) is the fixed unbundling, which you did not even mention in the changelog and the update notes. Adding a comment to the specfile and enabling verbose build outputs are things that can make your package more compliant to the packaging guidelines, but what is the point of pushing a Bodhi update for that? On the other hand, -4 enables the -debuginfo subpackage again, which is user-visible, even if the size issue is still there.
(In reply to Michael J Gruber from comment #19) > Yes, there were a lot of changes between mupdf 1.8 and 1.10a, which is why > incremental updates could have helped. > > Since I don't quite see where this is going here, I'm trying to develop the > spec step-by-step at > > https://github.com/mjg/mupdf.spec > > so that at each step, a koji scratch build (at least) for f25 succeeds and > one can see which step does what. I don't consider our testing repo a > testbed for non-finalized specs, let alone the stable repo. > > In -2, the maintainer already reverted the removal of "rm -rf thirdparty", > good. > > -3 corrects the wrong BR xulrunner-devel. Well... It's not about karma. mupdf has 4 (I can be mistaken but quite many) CVE's assigned so first and most important part was to patch it against security issues as I had bug opened to update the packages I decided to go with update instead of patching. I think this is understandable. That's why I've disabled debuginfo. I don't remember the reason of removing of rm -rf thirdparty libraries directory maybe it's was too late and I wanted to go to bed :) As we're on branching time right now I'm trying to fix all FTBFS bugs assigned to me to keep packages in the release. GCC7 brought many cakes for me and I have to eat them all. I really appreciate your comments and help to make the package better and I use mupdf on daily basis. But please don't forget about f26 branching and related load. Patches and co-maintainers are more than welcomed :)
The latest Fedora 25 RPM mupdf-1.10a-1.fc25.x86_64 has a size of 133 MB. From the package description: "A lightweight PDF viewer and toolkit" ;o)
So, I can get the size down to roughly 4M per binary now by avoiding the font embedding. PDFs seem to display fine. I'm wondering about a few things, though: - Do we need the embedded fons for something? - Why is this even smaller than the 1.8 binaries? - What to do with library? mupdf-devel shipped a static library, which is not quite what we should do. There's a Gentoo/LFS patch that changes this to a shared library, for which we would need to introduce versioning and maybe a different package split (main, devel, libs). Maybe do the down-sizing now (leaving the static devel as is) and worry about the library later?
So, spec and sources (for -5) as in https://github.com/mjg/mupdf.spec/tree/a735ad2b424987d966d2a4736c33283db01cc278 result in a small rpm (9M) and small binaries. They seem to work for me and build on f25, f26 and rawhide: https://koji.fedoraproject.org/koji/taskinfo?taskID=18137343 https://koji.fedoraproject.org/koji/taskinfo?taskID=18137438 https://koji.fedoraproject.org/koji/taskinfo?taskID=18137504 The library issue is treated exactly the same way as in 1.8, but it's 1.10a with the necessary patches for debug info and package size. Feel free to use that. If you prefer me to apply for comaintainership and push those changes, that's fine too. A new package split will require more planning and crosschecks with users of mupdf/tool in Fedora land, I think,
> - Do we need the embedded fons for something? If you checked that the system font loading already works, then no. You will probably want to Require or at least Recommend the expected fonts though. At least urw-fonts. > - Why is this even smaller than the 1.8 binaries? Because 1.8 already embedded plenty of fonts, just not the full-size CJK ones (only the lightweight Noto Sans Fallback for CJK). > - What to do with library? I'd just build it as an unversioned shared library. rpmlint will complain about it, but who cares? As long as no other package builds against it, versioning it is pointless.
mupdf-1.10a-4.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6
Thank you very much for testing and patches. Please test https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da and https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6 this bugfix and fix for https://bugzilla.redhat.com/show_bug.cgi?id=1425338 should be there.
mupdf-1.10a-4.fc24 has been submitted as an update to Fedora 24. https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da
mupdf-1.10a-4.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-9a819664a6
mupdf-1.10a-4.fc24 has been pushed to the Fedora 24 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-3b97b275da
mupdf-1.10a-4.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
mupdf-1.10a-4.fc24 has been pushed to the Fedora 24 stable repository. If problems still persist, please make note of it in this bug report.