Bug 2023061

Summary: Review Request: libdeflate - Fast implementation of DEFLATE, zlib, and gzip
Product: [Fedora] Fedora Reporter: Nick Black <dank>
Component: Package ReviewAssignee: Zbigniew Jędrzejewski-Szmek <zbyszek>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: package-review, zbyszek
Target Milestone: ---Flags: zbyszek: fedora-review+
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-23 20:15:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
patch to add missing debuginfo and fix stripping none

Description Nick Black 2021-11-14 13:20:44 UTC
Spec URL: https://nick-black.com/rpms/libdeflate.spec
SRPM URL: https://nick-black.com/rpms/libdeflate-1.8-1.fc36.src.rpm
Description: A fast implementation of DEFLATE, zlib, and gzip
Fedora Account System Username: nickblack

Comment 1 Nick Black 2021-11-14 13:24:20 UTC
looks like I've got a few rpmlint issues to handle:

[dank@localhost ~]$ rpmlint rpmbuild/SRPMS/libdeflate-1.8-1.fc36.src.rpm 
====================================================== rpmlint session starts ======================================================
rpmlint: 2.1.0
configuration:
    /usr/lib/python3.10/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/licenses.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 31, packages: 1

libdeflate.spec:69: W: macro-in-%changelog %autochangelog
======================= 1 packages and 0 specfiles checked; 0 errors, 1 warnings, 0 badness; has taken 0.1 s =======================
[dank@localhost ~]$ rpmlint rpmbuild/RPMS/
build.log                                libdeflate-1.8-1.fc36.x86_64.rpm         root.log
hw_info.log                              libdeflate-devel-1.8-1.fc36.x86_64.rpm   state.log
installed_pkgs.log                       libdeflate-static-1.8-1.fc36.x86_64.rpm  
libdeflate-1.8-1.fc36.src.rpm            libdeflate-utils-1.8-1.fc36.x86_64.rpm   
[dank@localhost ~]$ rpmlint rpmbuild/RPMS/*rpm
====================================================== rpmlint session starts ======================================================
rpmlint: 2.1.0
configuration:
    /usr/lib/python3.10/site-packages/rpmlint/configdefaults.toml
    /etc/xdg/rpmlint/fedora.toml
    /etc/xdg/rpmlint/licenses.toml
    /etc/xdg/rpmlint/scoring.toml
    /etc/xdg/rpmlint/users-groups.toml
    /etc/xdg/rpmlint/warn-on-functions.toml
checks: 31, packages: 5

libdeflate.x86_64: W: unstripped-binary-or-object /usr/lib64/libdeflate.so.0
libdeflate-utils.x86_64: W: unstripped-binary-or-object /usr/bin/libdeflate-gunzip
libdeflate-utils.x86_64: W: unstripped-binary-or-object /usr/bin/libdeflate-gzip
libdeflate-static.x86_64: E: static-library-without-debuginfo /usr/lib64/libdeflate.a
libdeflate.x86_64: E: shlib-policy-name-error 0
libdeflate-utils.x86_64: W: position-independent-executable-suggested /usr/bin/libdeflate-gunzip
libdeflate-utils.x86_64: W: position-independent-executable-suggested /usr/bin/libdeflate-gzip
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gunzip
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gzip
libdeflate-devel.x86_64: W: missing-dependency-on libdeflate*/libdeflate-libs/liblibdeflate* = 1.8
libdeflate.spec:69: W: macro-in-%changelog %autochangelog
======================= 5 packages and 0 specfiles checked; 2 errors, 9 warnings, 2 badness; has taken 0.6 s =======================
[dank@localhost ~]$

Comment 2 Zbigniew Jędrzejewski-Szmek 2021-11-21 12:30:52 UTC
> make USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr DESTDIR=$RPM_BUILD_ROOT %{?_smp_mflags}
This is the same as
%make_build USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr

> %install
> rm -rf $RPM_BUILD_ROOT
Uh, please drop ths line.

> make install DESTDIR=$RPM_BUILD_ROOT USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr  %{?_smp_mflags}
This is the same as 
%make_install USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr

Comment 3 Zbigniew Jędrzejewski-Szmek 2021-11-21 12:32:14 UTC
... and please figure out the rpmlint issues:
libdeflate.x86_64: W: unstripped-binary-or-object /usr/lib64/libdeflate.so.0
libdeflate-utils.x86_64: W: unstripped-binary-or-object /usr/bin/libdeflate-gunzip
libdeflate-utils.x86_64: W: unstripped-binary-or-object /usr/bin/libdeflate-gzip
libdeflate-static.x86_64: E: static-library-without-debuginfo /usr/lib64/libdeflate.a

Those all imply that compliation flags are not conveyed properly and postprocessing of files does not happen.

Comment 4 Nick Black 2021-11-21 21:22:00 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #2)
> > make USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr DESTDIR=$RPM_BUILD_ROOT %{?_smp_mflags}
> This is the same as
> %make_build USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr

done.

> > %install
> > rm -rf $RPM_BUILD_ROOT
> Uh, please drop ths line.

indeed. it came from https://rpm-packaging-guide.github.io/, which i see now is not fedora canon. i will mail the authors.

> > make install DESTDIR=$RPM_BUILD_ROOT USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr  %{?_smp_mflags}
> This is the same as 
> %make_install USE_SHARED_LIB=1 LIBDIR=%{_libdir} PREFIX=/usr

done.

Comment 5 Nick Black 2021-11-21 21:24:44 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #3)
> ... and please figure out the rpmlint issues:
> libdeflate.x86_64: W: unstripped-binary-or-object /usr/lib64/libdeflate.so.0
> libdeflate-utils.x86_64: W: unstripped-binary-or-object
> /usr/bin/libdeflate-gunzip
> libdeflate-utils.x86_64: W: unstripped-binary-or-object
> /usr/bin/libdeflate-gzip
> libdeflate-static.x86_64: E: static-library-without-debuginfo
> /usr/lib64/libdeflate.a
> 
> Those all imply that compliation flags are not conveyed properly and
> postprocessing of files does not happen.

working on these, thanks. i'll post here when i've got the issues resolved.

Comment 6 Nick Black 2021-11-22 00:42:23 UTC
I've eliminated the "unstripped-binary-or-object" warnings, though I had to put in a manual strip operation (I've only built CMake-based RPMs before; maybe this is typical for projects without a configuration process). Working on the "static-library-without-debuginfo" now. I've updated the SRPM/specfile.

Comment 7 Nick Black 2021-11-22 00:50:22 UTC
fixed up the "position-independent-executable-suggested" warnings; i've got this left:


libdeflate-static.x86_64: E: static-library-without-debuginfo /usr/lib64/libdeflate.a
libdeflate.x86_64: E: shlib-policy-name-error 0
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gunzip
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gzip
libdeflate-devel.x86_64: W: missing-dependency-on libdeflate*/libdeflate-libs/liblibdeflate* = 1.8
libdeflate.spec:70: W: macro-in-%changelog %autochangelog

Comment 8 Nick Black 2021-11-22 01:15:27 UTC
I dropped the static library package after noticing that most libraries on Fedora don't appear to ship their .as. This has eliminated the "static-library-without-debuginfo". Working on the 'shlib-policy-name-error' now. Updated specfile and SRPM.

Comment 9 Nick Black 2021-11-22 01:29:38 UTC
Got the "shlib-policy-name-error" resolved by renaming to libdeflate0. This eliminates all remaining errors from rpmlint (there remain several warnings). The new spec/SRPM can be found at:

https://nick-black.com/rpms/libdeflate0-1.8-1.fc36.src.rpm
https://nick-black.com/rpms/libdeflate0.spec

PTAL, and thanks for your time!

Comment 10 Zbigniew Jędrzejewski-Szmek 2021-11-22 08:55:10 UTC
Created attachment 1842971 [details]
patch to add missing debuginfo and fix stripping

> Got the "shlib-policy-name-error" resolved by renaming to libdeflate0. 

That's a strange one. I've never seen this message before. Are you using a non-Fedora version of rpmlint?
Fedora does not name packages after so-versions, please rename it back.

> %define debug_package %{nil}

This is wrong, we need debuginfo.

> CFLAGS="-fpic -pie -g"

This is also wrong. Packages must use standard compilation flags as provided by the macros.
[https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags]

> %{__strip}

General rule: do not use double-underscore-prefixed commands. Those were popular back in the day,
but they are useless and the guidelines finally have been changed to discourage them.

Specific rule: strip is called on all binaries through /usr/lib/rpm/redhat/brp-strip-lto,
so we need to figure out why that call doesn't work, and fix the compilation process so that
it does.

> I dropped the static library package after noticing that most libraries on Fedora don't appear to ship their .a

Yeah, that's probably a good solution. I assumed you want the the static library for something,
so I didn't suggest dropping it. But if it isn't needed for something else, it can go.

Removing it does not solve the original issue: the debug data is missing. Let's see what is needed to fix this.
...
Oh, it turns out that that this is the typical case of "upstream invents own build system, the
result is 90% correct". Please see the attached patch.

Comment 11 Nick Black 2021-11-22 09:13:03 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #10)
> That's a strange one. I've never seen this message before. Are you using a
> non-Fedora version of rpmlint?

i hope not?

> Fedora does not name packages after so-versions, please rename it back.

done.

> > %define debug_package %{nil}
> 
> This is wrong, we need debuginfo.
> 
> > CFLAGS="-fpic -pie -g"
> 
> This is also wrong. Packages must use standard compilation flags as provided
> by the macros.
> [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags]
> 
> > %{__strip}
> 
> General rule: do not use double-underscore-prefixed commands. Those were
> popular back in the day,
> but they are useless and the guidelines finally have been changed to
> discourage them.
> 
> Specific rule: strip is called on all binaries through
> /usr/lib/rpm/redhat/brp-strip-lto,
> so we need to figure out why that call doesn't work, and fix the compilation
> process so that
> it does.
> ...
> Oh, it turns out that that this is the typical case of "upstream invents own
> build system, the
> result is 90% correct". Please see the attached patch.

Applied.

Comment 12 Nick Black 2021-11-22 09:16:50 UTC
You had PREFIX=%{_prefix} in the build line and PREFIX=/usr in the install line. I assume you meant the former in both cases. Testing...

Comment 13 Nick Black 2021-11-22 09:17:43 UTC
there has been a great explosion of warnings and errors with this change, which seems to have undone the -fPIC -fpie -pie stuff on the binaries:


libdeflate-debuginfo.x86_64: W: unstripped-binary-or-object /usr/lib/debug/.dwz/libdeflate-1.8-1.fc36.x86_64
libdeflate-debuginfo.x86_64: W: unstripped-binary-or-object /usr/lib/debug/usr/lib64/libdeflate.so.0-1.8-1.fc36.x86_64.debug
libdeflate-utils-debuginfo.x86_64: W: unstripped-binary-or-object /usr/lib/debug/usr/bin/libdeflate-gunzip-1.8-1.fc36.x86_64.debug
libdeflate-utils-debuginfo.x86_64: W: unstripped-binary-or-object /usr/lib/debug/usr/bin/libdeflate-gzip-1.8-1.fc36.x86_64.debug
libdeflate-debuginfo.x86_64: E: statically-linked-binary /usr/lib/debug/.dwz/libdeflate-1.8-1.fc36.x86_64
libdeflate-utils-debuginfo.x86_64: E: statically-linked-binary /usr/lib/debug/usr/bin/libdeflate-gunzip-1.8-1.fc36.x86_64.debug
libdeflate-utils-debuginfo.x86_64: E: statically-linked-binary /usr/lib/debug/usr/bin/libdeflate-gzip-1.8-1.fc36.x86_64.debug
libdeflate.x86_64: E: shlib-policy-name-error 0
libdeflate-debuginfo.x86_64: E: shared-library-without-dependency-information /usr/lib/debug/usr/lib64/libdeflate.so.0-1.8-1.fc36.x86_64.debug
libdeflate-utils.x86_64: W: position-independent-executable-suggested /usr/bin/libdeflate-gunzip
libdeflate-utils.x86_64: W: position-independent-executable-suggested /usr/bin/libdeflate-gzip
libdeflate-utils-debuginfo.x86_64: W: position-independent-executable-suggested /usr/lib/debug/usr/bin/libdeflate-gunzip-1.8-1.fc36.x86_64.debug
libdeflate-utils-debuginfo.x86_64: W: position-independent-executable-suggested /usr/lib/debug/usr/bin/libdeflate-gzip-1.8-1.fc36.x86_64.debug
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gunzip
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gzip
libdeflate-devel.x86_64: W: missing-dependency-on libdeflate*/libdeflate-libs/liblibdeflate* = 1.8
libdeflate-debuginfo.x86_64: E: missing-PT_GNU_STACK-section /usr/lib/debug/.dwz/libdeflate-1.8-1.fc36.x86_64
libdeflate.spec:57: W: macro-in-%changelog %autochangelog
libdeflate-debuginfo.x86_64: W: hidden-file-or-dir /usr/lib/debug/.dwz
libdeflate-debuginfo.x86_64: W: hidden-file-or-dir /usr/lib/debug/.dwz
libdeflate-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/bb/c2ce132fcddce4617bdf4b5bd2c922f6db65a9 ../../../.build-id/bb/c2ce132fcddce4617bdf4b5bd2c922f6db65a9
libdeflate-utils-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459 ../../../.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459
libdeflate-utils-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459.1 ../../../.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459.1
====================== 7 packages and 0 specfiles checked; 6 errors, 17 warnings, 6 badness; has taken 1.1 s ======================

Comment 14 Zbigniew Jędrzejewski-Szmek 2021-11-22 09:20:59 UTC
Oh, OK.  "shlib-policy-name-error" is new in rpmlint-2, and I was doing the build on F33, which has rpmlint-1.x.
Googling for the phrase returned mostly opensuse results for some reason, that is why I asked.
I tried with rpmlint-2, and I see the error now…

Comment 15 Zbigniew Jędrzejewski-Szmek 2021-11-22 09:49:22 UTC
> You had PREFIX=%{_prefix} in the build line and PREFIX=/usr in the install line. I assume you meant the former in both cases.
Oh, good catch.

> -fPIC -fpie -pie stuff on the binaries:
Yeah, it seems that the pie flags should be added back where they were.

Most of the rpmlint warnings are bogus:
> libdeflate-debuginfo.x86_64: W: hidden-file-or-dir /usr/lib/debug/.dwz
> libdeflate-debuginfo.x86_64: W: hidden-file-or-dir /usr/lib/debug/.dwz
All debuginfo packages seem to have that.

> libdeflate.x86_64: E: shlib-policy-name-error 0
This is printed for various libs, and it seems to be just an error / stupid default configuration in rpmlint.

> libdeflate-devel.x86_64: W: missing-dependency-on libdeflate*/libdeflate-libs/liblibdeflate* = 1.8
The -devel package has Requires:libdeflate(x86-64) = 1.8-3.fc36, so that seems invalid too.

> libdeflate.spec:57: W: macro-in-%changelog %autochangelog
Bogus obviously.

> libdeflate-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/bb/c2ce132fcddce4617bdf4b5bd2c922f6db65a9 ../../../.build-id/bb/c2ce132fcddce4617bdf4b5bd2c922f6db65a9
> libdeflate-utils-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459 ../../../.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459
> libdeflate-utils-debuginfo.x86_64: W: dangling-relative-symlink /usr/lib/debug/.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459.1 ../../../.build-id/45/c5cea781642cd8998677a25ffdda2d5582e459.1

rpmlint doesn't understand the split into -debugsource packages...

Comment 16 Nick Black 2021-11-22 21:05:00 UTC
alright, i've added back "-fpic -fpie -pie", and that did eliminate the position-independent warnings once more.

almost everything else is debug package-related, aside from that shlib-policy-name-error business. i'm assuming we are continuing to ignore that, and name it libdeflate?

i've uploaded the specfile and SRPM.

i didn't kill the libdeflate0 variant in case we want that after all, but i'm assuming we're moving forward with:

https://nick-black.com/rpms/libdeflate-1.8-1.fc36.src.rpm
https://nick-black.com/rpms/libdeflate.spec

thanks again for your time and patience! it looks like we're just about ready to go, yes?

Comment 17 Zbigniew Jędrzejewski-Szmek 2021-11-23 11:56:49 UTC
+ package name is OK
+ license is acceptable (MIT)
+ license is specified correctly
+ latest version
+ builds and installs correctly
- %check is missing, see suggestion below
+ BR/R/P look OK
+ fedora-review finds no issues

rpmlint:
libdeflate.src: W: spelling-error Summary(en_US) gzip -> zip, grip, g zip
libdeflate.src: W: spelling-error Summary(en_US) zlib -> lib, glib, z lib
libdeflate.src: W: spelling-error %description -l en_US gzip -> zip, grip, g zip
libdeflate.src: W: spelling-error %description -l en_US zlib -> lib, glib, z lib
libdeflate.src:57: W: macro-in-%changelog %autochangelog
libdeflate.x86_64: W: spelling-error Summary(en_US) gzip -> zip, grip, g zip
libdeflate.x86_64: W: spelling-error Summary(en_US) zlib -> lib, glib, z lib
libdeflate.x86_64: W: spelling-error %description -l en_US gzip -> zip, grip, g zip
libdeflate.x86_64: W: spelling-error %description -l en_US zlib -> lib, glib, z lib
libdeflate-devel.x86_64: W: no-documentation
libdeflate-utils.x86_64: W: no-documentation
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gunzip
libdeflate-utils.x86_64: W: no-manual-page-for-binary libdeflate-gzip
7 packages and 0 specfiles checked; 0 errors, 13 warnings.
All OK. This is from rpmlint-1.11-19.fc33.noarch. Newer rpmlint has some additional (bogus) complaints.

Package is APPROVED.

I think it's be nice to add something like this:
%check
%{buildroot}%{_bindir}/libdeflate-gzip -c %{buildroot}%{_bindir}/libdeflate-gzip >gzip.gz
%{buildroot}%{_bindir}/libdeflate-gzip -t gzip.gz
%{buildroot}%{_bindir}/libdeflate-gunzip -c gzip.gz >gzip.gz.unzipped
diff %{buildroot}%{_bindir}/libdeflate-gzip gzip.gz.unzipped
to catch trivial build issues.

Comment 18 Gwyn Ciesla 2021-11-23 20:00:52 UTC
(fedscm-admin):  The Pagure repository was created at https://src.fedoraproject.org/rpms/libdeflate

Comment 19 Nick Black 2021-11-23 20:14:23 UTC
Thanks a bunch for the review and help, Zbigniew!