Description of problem: Please revert the generating of package notes for two reasons: 1) It is using the wrong Version/Release in case package has subpackages with different Version/Release to the main package 2) It breaks Ruby and possibly other languages, which embeds build flags for their ecosystem. This is the error I got when locally building rubygem-nio4r against Ruby 3.1: ~~~ gcc -shared -o nio4r_ext.so bytebuffer.o monitor.o nio4r_ext.o selector.o -L. -L/usr/lib64 -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-rubygem-nio4r-2.5.2-6.fc36.x86_64.ld -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-ruby-0.21.1-201~1.fc36.x86_64.ld -m64 -lruby -lm -lc /usr/bin/ld: cannot open linker script file /builddir/build/BUILD/.package_note-ruby-0.21.1-201~1.fc36.x86_64.ld: No such file or directory ~~~ Version-Release number of selected component (if applicable): redhat-rpm-config-209-1.fc36 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Notes are using wrong Version/Release and the build flags are embedded into language package managers for their reuse for language ecosystem Expected results: The correct Version/Release is used (or not used at all) and the flags are not embedded into the languages package managers (or at least it is understood what consequences it causes). Additional info:
Just FTR, this is the example of broken build: https://koji.fedoraproject.org/koji/taskinfo?taskID=81531596
I have disabled the package notes in Ruby: https://src.fedoraproject.org/rpms/ruby/c/a0bcb33eaa666d3e1d08ca45e77161ca05611487?branch=rawhide The build is running now: https://koji.fedoraproject.org/koji/taskinfo?taskID=81534724 This should prevent rubygem- build breakage, therefore from this POV, the revert is not super important anymore. Nevertheless: 1) I suspect Python (and possibly other packaging systems) might be broken in similar way. 2) If this should be re-enabled in Ruby, the issue mentioned in my original comment should be addressed.
At least from the Python perspective, this needs to be filtered from %extension_ldflags.
> 1) It is using the wrong Version/Release in case package has subpackages with different Version/Release to the main package Let's open a separate Bugzilla for that?
This should fix this for Python: https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/169
(In reply to Miro Hrončok from comment #4) > > 1) It is using the wrong Version/Release in case package has subpackages with different Version/Release to the main package > > Let's open a separate Bugzilla for that? Might be: bug 2043143
Just FTR, this is reported against redhat-rpm-config, because I have requested revert.
This needs to be solved in redhat-rpm-config, but I'd argue a revert is not the only solution.
I also filed https://bugzilla.redhat.com/show_bug.cgi?id=2043166 about how this patch has broken all ELN builds.
This seems to be fixed now.
Haskell packages (eg ghc*) still fail with redhat-rpm-config-210-1.fc36 eg for ghc-zlib: : [6 of 6] Compiling Codec.Compression.GZip /usr/bin/ld.gold: error: /var/home/petersen/fedora/haskell/hackage/ghc-zlib/.package_note-ghc-zlib-0.6.2.3-1.fc36.x86_64.ld:42:8: syntax error, unexpected STRING /usr/bin/ld.gold: fatal error: unable to parse script file /var/home/petersen/fedora/haskell/hackage/ghc-zlib/.package_note-ghc-zlib-0.6.2.3-1.fc36.x86_64.ld collect2: error: ld returned 1 exit status `gcc' failed in phase `Linker'. (Exit code: 1) error: Bad exit status from /var/tmp/rpm-tmp.75xrrd (%build) So I need to filter out in ghc-rpm-macros? Or is there a better way to handle this?
(ghc-zlib may not be the best example, since it somehow exhibits some linking errors with redhat-rpm-config-208-1.fc36, but other Haskell packages generally seem to build with 208 from my limited local testing.)
(In reply to Zbigniew Jędrzejewski-Szmek from comment #10) > This seems to be fixed now. Zbysek, this might be fixed for Python, but it is not addressed for Ruby (I have just applied workaround) and in essence, this is the same issue as bug #1284684. Apparently (and not surprisingly) Haskell suffers the same issues (and at minimum would need the same workaround probably). Honestly, I am quite disappointed. Such change would deserve some trial mass rebuild at minimum, it would deserve to be announced that it landed, it would deserve more coordination. And these are the reasons why I proposed revert, because this is not well prepared change.
Python 2.7 and PyPys are still affected as they do not use %extension_ldflags. We will need to opt-out from Python 2.7 and PyPy 2.7 and figure out if PyPy 3.X supports LDFLAGS_NODIST (or opt-out as well). Arguably, the fact that Ruby and Haskell and Python 2 always hardcodes all the flags they are built with is a problem that should be fixed. However, I agree that this change landed way too shortly before the masses rebuild. There was some initial trial rebuild, but it was small, not mass, which in retrospect was not good enough testing. As for whether to revert or not, I haven't said "don't revert" -- I've simply stated that it is not the only solution. But it surely is "a" solution.
Python 2.7: https://src.fedoraproject.org/rpms/python2.7/pull-request/23
PyPys don't seem to contain the cflags+ldflags used to build it anywhere in it.
Also why is .package_note-*.<arch>.ld file written to the pkg dist-git directory?
I can add '%undefine _package_note_flags' to ghc-rpm-macros but dunno who is going to rebuild all the Haskell packages.
(In reply to Jens Petersen from comment #17) > Also why is .package_note-*.<arch>.ld file written to the pkg dist-git directory? It's written to %{_builddir}, so we usually end up with something reasonable like /builddir/build/BUILD/.package_note-lirc-0.10.0-34.fc36.x86_64.ld. I would love to push it one level down, e.g. to /builddir/build/BUILD/lirc-0.10.0/.package_note-lirc-0.10.0-34.fc36.x86_64.ld, or even /builddir/build/BUILD/lirc-0.10.0/.package_note.x86_64.ld. Unfortunately rpm does not have a concept of a separate build directory under %{_builddir}, and it's up to each and every package to make a directory there. This usually happens, but not always, and even if it does, there is no macro to reference the name, so we have no choice but to use %{_builddir}. I you do a build with %{_builddir} set to '.', you will end up with a file in a '.'. That is why the name starts with a dot and we make an effort to make the file name unique. This is no different that the .build-*.log files, or any other files and directories that the build process creates directly under %{_builddir}. (Obviously, if you have a better idea, I'm all ears.)
I just pushed ghc-rpm-macros-2.3.12-1.fc36 to Koji * Fri Jan 21 2022 Jens Petersen <petersen> - 2.3.12-1 - disable package notes since breaks all Haskell packages (#2043092)
(In reply to Zbigniew Jędrzejewski-Szmek from comment #19) > (In reply to Jens Petersen from comment #17) > > Also why is .package_note-*.<arch>.ld file written to the pkg dist-git directory? > > It's written to %{_builddir}, so we usually end up with something reasonable > like > /builddir/build/BUILD/.package_note-lirc-0.10.0-34.fc36.x86_64.ld. > I would love to push it one level down, e.g. to > /builddir/build/BUILD/lirc-0.10.0/.package_note-lirc-0.10.0-34.fc36.x86_64. > ld, > or even /builddir/build/BUILD/lirc-0.10.0/.package_note.x86_64.ld. > Unfortunately rpm does not have a > concept of a separate build directory under %{_builddir}, and it's up to > each and every package to make > a directory there. This usually happens, but not always, and even if it > does, there is no macro to > reference the name, so we have no choice but to use %{_builddir}. > > I you do a build with %{_builddir} set to '.', you will end up with a file > in a '.'. That is why > the name starts with a dot and we make an effort to make the file name > unique. This is no different > that the .build-*.log files, or any other files and directories that the > build process creates > directly under %{_builddir}. > > (Obviously, if you have a better idea, I'm all ears.) We use %{_builddir}%{?buildsubdir:/%{buildsubdir}} in https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/macros.pyproject
(In reply to Miro Hrončok from comment #21) > We use %{_builddir}%{?buildsubdir:/%{buildsubdir}} in > https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/ > macros.pyproject I am all in for such directory. Unfortunately, it would mean defacto to standardize it Fedora wide. Again, I don't think it would be wrong, just this is probably not the right way.
(In reply to Vít Ondruch from comment #22) > (In reply to Miro Hrončok from comment #21) > > We use %{_builddir}%{?buildsubdir:/%{buildsubdir}} in > > https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/ > > macros.pyproject > > I am all in for such directory. Unfortunately, it would mean defacto to > standardize it Fedora wide. Again, I don't think it would be wrong, just > this is probably not the right way. I am not quite sure what you are trying to say here. Such directory is created and the macro is defined by %setup, not by pyproject-rpm-macros.
Thank you for the suggestion. I tested %{buildsubdir} and it seems to work. https://github.com/rpm-software-management/rpm/issues/551 has some interesting discussion about possible changes to it, but I think this wouldn't matter to this particular use case. I'll change %_package_note_file to include %{buildsubdir} if defined. I'll keep the "-%{name}-%{version}-%{release}" suffix though, because %buildsubdir might not be defined in all cases.
(In reply to Miro Hrončok from comment #23) Sorry, scrub that, neither I was sure what I was saying. I thought that this is some Python convention just to learn that the `%setup` macro sets this variable. I was hoping, that it creates something like temporary directory.
(In reply to Jens Petersen from comment #20) > * Fri Jan 21 2022 Jens Petersen <petersen> - 2.3.12-1 > - disable package notes since breaks all Haskell packages (#2043092) I think this is OK. > but dunno who is going to rebuild all the Haskell packages. So I looked in koji, and they seem to have all failed. So releng will rebuild them automatically in the next round ;)
Arriving here via bug 2044028. The change to add %buildsubdir to the note file path has an unfortunate consequence: it means that %{build_ldflags} expands to different values in %prep (where %buildsubdir is not defined) and subsequent scripts, such as %build. I think %prep is the right place to fiddle with upstream build scripts, and I have a few packages that insert %{build_ldflags} into build scripts in %prep. Those packages are now broken. I think %buildsubdir should be removed from the notes file path. The use of %buildsubdir for python noted in comment 21 is different, because it is only used in macros that make sense in %build (%pyproject_wheel, %pyproject_build_lib) or %install (%pyproject_install).
Also, the ocaml package bakes %{build_ldflags} into the ocamlopt tool, which is used to build native OCaml libraries and binaries. The reasoning is that this ensures that all OCaml libraries and binaries are linked with Fedora hardening flags. But now there is a package-specific flag in %{build_ldflags}. What is going to happen when I am building, say, ocaml-integers and ocamlopt is invoked, which then passes the path for the ocaml package's notes file to the linker instead of the path for the ocaml-integers package? I'm afraid that the entire OCaml ecosystem is going to FTBFS the second the mass rebuild is tagged back into Rawhide.
The best solution would be to use %extension_cflags, %extension_ldflags, etc, as the flags that are passed down to build extensions. This is what python3 does, and it allows the two sets to be disconnected.
(In reply to Jerry James from comment #28) > Also, the ocaml package bakes %{build_ldflags} into the ocamlopt tool, which > is used to build native OCaml libraries and binaries. The reasoning is that > this ensures that all OCaml libraries and binaries are linked with Fedora > hardening flags. > > But now there is a package-specific flag in %{build_ldflags}. What is going > to happen when I am building, say, ocaml-integers and ocamlopt is invoked, > which then passes the path for the ocaml package's notes file to the linker > instead of the path for the ocaml-integers package? > > I'm afraid that the entire OCaml ecosystem is going to FTBFS the second the > mass rebuild is tagged back into Rawhide. This specifically affects ocaml-findlib: https://koji.fedoraproject.org/koji/taskinfo?taskID=81939337 and I suppose it explains why even setting "%undefine _package_note_file" does not help here. It would have been nice to know about this a bit earlier rather than the two of us having to rebuild the whole OCaml ecosystem at such short notice. (In reply to Zbigniew Jędrzejewski-Szmek from comment #29) > The best solution would be to use %extension_cflags, %extension_ldflags, > etc, as the flags that are passed down > to build extensions. This is what python3 does, and it allows the two sets > to be disconnected. I think that would be this (untested) patch: diff --git a/ocaml.spec b/ocaml.spec index 7ddabff..cf7e156 100644 --- a/ocaml.spec +++ b/ocaml.spec @@ -185,8 +185,8 @@ make=make # necessary to ensure that generated binaries have Fedora hardening # features. %configure \ - OC_CFLAGS="$CFLAGS" \ - OC_LDFLAGS="$LDFLAGS" \ + OC_CFLAGS="%extension_cflags" \ + OC_LDFLAGS="%extension_ldflags" \ --libdir=%{_libdir}/ocaml \ --host=`./build-aux/config.guess` $make world
(In reply to Richard W.M. Jones from comment #30) > It would have been nice to know about this a bit earlier rather than > the two of us having to rebuild the whole OCaml ecosystem at such short > notice. Hopefully it won't be necessary to rebuild the whole OCaml ecosystem. Let's try adding your patch to the ocaml package. That alone may be enough.
I'm going to play with a local build first and if it works out I'll push it.
(In reply to Richard W.M. Jones from comment #32) > I'm going to play with a local build first and if it works out I'll push it. I had to disable _package_note_flags. The reason is that the OCaml build system mixes up flags for ocamlopt and flags used for building tools too much and I don't want to go extremely deeply into the build system to fix all that. Anyway a new OCaml package is building. (If you want to have a go at fixing this properly fine, but I'm not sure what we're gaining with this package note stuff anyway - it seems like a superfluous feature only designed because of yet another shortcoming of containers - so it feels like disabling it is the way to go).
For 389-ds-base undefining _package_note_flags didn't help. Somehow linker parameters are still inserted with *another* package note file: https://kojipkgs.fedoraproject.org//work/tasks/3806/81993806/build.log libtool: link: gcc -g -O2 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,defs -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -o ldap-agent ldap/servers/snmp/agent-main.o ldap/servers/snmp/agent-ldap-agent.o ldap/servers/slapd/agent-agtmmap.o -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT -Wl,/builddir/build/BUILD/.package_note-net-snmp-5.9.1-13.fc36.x86_64.ld -Wl,--enable-new-dtags -Wl,-z -Wl,relro -Wl,--as-needed -Wl,-z -Wl,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -lcrack -lgcc_s -lc -lrt -lutil -lldap -llber -lsasl2 -lssl3 -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -ldl -L/usr/lib64 -lnetsnmpmibs -lsensors -lrpm -lrpmio -lnetsnmpagent -lnetsnmp -lm -lssl -lcrypto -lpthread /usr/bin/ld: cannot open linker script file /builddir/build/BUILD/.package_note-net-snmp-5.9.1-13.fc36.x86_64.ld: No such file or directory
Looks like it gets it from pkg-config file: [root@a619b6dd5882 /]# cat /usr/lib64/pkgconfig/netsnmp.pc prefix=/usr exec_prefix=/usr includedir=/usr/include libdir=/usr/lib64 Name: netsnmp (Net-SNMP) Description: SNMP (Simple Network Management Protocol) daemon and applications. URL: http://www.net-snmp.org Version: 5.9.1 Cflags: -I${includedir} Libs: -L${libdir} -lnetsnmp Libs.private: -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-net-snmp-5.9.1-13.fc36.x86_64.ld -lm -lm -lssl -lssl -lcrypto -Wl,--enable-new-dtags -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 [root@a619b6dd5882 /]# rpm -q net-snmp-devel net-snmp-devel-5.9.1-13.fc36.x86_64
At least 2 more packages (what is available in the latest rawhide compose) have the same issue: sane-backends.pc:5:ldflags=-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-sane-backends-1.1.1-1.fc36.x86_64.ld dolfin.pc:16:Libs: -L/usr/lib64 -lpetsc -L/lib64 -lhdf5 -lboost_timer -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/dolfin-2019.1.0.post0/.package_note-dolfin-2019.1.0.post0-28.fc36.x86_64.ld -L${libdir} -ldolfin
Just about every *.cma, *.cmt and *.cmti file in Rawhide now has a /builddir/ path embedded in it. Which means we will need to disable this feature in every OCaml package and rebuild them all. $ cd /usr/lib64/ocaml $ for f in `find -type f`; do if grep -sq /builddir $f; then echo $f; fi; done | wc -l 2683
I have been fighting this problem with the xen package as it runs ld directly and ld does not like the -Wl,-z,relro format from LDFLAGS . I do now have a workround for that now (a sed command that tidies up LDFLAGS) though there is still work to do to get the xen package ready for a rawhide build.
The krb5-devel package's krb5-config --libs now gives this output: $ krb5-config --libs -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-krb5-1.19.2-4.fc36.1.x86_64.ld -lkrb5 -lk5crypto -lcom_err which breaks the build of our (external to fedora) packages, and probably any other application that relies on krb5-config to provide link options.
Sorry to be pushy, but I have seen no reponse to this: (In reply to Jerry James from comment #27) > Arriving here via bug 2044028. The change to add %buildsubdir to the note > file path has an unfortunate consequence: it means that %{build_ldflags} > expands to different values in %prep (where %buildsubdir is not defined) and > subsequent scripts, such as %build. I think %prep is the right place to > fiddle with upstream build scripts, and I have a few packages that insert > %{build_ldflags} into build scripts in %prep. Those packages are now broken. > > I think %buildsubdir should be removed from the notes file path. The use of > %buildsubdir for python noted in comment 21 is different, because it is only > used in macros that make sense in %build (%pyproject_wheel, > %pyproject_build_lib) or %install (%pyproject_install). I have just come across yet another package that is broken by %{build_ldflags} having an incorrect note path in %prep. Is %buildsubdir going to be removed?
There's some discussion on the devel list: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/WZKX43VDK4R7BJMA3DEPOHY35RHMGDRX/#O4RK2PKCLU5GD4BTRFARUFOG2W465GS3
I pushed fixed builds of dolfin, sane-backends, net-snmp, 389-ds-base. Unfortunately I might not be able to work on this over the next few days, because I'll be on vacation for a week. Please fix the other packages if you can. (In reply to Marc Dionne from comment #39) > The krb5-devel package's krb5-config --libs now gives this output: > > $ krb5-config --libs > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 > -Wl,-dT,/builddir/build/BUILD/.package_note-krb5-1.19.2-4.fc36.1.x86_64.ld > -lkrb5 -lk5crypto -lcom_err > > which breaks the build of our (external to fedora) packages, and probably > any other application that relies on krb5-config to provide link options. The package was already broken before. "--as-needed" and "--build-id-sha1" are link settings that the package may use, but it doesn't make any sense to advertise them to dependent packages. The "--libs" param is documented as "the compiler options needed to link against library". Similarly to pkgconf files, this should only print the "-l" option (and possibly "-L" if appropriate). Those extraneous options didn't cause build failures, but were wrong anyway. I tried to figure out why "-lcom_err" is included in the list: this looks very suspicious too, since this is a completely separate library provided by e2fsprogs. It seems that krb5 links to com_err. This in itself is no reason to link programs or libraries that want to use libkrb5 directly to com_err. If my conjecture about the reason is true, it should be dropped too. Please just change the package to drop "-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -Wl,-dT,/builddir/build/BUILD/.package_note-krb5-1.19.2-4.fc36.1.x86_64.ld -lcom_err" from --libs output.
The krb5 package_note is preventing certmonger from building which is FTBFS in rawhide for other reasons.
Zbigniew, would you please respond to comment 27 / comment 40?
I would also like to ask a question of Zbigniew: It's my understanding that if we add %undefine _package_note_flags to a file in ocaml-srpm-macros then it would affect every package build when ocaml-srpm-macros is installed. Therefore we will need to add this undefine to every OCaml package instead (about 150 of them). Is there another way to do this only for OCaml package that doesn't involve editing every spec file? I would also like to know if you are considering fixing package notes so it works the same way as strip/debuginfo, as described here: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/K3LNDC3IBSMSDJAA4SBVMKO2ZZD5VW4L/
I just wonder, does the package note file need to have dynamic name? If the name was static, just something like `.package_note`, I think that we would have less issues, because simply, the file would always be there on the build system (assuming that the linker can cope with multiple definitions of single file).
(In reply to Vít Ondruch from comment #46) > I just wonder, does the package note file need to have dynamic name? If the > name was static, just something like `.package_note`, I think that we would > have less issues, because simply, the file would always be there on the > build system (assuming that the linker can cope with multiple definitions of > single file). This would fix Koji builds, but wouldn't fix builds on end user machines where the /builddir/.../.package_note path still wouldn't exist.
(In reply to Richard W.M. Jones from comment #47) > (In reply to Vít Ondruch from comment #46) > > I just wonder, does the package note file need to have dynamic name? If the > > name was static, just something like `.package_note`, I think that we would > > have less issues, because simply, the file would always be there on the > > build system (assuming that the linker can cope with multiple definitions of > > single file). > > This would fix Koji builds, but wouldn't fix builds on end user machines > where the /builddir/.../.package_note path still wouldn't exist. Yes, that is why I mentioned "build system". But in Ruby case, we had the issue already because of the hardening and I don't think that would be different for other ecosystems, unless they chose to apply some workaround such as always installing the redhat-rpm-config.
(In reply to Jerry James from comment #27) > The change to add %buildsubdir to the note > file path has an unfortunate consequence: it means that %{build_ldflags} > expands to different values in %prep (where %buildsubdir is not defined) and > subsequent scripts, such as %build. I think %prep is the right place to > fiddle with upstream build scripts, and I have a few packages that insert > %{build_ldflags} into build scripts in %prep. Those packages are now broken. This is unfortunate. I'll talk with the rpm maintainers and ask for %buildsubdir to be exposed more consistently, probably under a different name. The reason that %buildsubdir was used is that with 'fedpkg local' and similar builds, the linker script ends up in '.', i.e. the dist-git repo root, and it is very awkward to have it there. This is ugly, and also problematic in other respects, because the file will not be cleaned up properly. So despite the problems that this causes, I think we're better of with having %buildsubdir in the default %_package_note_file. Packages where this doesn't work will have to override the macro for now. (In reply to Vít Ondruch from comment #46) > I just wonder, does the package note file need to have dynamic name? If the > name was static, just something like `.package_note`, I think that we would > have less issues, because simply, the file would always be there on the > build system (assuming that the linker can cope with multiple definitions of > single file). Originally, the file name needed to be unique because it was stored in %_builddir, and %_builddir can be shared between multiple parallel builds. With the addition of %buildsubdir, we could get away with a fixed *file* name. But the *directory* part would still vary, and the linker is called from multiple directories, so it needs an absolute path to the linker script anyway. So having a fixed filename wouldn't change much. And I think that having the filename include the package vra is useful because it makes it obvious when the wrong linker script is used. (For example, if by any chance the file from a previous build was used, you'd need to look inside of the file to see that it is wrong. With the details in the file name, it is immediately obvious from the log.) In addition, for some packages %buildsubdir is not defined: no archive is extracted. So we would need to use the current pattern in those cases anyway. So overall, I think the balance is strongly against the fixed name. (In reply to Marc Dionne from comment #39) > The krb5-devel package's krb5-config --libs now gives this output: > > $ krb5-config --libs > -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 > -Wl,-dT,/builddir/build/BUILD/.package_note-krb5-1.19.2-4.fc36.1.x86_64.ld > -lkrb5 -lk5crypto -lcom_err > > which breaks the build of our (external to fedora) packages, and probably > any other application that relies on krb5-config to provide link options. I understand that ocaml has been fixed by opt-out. Sorry for the trouble and thank you for fixing it! (In reply to Richard W.M. Jones from comment #45) > I would also like to know if you are considering fixing package notes > so it works the same way as strip/debuginfo, as described here: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/ > message/K3LNDC3IBSMSDJAA4SBVMKO2ZZD5VW4L/ Just to tie this off: the conclusion of the discussion on the mailing list was that this is not feasible. (In particular, see the mail from Florian Weimer.)
> Just to tie this off: the conclusion of the discussion on the mailing list was that this is not feasible. (In particular, see the mail from Florian Weimer.) That's not a fair summary of what Florian said. He said it's not possible to add the section later, but that space could be reserved up front and then overwritten later, and that seems like a much better idea than this business of messing with build flags and breaking 100s of packages.
Yeah, but reserving the space and then filling it in and adjusting the hashsums and whatnot is at least twice as complicated as simply using the linker script. And obviously it would also require "messing with build flags and breaking 100s of packages". I interpreted Florian's reply to mean that it is not feasible in a realistic way.
(In reply to Zbigniew Jędrzejewski-Szmek from comment #51) > Yeah, but reserving the space and then filling it in and adjusting the > hashsums and whatnot is > at least twice as complicated as simply using the linker script. And > obviously it would also > require "messing with build flags and breaking 100s of packages". The key difference is that the build flags required for the preallocation would be constant, independently of the package being built. This means that it matters less if the flags leaked elsewhere. The dynamic nature of the required build flags is probably the most problematic aspect of the change.
We already modify ELF files in rpm-build late stage for stripping and debuginfo separation. It's quite clearly better to do it there than through the build flags (which in the practical here and now has broken hundreds of packages in Fedora).
(In reply to Zbigniew Jędrzejewski-Szmek from comment #49) > Packages where this doesn't work will have to override the macro for now. Unless this is resolved reasonably quickly, I do not think the "for now" part is accurate(*). I would expect that in a fair number of packages where this is causing a problem the packager will choose to override the macro (for now), and that override will never be reverted in the spec file (as cruft tends to only accumulate). (*) Unless the change owner intends to review and revise the packages for which the override needs to be added today.
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle. Changing version to 36.
(In reply to Gary Buhrmaster from comment #54) > (*) Unless the change owner intends to review and revise the packages for > which the override needs to be added today. I do plan to create statistics over all ELF files. I'll wait for the post post-mass-rebuild package cleanup to wrap up before doing this. Let's see what large groups are excluded and discuss the options then. (In reply to Richard W.M. Jones from comment #53) > We already modify ELF files in rpm-build late stage for stripping and > debuginfo separation. It's quite clearly better to do it there than through the build > flags Hmm, maybe. Do you have an example of how to do this? Saying it's obviously better is one thing, but having an actual implementation is another. I'm happy to switch to a better approach, but please don't make very strong assertions without actual proof.
I just spent a whole day rebuilding all the OCaml packages to fix them, so excuse me if I don't have time to fix the rest of your stuff.
To summarize current situation: - after the mass rebuild, we have 75% ELF files with the note [1], - 4659 packages have and ELF files without a note, - This 4659 includes 722 packages are currently listed as failing in the mass rebuild, those won't have notes, and opt-outs: 895 ghc packages, 210+ ocaml packages, 197 R packages. In other words, the reason for the missing note is known in about half of the cases. [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/RFB45VLZATBJAETGHWDTMJRTADMZY2XI/ -- If you maintain a package that embeds LDFLAGS please follow those general rules: - in pkgconfig files, Libs should contain just -L… and -l…. Flags like -specs=… and -Wl,… must be filtered out. (Libs is used for pkgconf --libs.) - In Fedora we don't have many static libraries, so we don't use Libs.private, but in general the same filtering should be done for Libs.private as for Libs. - in "helper scripts" like krb5-config, --libs/--ldflags and similar switches should follow the same rules as pkgconfig --libs, i.e. filter out everything apart from -L/-l. - packages which specify flags for building extensions should use %extension_cflags/%extension_cxxflags/%extension_fflags/%extension_ldflags, not %build_*. Examples: https://src.fedoraproject.org/rpms/krb5/c/970430cbffb6170964d18fc96dee98d787f6ea49 https://src.fedoraproject.org/rpms/dolfin/c/b0d7edf0e998bf44ee7f66be8fb3a6fdfedc5a10 https://src.fedoraproject.org/rpms/net-snmp/c/a55907366ff0e7d7af4896d76e375819eb0f7e0a https://src.fedoraproject.org/rpms/sane-backends/c/f254fc3fc232e32d85220325f3b0fe044fb03ac5 https://src.fedoraproject.org/rpms/staden-io_lib/c/f9574fa9c417aa1d82ee5ffe11bb857c1213c400 https://src.fedoraproject.org/rpms/ruby/pull-request/110#request_diff
(In reply to Zbigniew Jędrzejewski-Szmek from comment #58) > If you maintain a package that embeds LDFLAGS please follow those general > rules: > > - in pkgconfig files, Libs should contain just -L… and -l…. > Flags like -specs=… and -Wl,… must be filtered out. > (Libs is used for pkgconf --libs.) > > - In Fedora we don't have many static libraries, so we don't use > Libs.private, > but in general the same filtering should be done for Libs.private as for > Libs. I still wonder, how can pkg-config help with this. If this is the state of the art practice, then pkg-config itself should help to filter such options. Or we should have BRP to fix this. Or something. I don't think this is fair to leave it to be solved by individual packages.
pkgconf files are not a really a problem, because they are all easy to fix and have been all fixed (*). I think trying to improve the output in the reader is the wrong place: in particular such fixup cannot solve overlinking, i.e. the common practice of inserting the libraries own dependnecies into the Libs line. It seems nicer to just do this cleanly at the origin. The big problem are the various "*-config" helpers. Those can't be cleaned up automatically, since they don't even have a standarized interface and for an outsider it's hard to say what various options are used for. I also think that spending time on them is a waste: pkgconfig is a much more scalable solution, let's just switch to that everywhere. Nowadays libraries which do not provide a pkgconfig file are rare. (*) Some .pc files have garbage in Libs.private, but since we don't ship static libraries, this is very unlikely to matter in practice. Some .pc files have "-specs=" or "-Wl,-z,relro -Wl,--as-needed -Wl,-z,now" (libresample, cloog-isl, libR), but meh, I'll leave it to those maintainers to clean up.
What is the status here? I guess that the (1) from comment 0 is still not addressed. Not sure about (2).
Still breaks OCaml. We have disabled this in the ocaml-* packages until the implementation of it is fixed properly in Fedora.
Yes, both (1) and (2) from the original report are "unaddressed". For (1), I don't expect any generic solution. The desire to have different package versions is incompatible with the current approach of injecting the metadata during build. The build phase is common to all subpackages, and the division into subpackages happens later and in a different layer. Right now the macros check if %{name} and %{_target_cpu} are defined and then the spec script uses $RPM_PACKAGE_NAME, $RPM_PACKAGE_VERSION, $RPM_PACKAGE_RELEASE, $RPM_ARCH. If somebody really wants to, they could override those in the right place in %build. (But does it really matter? The idea is that the metadata identifies the package. I think that the most common case is when the -doc subpackage has a different version, which isn't relevant for this case. And even in the cases where the package with different versions has ELF binaries, if we use the "main" package version, we still end up with something reasonable that identifies the main package unambiguously.) For (2), the solution needs to happen in the package that "captures" the build flags. As described above, %extension_cflags/%extension_cxxflags/%extension_fflags/%extension_ldflags are the right tools to use. Obviously, skipping the notes is always a valid option. I'd love to have 100% coverage, but it's OK if it's not.
You're again ignoring that there's a much better way than changing build flags. Do it the same way every other RPM feature like this is implemented such as strip & debuginfo - as a post-processing phase.
(In reply to Zbigniew Jędrzejewski-Szmek from comment #63) > For (2), the solution needs to happen in the package that "captures" the > build flags. > As described above, > %extension_cflags/%extension_cxxflags/%extension_fflags/%extension_ldflags > are the right tools to use. Actually the (2) might be better. This is the current situation if I enable package notes for Ruby: ~~~ gcc -shared -o nio4r_ext.so bytebuffer.o monitor.o nio4r_ext.o selector.o -L. -L/usr/lib64 -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -m64 -lruby -lm -lc ~~~ Not sure what package pulls in the `/usr/lib/rpm/redhat/redhat-package-notes` file, so this might still be issue for packages installed using `gem` command, but at least, it does not try to read non-existing file. I think I'll try to re-enable the package notes for Ruby.
> Fixed In Version: redhat-rpm-config-210-1.fc36 Just let me put the commit that actually fixed this issue at redhat-rpm-config-210-1.fc36 for the record. The fix is to strip the package_note flag in the %__extension_strip_flags macro. https://src.fedoraproject.org/rpms/redhat-rpm-config/c/357f950c28ce91540160a26f3a05a529c0ed142c?branch=f36
I faced the issue below with the redhat-rpm-config-233-1.fc38.noarch, package-notes-srpm-macros-0.5-6.fc38.noarch, rpm-4.18.0-3.fc38.x86_64, when building extension module in Ruby. ``` <mock-chroot> sh-5.2$ gem install byebug Fetching byebug-11.1.3.gem ... gcc -shared -o byebug.so breakpoint.o byebug.o context.o locker.o threads.o -L. -L/usr/lib64 -L. -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -m64 -lruby -lm -lpthread -lc gcc: fatal error: environment variable ‘RPM_ARCH’ not defined ``` You can see the https://bugzilla.redhat.com/show_bug.cgi?id=2142119 for details.
Ah, so I was wrong and the situation is much worse, because now not just the file needs to be installed but it also assumes that the build is done under rpmbuild environment. Oh my. @Jun: thx for spotting this.
This message is a reminder that Fedora Linux 36 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '36'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 36 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
we can now the values values of the variables with `rpm -E %___build_pre_env` [1] if I copy an run the result on terminal , we will to be able to run locally the build commands , I guess [1] RPM_SOURCE_DIR="/home/sergio/rpmbuild/SOURCES" RPM_BUILD_DIR="/home/sergio/rpmbuild/BUILD" RPM_OPT_FLAGS="-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection" RPM_LD_FLAGS="-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 " RPM_ARCH="x86_64" RPM_OS="linux" RPM_BUILD_NCPUS="8" export RPM_SOURCE_DIR RPM_BUILD_DIR RPM_OPT_FLAGS RPM_ARCH RPM_OS RPM_BUILD_NCPUS RPM_LD_FLAGS RPM_DOC_DIR="%{_docdir}" export RPM_DOC_DIR RPM_PACKAGE_NAME="%{NAME}" RPM_PACKAGE_VERSION="%{VERSION}" RPM_PACKAGE_RELEASE="%{RELEASE}" export RPM_PACKAGE_NAME RPM_PACKAGE_VERSION RPM_PACKAGE_RELEASE LANG=C export LANG unset CDPATH DISPLAY ||: unset DEBUGINFOD_URLS ||: RPM_BUILD_ROOT="/home/sergio/rpmbuild/BUILDROOT/%{NAME}-%{VERSION}-%{RELEASE}.x86_64" export RPM_BUILD_ROOT PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:/usr/lib64/pkgconfig:/usr/share/pkgconfig" export PKG_CONFIG_PATH CONFIG_SITE=${CONFIG_SITE:-NONE} export CONFIG_SITE
(*) we can see the values values of the variables with `rpm -E %___build_pre_env`
(*) we can see the values of the variables with `rpm -E %___build_pre_env`
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle. Changing version to 39.
Building raku modules is also affected by this.
This is still broken in Fedora Rawhide. Because it affects the linker, it breaks all uses of the linker in configure scripts unless they happen to be run in an RPM.
I'm able to work around this problem by clearing the unwanted linker options file. I made a to-do (for later) to add a warning from configure scripts in case the problem doesn't go away. Looking at the absence of activity in the git repo, that may be a while.
This message is a reminder that Fedora Linux 39 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 39 on 2024-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '39'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 39 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
I will close this. We keep beating this dead horse with very little movement in the last two years. The notes are present in a big majority of ELF files and are extremely useful in debugging all kinds of crashes and issues. Removing them would make debugging and introspection of the system harder. > 1) It is using the wrong Version/Release in case package has subpackages with different Version/Release to the main package As discussed above, this is not a problem (because the goal is to be able to identify the srpm, so even if some subpackage overrides the version, identification is still possible), and packages that could be affected by this are very rare. If the maintainer still doesn't like this, they can opt out. > 2) It breaks Ruby and possibly other languages, which embeds build flags for their ecosystem. We went through a few different approaches, trying to find a solution that works best. In the final solution the linkers provide --package-metadata option natively and this doesn't require the temporary file in the build directory. This works without problems for a great majority of packages. Language ecosystems need to define extension flags appropriately. The biggest ecosystem is Python and it handles this nicely. The second biggest is Rust and it also handles it nicely. If some other language ecosystem needs to adjust, that it's A PROBLEM IN THAT ECOSYSTEM and needs to be fixed by the maintainers of that ecosystem. Asking for a revert of this feature in redhat-rpm-config is not useful.
Never crossed my mind removing the notes , but in case of errors I think we should have documentation to help people who want build a package that is not already supported. for example the python2.7 (https://src.fedoraproject.org/rpms/python2.7/pull-request/23) # https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects # https://bugzilla.redhat.com/2043092 # The default %%build_ldflags macro contains a reference to a file that only # exists in the builddir of this very package. # The flag is stored in distutils/sysconfig and is used to build extension modules. # As a result, 3rd party extension modules cannot be built, # because the file does not exist when this package is installed. # Python 3 solves this by using %%extension_ldflags in LDFLAGS_NODIST, # however Python 2 does not support LDFLAGS_NODIST, so we opt-out completely. # The exact opt-out mechanism is still not finalized, so we use all of them: %undefine _package_note_flags %undefine _package_note_file %undefine _generate_package_note_file
Please just refer to the "official" documentation, found either in /usr/lib/rpm/macros.d/macros.package-notes-srpm or at https://src.fedoraproject.org/rpms/package-notes/raw/rawhide/f/macros.package-notes-srpm