> Note section size too big (124669428 > 67108864) (Invalid argument)

That's a newish sanity check in libmagic tripping up, added in https://github.com/file/file/commit/e1233247bbe4d2d66b891224336a23384a93cce1

The commit message says to limit to 128M but the limit added in the code is 64M :-/

(In reply to Panu Matilainen from comment #1)
> > Note section size too big (124669428 > 67108864) (Invalid argument)
> 
> That's a newish sanity check in libmagic tripping up, added in
> https://github.com/file/file/commit/e1233247bbe4d2d66b891224336a23384a93cce1
> 
> The commit message says to limit to 128M but the limit added in the code is
> 64M :-/

Do we know why the notes section is so darn large in the first place?

Adding nickc to the CC because I suspect this might be annobin (if not, sorry Nick).

This is also related to this debugedit bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1609013
[RFE] find-debuginfo.sh should use a reliable, dedicated ELF file id tool, not rely on parsing 'file' output
We have the elfutils eu-elfclassify tool, which probably is more approriate than using file.

(In reply to Mark Wielaard from comment #2)
> (In reply to Panu Matilainen from comment #1)
 
> Do we know why the notes section is so darn large in the first place?
> 
> Adding nickc to the CC because I suspect this might be annobin (if not,
> sorry Nick).
 
Oh - it almost certainly is annobin's fault.  As noted in this PR:

https://sourceware.org/bugzilla/show_bug.cgi?id=29993

The annobin notes in libxul.so are ~118MB.  (This is before they are processed
by objcopy to remove duplicates, and before they are shifted into the associated
debuginfo file).

I am not sure if this helps you though.  If necessary it is possible to build
things with annobin disabled, although I would not actually recommend it unless
there is no other solution.

Fix seems to be on the way with https://src.fedoraproject.org/rpms/file/pull-request/22

Not sure if 128MB is future proof, it's still quite near that Firefox 118.9MB, and Thunderbird even 122.3MB ...

FEDORA-2023-eeda237137 has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-eeda237137

FEDORA-2023-eeda237137 has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

That should go into f38 as well, please.

Submitted https://src.fedoraproject.org/rpms/file/pull-request/23

FEDORA-2023-a98e46d6c3 has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2023-a98e46d6c3

FEDORA-2023-a98e46d6c3 has been pushed to the Fedora 38 stable repository.
If problem still persists, please make note of it in this bug report.

This is still present in recent builds of mame even with the recent fix:

error: Recognition of file "/builddir/build/BUILDROOT/mame-0.251-3.fc39.x86_64/usr/bin/mame" failed: mode 100755 , dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=4cb9f6821033586b869d516b865fa1d8e7b8b5f9, for GNU/Linux 3.2.0 Note section size too big (137005660 > 134217728) (Invalid argument)

Example of failing build:
https://koji.fedoraproject.org/koji/taskinfo?taskID=97870540

also for build of nodejs-electron 

error: Recognition of file "/builddir/build/BUILDROOT/nodejs-electron-22.3.2-1.fc38.x86_64/usr/lib64/electron/electron" failed: mode 100755 , dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=edd0f817ba466c988d244bf687760c5dc2d17ddf Note section size too big (138030608 > 134217728) (Invalid argument)

https://copr.fedorainfracloud.org/coprs/sergiomb/electrons/build/5626944/
https://copr.fedorainfracloud.org/coprs/sergiomb/electrons/package/nodejs-electron/

FEDORA-2023-ed3c29231e has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2023-ed3c29231e

FEDORA-2023-ed3c29231e has been pushed to the Fedora 39 stable repository.
If problem still persists, please make note of it in this bug report.

Hi Guys,

  Would anyone be interested in helping me test out an experimental change to annobin
  that produces smaller notes ?

  In order to test out the new feature you need to build with annobin 12.03 or later
  (which should be in the rawhide buildroot by now) and you need to set an environment
  variable called ANNOBIN to "note-format=strings".  This should make the annobin
  plugin produce notes in a new, much smaller format.

  In my local tests building firefox 111.0.1 the build time was reduced from 15100 cpu
  seconds to 14603 seconds, the size of the libxul.so file was reduced from 3.585G to
  3.394G and the annobin data was reduced from 100M to 0.7M.

Cheers
  Nick

PS.  The new format can also be triggered by a command line option to the annobin plugin
  but there are various bugs with gcc's ability to pass options to plugins.  So that is
  why I have implemented support for the environment variable.  It also means that the
  feature can be tested with older versions of annobin - the environment will be ignored,
  and the new note format will not be used, but you will also not receive error messages
  about unrecognised command line options.

How can I check the size of the annobin data exactly? Additionally, when I tried building mame with the environmental variable enabled, the build failed fairly quickly:

Compiling 3rdparty/asmjit/src/asmjit/arm/a64formatter.cpp...
g++    -MMD -MP -MP -DPTR64=1 -DNDEBUG -DCRLF=2 -DLSB_FIRST -DXMD_H -DFLAC__NO_DLL -DNATIVE_DRC=drcbe_x64 -DLUA_COMPAT_ALL -DLUA_COMPAT_5_1 -DLUA_COMPAT_5_2   -m64 -std=c++17 -pipe -O2 -fno-strict-aliasing -O2 -fexceptions -g1 -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -Wno-unknown-pragmas -Wall -Wcast-align -Wformat-security -Wundef -Wwrite-strings -Wno-conversion -Wno-sign-compare -Wno-error=deprecated-declarations -Wno-error=unused-result -Wno-array-bounds -Wno-error=attributes -Wno-stringop-truncation -Wno-stringop-overflow -Wno-nonnull -Wno-stringop-overread -Wno-error=maybe-uninitialized -Wno-error=uninitialized -m64 -std=c++17 -Woverloaded-virtual -Wvla -Wimplicit-fallthrough -Wno-class-memaccess  -o "../../../../linux_gcc/obj/x64/Release/3rdparty/asmjit/src/asmjit/arm/a64formatter.o" -c "../../../../../3rdparty/asmjit/src/asmjit/arm/a64formatter.cpp"
*** buffer overflow detected ***: terminated
*** WARNING *** there are active plugins, do not report this as a bug unless you can reproduce it without enabling any plugins.
Event                            | Plugins
PLUGIN_FINISH_UNIT               | annobin: Generate final annotations
PLUGIN_START_UNIT                | annobin: Generate global annotations
PLUGIN_ALL_PASSES_START          | annobin: Generate per-function annotations
PLUGIN_ALL_PASSES_END            | annobin: Register per-function end symbols
../../../../../3rdparty/asmjit/src/asmjit/arm/a64formatter.cpp: In function 'asmjit::_abi_1_9::Error asmjit::_abi_1_9::a64::FormatterInternal::formatRegister(asmjit::_abi_1_9::String&, asmjit::_abi_1_9::FormatFlags, const asmjit::_abi_1_9::BaseEmitter*, asmjit::_abi_1_9::Arch, asmjit::_abi_1_9::RegType, uint32_t, uint32_t, uint32_t)':
../../../../../3rdparty/asmjit/src/asmjit/arm/a64formatter.cpp:25:25: internal compiler error: Aborted
   25 | ASMJIT_FAVOR_SIZE Error FormatterInternal::formatRegister(
      |                         ^~~~~~~~~~~~~~~~~
Please submit a full bug report, with preprocessed source.
See <http://bugzilla.redhat.com/bugzilla> for instructions.
Preprocessed source stored into /tmp/ccGFkHgx.out file, please attach this to your bugreport.
make[2]: *** [asmjit.make:662: ../../../../linux_gcc/obj/x64/Release/3rdparty/asmjit/src/asmjit/arm/a64formatter.o] Error 1

(In reply to Julian Sikorski from comment #17)
Hi Juliam

> How can I check the size of the annobin data exactly? 

You can use the readelf program to find the size of the sections.  For
normal annobin notes the section is called .gnu.build.attributes, so:

  $ readelf --wide --sections libxul.so | grep .gnu.build.attributes
  [28] .gnu.build.attributes NOTE 00000000097b6d08 96d9278 6406ad8 00      0   0  4

The third number (6406ad8 in this case) is the size of the section in bytes.

With the new notes the section is called .annobin.notes.  So:

  $ readelf --wide --sections libxul.so | grep -e .gnu.build.attributes -e .annobin.notes
  [28] .annobin.notes        STRTAB   0000000000000000 96d8276 000109 01  MS  0   0  1
  [29] .gnu.build.attributes NOTE     00000000097b5d08 96d8380 000168 00      0   0  4

So the size in this case is 0x168.  Plus there are still a few of the 
old format notes around - presumably from something linked into the
libxul.so file but not built by the firefox build process.


> Additionally, when I
> tried building mame with the environmental variable enabled, the build
> failed fairly quickly:

> *** buffer overflow detected ***: terminated
> *** WARNING *** there are active plugins, do not report this as a bug unless

Hmm.  I take it that this does not happen if you do not enable the new note format ?

I am currently using a static buffer in the plugin to assemble each note before 
it is written out, so it looks like I underestimated the size I would need.  Or
better yet, I should make the buffer dynamic and resize it as necessary.  I will
look into this.

Thanks for giving the new format a try. :-)

Cheers
  Nick

Correct, the error only happens when ANNOBIN="note-format=strings" is prepended to %make_build.

(In reply to Julian Sikorski from comment #19)
Hi Julian,

  I have put a new version of annobin into rawhide (12.07) which should fix the buffer overflow problem.  If you have the time, please could you give it a go ?

Cheers
  Nick

I believe you meant 12.06. Anyway, with the new format:
$ ls -lh /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/mame-mame0253/
insgesamt 1,4G
--snip--
-rwxr-xr-x. 1 julas mock 1,3G 25. Apr 20:36 mame

$ readelf --wide --sections /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/mame-mame0253/mame | grep -e .gnu.build.attributes -e .annobin.notes
  [32] .annobin.notes    STRTAB          0000000000000000 13d7e214 13c2f8 01  MS  0   0  1
  [33] .gnu.build.attributes NOTE            00000000144774d0 13eba50c 000fc8 00   L 16   0  4

With the old format:
$ ls -lh /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/mame-mame0253/mame
-rwxr-xr-x. 1 julas mock 1,5G 25. Apr 21:10 /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/mame-mame0253/mame

$ readelf --wide --sections /var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/mame-mame0253/mame | grep .gnu.build.attributes
  [32] .gnu.build.attributes NOTE            00000000144774d0 13d7e214 6caf4b0 00   L 16   0  4

(In reply to Julian Sikorski from comment #21)
Hi Julian,

> I believe you meant 12.06.

True - although there are newer versions in rawhide now.


> Anyway, with the new format:
> -rwxr-xr-x. 1 julas mock 1,3G 25. Apr 20:36

> With the old format:
> -rwxr-xr-x. 1 julas mock 1,5G 25. Apr 21:10 

So a saving of ~200 M.  Not bad.  Hopefully the build time was also less with the new format.

Thanks for giving the feature a test run. :-)

Cheers
  Nick