Bug 2043178 - ld.gold: syntax error, unexpected STRING
Summary: ld.gold: syntax error, unexpected STRING
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: package-notes
Version: 36
Hardware: All
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Zbigniew Jędrzejewski-Szmek
QA Contact:
URL:
Whiteboard:
: 2043758 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-20 18:57 UTC by Pavel Filipensky
Modified: 2023-03-04 15:13 UTC (History)
20 users (show)

Fixed In Version: package-notes-0.4-14.fc36
Clone Of:
Environment:
Last Closed: 2022-03-28 11:04:18 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
.package_note-samba-4.15.4-0.fc36.x86_64.ld (2.06 KB, text/plain)
2022-01-20 18:57 UTC, Pavel Filipensky
no flags Details

Description Pavel Filipensky 2022-01-20 18:57:04 UTC
Created attachment 1852288 [details]
.package_note-samba-4.15.4-0.fc36.x86_64.ld

Description of problem:

fedpkg build fails for f36 for samba. 

config.log shows:

err: /usr/bin/ld.gold: error: /builddir/build/BUILD/.package_note-samba-4.15.4-0.fc36.x86_64.ld:41:8: syntax error, unexpected STRING                                                                                                     
/usr/bin/ld.gold: fatal error: unable to parse script file /builddir/build/BUILD/.package_note-samba-4.15.4-0.fc36.x86_64.ld             


Version-Release number of selected component (if applicable):

binutils-gold-2.37-22.fc36.x86_64

How reproducible:

Always. I am attaching the file /builddir/build/BUILD/.package_note-samba-4.15.4-0.fc36.x86_64.ld             

The problem can be than reproduced using this:

/usr/bin/ld.gold --debug  --script .package_note-samba-4.15.4-0.fc36.x86_64.ld

And the output is:

==============
/usr/bin/ld.gold: error: ../f36.issue/.package_note-samba-4.15.4-0.fc36.x86_64.ld:41:8: syntax error, unexpected STRING
/usr/bin/ld.gold: error: ../f36.issue/.package_note-samba-4.15.4-0.fc36.x86_64.ld: not an object or archive
/usr/bin/ld.gold: Dumping linker script
SECTIONS {
  .note.package READONLY : ALIGN(0x4) {
    BYTE(0x4)
...
    BYTE(0x7d)
  }
}
/usr/bin/ld.gold: error: undefined symbol 'READONLY' referenced in expression
==============

It seems that f36 introduced changes to package information on ELF objects:

https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects#New_system:_.note.package

and that ld.gold is having issues with that...

Comment 1 Nick Clifton 2022-01-21 15:19:02 UTC
(In reply to Pavel Filipensky from comment #0)
Hi Pavel,
 
> /builddir/build/BUILD/.package_note-samba-4.15.4-0.fc36.x86_64.ld           
  
Do you know how this file is generated ?

It is definitely incompatible with the ld.gold linker.  Not is the READONLY
keyword not supported, but the INSERT and AFTER keywords are likewise not 
allowed.

It seems to me that whatever generates this package_note script fragment 
should take into account the linker that is being used.

Comment 2 Jerry James 2022-01-21 23:57:48 UTC
The cvc4 package failed the mass rebuild due to this issue.  There is now a package named package-notes: https://src.fedoraproject.org/rpms/package-notes.  It installs a file named /usr/lib/rpm/macros.d/macros.package-notes-srpm that contains the macros used to generate that file.

Comment 3 Robert-André Mauchin 🐧 2022-01-22 00:17:06 UTC
I have the same issue with package libquentier: https://koji.fedoraproject.org/koji/taskinfo?taskID=81634360

/usr/bin/ld.gold: error: /builddir/build/BUILD/libquentier-69b6cf7eedf8f79e5b2ebd7de15f15c9ca0e891a/.package_note-libquentier-0.5.0-10.fc36.x86_64.ld:43:8: syntax error, unexpected STRING
/usr/bin/ld.gold: fatal error: unable to parse script file /builddir/build/BUILD/libquentier-69b6cf7eedf8f79e5b2ebd7de15f15c9ca0e891a/.package_note-libquentier-0.5.0-10.fc36.x86_64.ld
collect2: error: ld returned 1 exit status


.package_note-libquentier-0.5.0-10.fc36.x86_64.ld is:

SECTIONS
{
    .note.package (READONLY) : ALIGN(4) {
        BYTE(0x04) BYTE(0x00) BYTE(0x00) BYTE(0x00) /* Length of Owner including NUL */
        BYTE(0x80) BYTE(0x00) BYTE(0x00) BYTE(0x00) /* Length of Value including NUL */
        BYTE(0x7e) BYTE(0x1a) BYTE(0xfe) BYTE(0xca) /* Note ID */
        BYTE(0x46) BYTE(0x44) BYTE(0x4f) BYTE(0x00) /* Owner: 'FDO' */
        /* Value: '{"type":"rpm","name":"libquentier","version":"0.5.0-10.fc36","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:36"}' */
        BYTE(0x7b) BYTE(0x22) BYTE(0x74) BYTE(0x79)
        BYTE(0x70) BYTE(0x65) BYTE(0x22) BYTE(0x3a)
        BYTE(0x22) BYTE(0x72) BYTE(0x70) BYTE(0x6d)
        BYTE(0x22) BYTE(0x2c) BYTE(0x22) BYTE(0x6e)
        BYTE(0x61) BYTE(0x6d) BYTE(0x65) BYTE(0x22)
        BYTE(0x3a) BYTE(0x22) BYTE(0x6c) BYTE(0x69)
        BYTE(0x62) BYTE(0x71) BYTE(0x75) BYTE(0x65)
        BYTE(0x6e) BYTE(0x74) BYTE(0x69) BYTE(0x65)
        BYTE(0x72) BYTE(0x22) BYTE(0x2c) BYTE(0x22)
        BYTE(0x76) BYTE(0x65) BYTE(0x72) BYTE(0x73)
        BYTE(0x69) BYTE(0x6f) BYTE(0x6e) BYTE(0x22)
        BYTE(0x3a) BYTE(0x22) BYTE(0x30) BYTE(0x2e)
        BYTE(0x35) BYTE(0x2e) BYTE(0x30) BYTE(0x2d)
        BYTE(0x31) BYTE(0x30) BYTE(0x2e) BYTE(0x66)
        BYTE(0x63) BYTE(0x33) BYTE(0x36) BYTE(0x22)
        BYTE(0x2c) BYTE(0x22) BYTE(0x61) BYTE(0x72)
        BYTE(0x63) BYTE(0x68) BYTE(0x69) BYTE(0x74)
        BYTE(0x65) BYTE(0x63) BYTE(0x74) BYTE(0x75)
        BYTE(0x72) BYTE(0x65) BYTE(0x22) BYTE(0x3a)
        BYTE(0x22) BYTE(0x78) BYTE(0x38) BYTE(0x36)
        BYTE(0x5f) BYTE(0x36) BYTE(0x34) BYTE(0x22)
        BYTE(0x2c) BYTE(0x22) BYTE(0x6f) BYTE(0x73)
        BYTE(0x43) BYTE(0x70) BYTE(0x65) BYTE(0x22)
        BYTE(0x3a) BYTE(0x22) BYTE(0x63) BYTE(0x70)
        BYTE(0x65) BYTE(0x3a) BYTE(0x2f) BYTE(0x6f)
        BYTE(0x3a) BYTE(0x66) BYTE(0x65) BYTE(0x64)
        BYTE(0x6f) BYTE(0x72) BYTE(0x61) BYTE(0x70)
        BYTE(0x72) BYTE(0x6f) BYTE(0x6a) BYTE(0x65)
        BYTE(0x63) BYTE(0x74) BYTE(0x3a) BYTE(0x66)
        BYTE(0x65) BYTE(0x64) BYTE(0x6f) BYTE(0x72)
        BYTE(0x61) BYTE(0x3a) BYTE(0x33) BYTE(0x36)
        BYTE(0x22) BYTE(0x7d) BYTE(0x00) BYTE(0x00)
    }
}
INSERT AFTER .note.gnu.build-id;

What can help find the root cause of this?

Comment 4 Robert-André Mauchin 🐧 2022-01-22 00:21:46 UTC
The package https://github.com/systemd/package-notes states that it requires binutils (>= 2.38), so I don't get why it was approved for prod.

Comment 5 Zbigniew Jędrzejewski-Szmek 2022-01-22 09:32:10 UTC
The patch to allow READONLY was backported:
https://src.fedoraproject.org/rpms/binutils/blob/rawhide/f/binutils-ld-read-only-script.patch
but I guess it works only with bfd.

> /usr/bin/ld.gold: error: undefined symbol 'READONLY' referenced in expression
Does it work if READONLY is removed?

Comment 6 Zbigniew Jędrzejewski-Szmek 2022-01-22 10:55:01 UTC
It turns out that not only READONLY needs to be removed, but also INSERT_AFTER.
I'll add a switch to the package-notes macros to disable those.

Comment 7 Zbigniew Jędrzejewski-Szmek 2022-01-22 11:31:22 UTC
> I have the same issue with package libquentier: https://koji.fedoraproject.org/koji/taskinfo?taskID=81634360
Oh, I see this was already solved by switching to bfd.

> The cvc4 package failed the mass rebuild due to this issue.
https://src.fedoraproject.org/rpms/cvc4/pull-request/2 to use bfd here too.
The build unfortunately still fails on some C++ issues.

> err: /usr/bin/ld.gold: error: /builddir/build/BUILD/.package_note-samba-4.15.4-0.fc36.x86_64.ld:41:8: syntax error, unexpected STRING                                                                                                     
I see that this was already solved by switching to bfd.

I added functionality to the note generation script to disable READONLY and use -T instead of -dT.
But it seems that it will not be necessary anyway.

Comment 8 Jakub Jelinek 2022-01-22 11:34:06 UTC
Note, ld.gold is abandoned/barely supported in binutils, switching to ld.bfd is what I'd strongly recommend in any case.

Comment 9 Zbigniew Jędrzejewski-Szmek 2022-01-22 12:35:14 UTC
With package-notes-0.4-11.fc36 it is possible to add

%global _package_notes_linker gold

and the scripts will try to generate a linker script that works with gold. But as mentioned above,
switching to ld.bfd is recommended.

I'll close this now. Please add me to any bugs that require changes in the package notes macros
or scriptlets.

Comment 10 Kevin Kofler 2022-01-22 13:31:23 UTC
This also affects qt5-qtwebengine. Trying this now:
https://src.fedoraproject.org/rpms/qt5-qtwebengine/c/b36104f4bb5c9ac82056c46249d5a404a6b4b228?branch=rawhide
(drops the use_gold_linker qmake option, so it should hopefully pick up ld.bfd).

Comment 11 Michael Catanzaro 2022-01-22 14:25:19 UTC
*** Bug 2043758 has been marked as a duplicate of this bug. ***

Comment 12 Michael Catanzaro 2022-01-22 14:44:46 UTC
(In reply to Jakub Jelinek from comment #8)
> Note, ld.gold is abandoned/barely supported in binutils, switching to ld.bfd
> is what I'd strongly recommend in any case.

Problem is ld.gold is required for debug fission https://gcc.gnu.org/wiki/DebugFission, without which debug builds take impractically long. So migrating from ld.gold doesn't seem like a good idea without a solution for that.

I can time some builds if requested to compare the impact, but at the time debug fission was released, the impact was huge, and I don't see why it would have changed since then.

Comment 13 Michael Catanzaro 2022-01-22 15:03:24 UTC
(In reply to Michael Catanzaro from comment #12)
> Problem is ld.gold is required for debug fission
> https://gcc.gnu.org/wiki/DebugFission, without which debug builds take
> impractically long. So migrating from ld.gold doesn't seem like a good idea
> without a solution for that.

Um, to be clear, this matters more for incremental builds (dev changes one line in a source code file, then must wait 5+ minutes for ld.bfd to link) than it does for full rebuilds (Fedora package builds will be slow no matter what linker is used).

Comment 14 Kevin Kofler 2022-01-22 15:05:23 UTC
Linking qt5-qtwebengine with ld.bfd also fails, due to a warning and due to -Wl,--fatal-warnings being passed:

ulimit -n 4096 && g++ @/builddir/build/BUILD/qtwebengine-everywhere-src-5.15.8/src/core/release/QtWebEngineCore_o.rsp -Wl,--start-group @/builddir/build/BUILD/qtwebengine-everywhere-src-5.15.8/src/core/release/QtWebEngineCore_a.rsp -Wl,--end-group -Wl,-z,noexecstack -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,relro -Wl,-z,now -Wl,-z,defs -m32 -Wl,-O2 -Wl,--gc-sections -rdynamic -Wl,--enable-new-dtags -Wl,-whole-archive -lqtwebenginecoreapi -Wl,-no-whole-archive -Wl,--no-undefined -Wl,--version-script,QtWebEngineCore.version -Wl,--enable-new-dtags -shared -Wl,-soname,libQt5WebEngineCore.so.5 -o libQt5WebEngineCore.so.5.15.8   /usr/lib/libQt5Quick.so /usr/lib/libQt5Gui.so /usr/lib/libQt5QmlModels.so /usr/lib/libQt5WebChannel.so /usr/lib/libQt5Qml.so /usr/lib/libQt5Network.so /usr/lib/libQt5Positioning.so /usr/lib/libQt5Core.so -lpthread -lGL -lpthread -ldl -lrt -licui18n -licuuc -licudata -lsmime3 -lnss3 -lnssutil3 -lplds4 -lplc4 -lnspr4 -levent -lresolv -ljpeg -lm -lopus -lvpx -lX11 -lXcomposite -lXdamage -lXext -lXfixes -lXrender -lXrandr -lXtst -lpng16 -lz -lwebpmux -lwebpdemux -lwebp -lfreetype -lexpat -lfontconfig -lharfbuzz-subset -lharfbuzz -lre2 -lX11-xcb -lxcb -lxkbcommon -ldbus-1 -lpci -lasound -lsnappy -llcms2 -L/builddir/build/BUILD/qtwebengine-everywhere-src-5.15.8/src/core/api/release -lGL   
/usr/bin/ld: /builddir/build/BUILD/qtwebengine-everywhere-src-5.15.8/src/core/release/obj/third_party/ffmpeg/ffmpeg_nasm/dct32.o: warning: relocation in read-only section `.text'
/usr/bin/ld: warning: creating DT_TEXTREL in a shared object
collect2: error: ld returned 1 exit status

I guess I will have to try "%global _package_notes_linker gold" next.

Comment 15 Kevin Kofler 2022-01-22 15:10:38 UTC
Reverted the previous commit to qt5-qtwebengine (i.e., restored the use_gold_linker flag), trying this instead:
https://src.fedoraproject.org/rpms/qt5-qtwebengine/c/6d39634c0ea5d1d08e51a9816bc8be3af944603f?branch=rawhide

Comment 16 Kevin Kofler 2022-01-22 17:09:24 UTC
Looks like this is not going to work, because the Rawhide buildroot still has only package-notes-srpm-macros-0.4-9.fc36, not 0.4-11.fc36.

Comment 17 Kevin Kofler 2022-01-22 17:11:49 UTC
Actually, "koji wait-repo f36-build --build=package-notes-0.4-11.fc36" tells me it should be in the buildroot now, so I am canceling and resubmitting my qt5-qtwebengine build.

Comment 18 Kevin Kofler 2022-01-22 17:19:36 UTC
Confirmed, package-notes-srpm-macros-0.4-11.fc36 is now in the buildroot according to root.log.

Comment 19 Kevin Kofler 2022-01-22 22:16:09 UTC
The fix does not work:
https://koji.fedoraproject.org/koji/getfile?taskID=81675407&volume=DEFAULT&name=build.log&offset=-4000

Still the same error:
/usr/bin/ld.gold: error: /builddir/build/BUILD/qtwebengine-everywhere-src-5.15.8/.package_note-qt5-qtwebengine-5.15.8-1.fc36.aarch64.ld:44:8: syntax error, unexpected STRING
despite having added:
%global _package_notes_linker gold

Comment 20 Kevin Kofler 2022-01-22 22:50:08 UTC
Trying a build with "%undefine _package_note_file" now:
https://src.fedoraproject.org/rpms/qt5-qtwebengine/c/9ae22d8d063baaa00144ec6c2538c96454e8bdc3?branch=rawhide

Comment 21 Robert-André Mauchin 🐧 2022-01-22 23:27:01 UTC
(In reply to Robert-André Mauchin 🐧 from comment #4)
> The package https://github.com/systemd/package-notes states that it requires
> binutils (>= 2.38), so I don't get why it was approved for prod.

I've elected to not use gold since there was a CMake option to do so in libquentier.

Comment 22 Robert-André Mauchin 🐧 2022-01-22 23:42:46 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #9)
> With package-notes-0.4-11.fc36 it is possible to add
> 
> %global _package_notes_linker gold
> 
> and the scripts will try to generate a linker script that works with gold.
> But as mentioned above,
> switching to ld.bfd is recommended.
> 
> I'll close this now. Please add me to any bugs that require changes in the
> package notes macros
> or scriptlets.

I don't get hoz the change could work:

"-Wl,%["%_package_note_linker" == "bfd"?"-dT":"-T"],%{_package_note_file}"

bfd or not bfd, -T %{_package_note_file} is added, so it still fails when using gold.

Comment 23 Robert-André Mauchin 🐧 2022-01-22 23:50:51 UTC
(In reply to Robert-André Mauchin 🐧 from comment #22)
> (In reply to Zbigniew Jędrzejewski-Szmek from comment #9)
> > With package-notes-0.4-11.fc36 it is possible to add
> > 
> > %global _package_notes_linker gold
> > 
> > and the scripts will try to generate a linker script that works with gold.
> > But as mentioned above,
> > switching to ld.bfd is recommended.
> > 
> > I'll close this now. Please add me to any bugs that require changes in the
> > package notes macros
> > or scriptlets.
> 
> I don't get hoz the change could work:
> 
> "-Wl,%["%_package_note_linker" == "bfd"?"-dT":"-T"],%{_package_note_file}"
> 
> bfd or not bfd, -T %{_package_note_file} is added, so it still fails when
> using gold.

I'm not sure if we are on the same page. The problem is that 'INSERT AFTER .note.gnu.build-id;\n' can't work with gold, this patch doesn't address it.

Comment 24 Kevin Kofler 2022-01-23 05:33:45 UTC
The "%undefine _package_note_file" sledgehammer works (qt5-qtwebengine built successfully with it). It does not really fix the package-notes issue though, it just disables package-notes completely for that specfile's output package(s).

Comment 25 Zbigniew Jędrzejewski-Szmek 2022-01-23 22:12:20 UTC
Apologies. I pushed a build with an incomplete change: there was supposed to be an additional
check which skips "INSERT AFTER", and I wrote the docs for it, but not the actual change. (I was editing
the script by hand to check what works and forgot to actually change the generator).

But it seems that gold isn't happy even with that line removed. So please keep "%undefine _package_note_file"
for now. I'll need more time to figure out how to deal with gold (and lld).

Comment 26 Zbigniew Jędrzejewski-Szmek 2022-01-24 11:42:35 UTC
Yeah, it turns out that gold does something wrong with sections.
When we add the new section, it shifts existing sections (and even gets one section less than before):
│ -  Entry point address:               0x500
│ +  Entry point address:               0x50
│    Start of program headers:          64 (bytes into file)
│ -  Start of section headers:          6328 (bytes into file)
│ +  Start of section headers:          8344 (bytes into file)
│    Flags:                             0x0
│    Size of this header:               64 (bytes)
│    Size of program headers:           56 (bytes)
│ -  Number of program headers:         9
│ +  Number of program headers:         8
│    Size of section headers:           64 (bytes)
│ -  Number of section headers:         36
│ -  Section header string table index: 35
│ +  Number of section headers:         38
│ +  Section header string table index: 37

There's a bug open to add INSERT AFTER [https://sourceware.org/bugzilla/show_bug.cgi?id=15373],
but it's 8 years old.

Opting out seems to be the best we can do for now.

I'll also change %_package_note_linker != "bfd" to disable note insertion, so that it also works
as another opt-out mechanism for consistency.

Comment 27 Nikita Popov 2022-01-25 15:48:05 UTC
When using the clang toolchain, the default linker on armv7hl is lld. Would it be possible to automatically default _package_note_linker to lld in that case, so it's not necessary to explicitly adjust all packages that use or can use a clang toolchain?

Comment 28 Zbigniew Jędrzejewski-Szmek 2022-01-25 17:22:29 UTC
(In reply to Nikita Popov from comment #27)
> When using the clang toolchain, the default linker on armv7hl is lld. Would
> it be possible to automatically default _package_note_linker to lld in that
> case, so it's not necessary to explicitly adjust all packages that use or
> can use a clang toolchain?

I pushed the following change now to rawhide:
%_package_note_linker   %["%_target_cpu" == "armv7hl" && "%{toolchain}" == "clang" ? "lld" : "bfd"]

This will effectively disable the notes if the condition is satisfied.

Comment 29 Tom Stellard 2022-01-25 19:00:50 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #28)
> (In reply to Nikita Popov from comment #27)
> > When using the clang toolchain, the default linker on armv7hl is lld. Would
> > it be possible to automatically default _package_note_linker to lld in that
> > case, so it's not necessary to explicitly adjust all packages that use or
> > can use a clang toolchain?
> 
> I pushed the following change now to rawhide:
> %_package_note_linker   %["%_target_cpu" == "armv7hl" && "%{toolchain}" ==
> "clang" ? "lld" : "bfd"]
> 
> This will effectively disable the notes if the condition is satisfied.

Is there some way we can make this change in redhat-rpm-config?  It would be easier to have it with the rest of the toolchain macros.

Comment 30 Ben Cotton 2022-02-08 20:20:52 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 31 Zbigniew Jędrzejewski-Szmek 2022-03-28 11:04:18 UTC
I'll close this for now. The workaround to disable notes with lld will have to remain in place
until lld gets additional functionality, which doesn't seem to be likely to happen soon at this
point.

> Is there some way we can make this change in redhat-rpm-config?

It'd make some changes easier, but other changes equivalently harder… I'd prefer not to move things
around unless it's clearly better.

Comment 32 Michal Ambroz 2023-03-04 15:13:25 UTC
Iaito in f36. There are no issues with this on f39/38/37, but has newly this problem in f36.
https://kojipkgs.fedoraproject.org//work/tasks/7676/98277676/build.log


Note You need to log in before you can comment on or make changes to this bug.