Bug 2045328 - dwz: FTBFS in Fedora rawhide/f37
Summary: dwz: FTBFS in Fedora rawhide/f37
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dwz
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Mark Wielaard
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: F36FTBFS F37FTBFS
TreeView+ depends on / blocked
 
Reported: 2022-01-25 16:26 UTC by Fedora Release Engineering
Modified: 2022-07-01 23:20 UTC (History)
4 users (show)

Fixed In Version: dwz-0.14-5.fc37 dwz-0.14-4.fc36
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-07-01 01:06:33 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
build.log (24.03 KB, text/plain)
2022-01-25 16:26 UTC, Fedora Release Engineering
no flags Details
root.log (32.00 KB, text/plain)
2022-01-25 16:26 UTC, Fedora Release Engineering
no flags Details
state.log (936 bytes, text/plain)
2022-01-25 16:26 UTC, Fedora Release Engineering
no flags Details

Description Fedora Release Engineering 2022-01-25 16:26:28 UTC
dwz failed to build from source in Fedora rawhide/f36

https://koji.fedoraproject.org/koji/taskinfo?taskID=81770823


For details on the mass rebuild see:

https://fedoraproject.org/wiki/Fedora_36_Mass_Rebuild
Please fix dwz at your earliest convenience and set the bug's status to
ASSIGNED when you start fixing it. If the bug remains in NEW state for 8 weeks,
dwz will be orphaned. Before branching of Fedora 37,
dwz will be retired, if it still fails to build.

For more details on the FTBFS policy, please visit:
https://docs.fedoraproject.org/en-US/fesco/Fails_to_build_from_source_Fails_to_install/

Comment 1 Fedora Release Engineering 2022-01-25 16:26:32 UTC
Created attachment 1854007 [details]
build.log

Comment 2 Fedora Release Engineering 2022-01-25 16:26:34 UTC
Created attachment 1854008 [details]
root.log

file root.log too big, will only attach last 32768 bytes

Comment 3 Fedora Release Engineering 2022-01-25 16:26:35 UTC
Created attachment 1854009 [details]
state.log

Comment 4 Ben Cotton 2022-02-08 21:00:59 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 5 Stephen Gallagher 2022-06-24 15:46:03 UTC
Still having the same issue in F37 and prevents ELN rebuild.

Comment 6 Fedora Update System 2022-06-25 23:21:06 UTC
FEDORA-2022-319033d3d8 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-319033d3d8

Comment 7 Mark Wielaard 2022-06-25 23:23:00 UTC
This was fixed for f36 by backporting some test issues.
Sadly rawhide still sees a (new?) issue.

Comment 8 Fedora Update System 2022-06-26 02:09:12 UTC
FEDORA-2022-319033d3d8 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-319033d3d8`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-319033d3d8

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Mark Wielaard 2022-06-29 16:16:39 UTC
This should now also be resolved for f37/rawhide with dwz-0.14-5.fc37

https://koji.fedoraproject.org/koji/taskinfo?taskID=88887572

Comment 10 Stephen Gallagher 2022-06-29 17:33:52 UTC
It is not actually resolved in Rawhide yet, because it's blocked[1] on a failing test[1].


[1] https://bodhi.fedoraproject.org/updates/FEDORA-2022-6bf9cd01c6
[2] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-pipeline/job/master/150412/testReport/(root)/tests/_tests_dwz_quick_test/

Comment 11 Mark Wielaard 2022-06-29 18:44:35 UTC
(In reply to Stephen Gallagher from comment #10)
> It is not actually resolved in Rawhide yet, because it's blocked[1] on a
> failing test[1].
> 
> 
> [1] https://bodhi.fedoraproject.org/updates/FEDORA-2022-6bf9cd01c6
> [2]
> https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/dist-git-
> pipeline/job/master/150412/testReport/(root)/tests/_tests_dwz_quick_test/

Thanks for noticing. That same test failed for f36.
I waived it with "different gdb behavior. dwz is innocent."
QE is aware.

But for some reason I cannot waive if for rawhide "Token required but invalid"

Comment 12 Mark Wielaard 2022-06-30 11:44:35 UTC
(In reply to Mark Wielaard from comment #11)
> Thanks for noticing. That same test failed for f36.
> I waived it with "different gdb behavior. dwz is innocent."
> QE is aware.
> 
> But for some reason I cannot waive if for rawhide "Token required but
> invalid"

Something waived the result anyway. So dwz now moved into f37 stable.
There is also a dwz-0.14-5.eln120 build, but that failed with:
FAIL: /builddir/build/BUILD/dwz/testsuite/dwz.tests/gdb-add-index.sh
I don't know what that is about though (maybe an issue with gdb in ELN?)
There is no other log and I am not sure how to replicate that build environment.

Comment 13 Stephen Gallagher 2022-06-30 14:22:04 UTC
(In reply to Mark Wielaard from comment #12)
> Something waived the result anyway. So dwz now moved into f37 stable.

I asked the Fedora Infra folks to look into the issue and they resolved it.

> There is also a dwz-0.14-5.eln120 build, but that failed with:
> FAIL: /builddir/build/BUILD/dwz/testsuite/dwz.tests/gdb-add-index.sh
> I don't know what that is about though (maybe an issue with gdb in ELN?)
> There is no other log and I am not sure how to replicate that build
> environment.

I think the relevant note in that log is: "Support for debuginfod is not compiled into GDB."

I've been trying to reproduce this, but for some reason I can't get it to fail on a local build in an ELN container. It fails consistently in mock, however. So a simple `fedpkg --release eln mock-build` will reproduce the issue at least.

Comment 14 Mark Wielaard 2022-06-30 21:06:33 UTC
(In reply to Stephen Gallagher from comment #13)
> (In reply to Mark Wielaard from comment #12)
> > Something waived the result anyway. So dwz now moved into f37 stable.
> 
> I asked the Fedora Infra folks to look into the issue and they resolved it.

Thanks

> > There is also a dwz-0.14-5.eln120 build, but that failed with:
> > FAIL: /builddir/build/BUILD/dwz/testsuite/dwz.tests/gdb-add-index.sh
> > I don't know what that is about though (maybe an issue with gdb in ELN?)
> > There is no other log and I am not sure how to replicate that build
> > environment.
> 
> I think the relevant note in that log is: "Support for debuginfod is not
> compiled into GDB."

That should really not matter because dwz should not use/need debuginfod support.
But maybe that message interferes with the testsuite somehow.

> I've been trying to reproduce this, but for some reason I can't get it to
> fail on a local build in an ELN container. It fails consistently in mock,
> however. So a simple `fedpkg --release eln mock-build` will reproduce the
> issue at least.

I am happy to take a look, but not being familiar with eln I don't know how to do a mockbuild against it.

I assume you meant fedpkg --release eln mockbuild but that gives:

setting SOURCE_DATE_EPOCH=1656460800
Wrote: /home/mark/fedora/dwz/dwz-0.14-5.eln120.src.rpm
INFO: mock.py version 3.0 starting (python version = 3.10.4, NVR = mock-3.0-1.fc35)...
Start(bootstrap): init plugins
INFO: selinux enabled
Finish(bootstrap): init plugins
Start: init plugins
INFO: selinux enabled
Finish: init plugins
INFO: Signal handler active
Start: run
INFO: Start(/home/mark/fedora/dwz/dwz-0.14-5.eln120.src.rpm)  Config(fedora-eln-x86_64)
Start: clean chroot
Finish: clean chroot
Start(bootstrap): chroot init
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled package manager cache
Start(bootstrap): cleaning package manager metadata
Finish(bootstrap): cleaning package manager metadata
INFO: enabled HW Info plugin
Mock Version: 3.0
INFO: Mock Version: 3.0
Start(bootstrap): dnf install
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
Fedora - ELN - Developmental packages for the next Enterprise Linux rel 237  B/s | 196  B     00:00    
Errors during downloading metadata for repository 'eln':
  - Status code: 404 for https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/compose/Everything/x86_64/os/repodata/repomd.xml (IP: 185.141.165.254)
Error: Failed to download metadata for repo 'eln': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
ERROR: Exception(/home/mark/fedora/dwz/dwz-0.14-5.eln120.src.rpm) Config(fedora-eln-x86_64) 0 minutes 1 seconds
INFO: Results and/or logs in: /home/mark/fedora/dwz/results_dwz/0.14/5.eln120
INFO: Cleaning up build root ('cleanup_on_failure=True')
Start: clean chroot
Finish: clean chroot
ERROR: Command failed: 
 # /usr/bin/dnf --installroot /var/lib/mock/fedora-eln-x86_64-bootstrap/root/ --releasever eln --setopt=deltarpm=False --allowerasing --disableplugin=local --disableplugin=spacewalk --disableplugin=versionlock install dnf dnf-plugins-core --setopt=tsflags=nocontexts
No matches found for the following disable plugin patterns: local, spacewalk, versionlock
Fedora - ELN - Developmental packages for the next Enterprise Linux rel 237  B/s | 196  B     00:00    
Errors during downloading metadata for repository 'eln':
  - Status code: 404 for https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/compose/Everything/x86_64/os/repodata/repomd.xml (IP: 185.141.165.254)
Error: Failed to download metadata for repo 'eln': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

Could not execute mockbuild: Failed to execute command.

The same with --release rawhide works fine.

Comment 15 Fedora Update System 2022-07-01 01:06:33 UTC
FEDORA-2022-319033d3d8 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Stephen Gallagher 2022-07-01 13:40:09 UTC
(In reply to Mark Wielaard from comment #14)
> Errors during downloading metadata for repository 'eln':
>   - Status code: 404 for
> https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/compose/
> Everything/x86_64/os/repodata/repomd.xml (IP: 185.141.165.254)
> Error: Failed to download metadata for repo 'eln': Cannot download
> repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
> 

Sorry, you're right. Looks like mock-core-configs is out of date and needs to be updated to point at the correct repositories.

If you edit /etc/mock/templates/fedora-eln.tpl and disable the [eln] repo and enable the [local] repo, it should work fine. (That will switch it from trying to access the old Everything repo and instead point it at Koji's eln-build builtroot repo.)

Comment 17 Mark Wielaard 2022-07-01 23:20:17 UTC
(In reply to Stephen Gallagher from comment #16)
> If you edit /etc/mock/templates/fedora-eln.tpl and disable the [eln] repo
> and enable the [local] repo, it should work fine. (That will switch it from
> trying to access the old Everything repo and instead point it at Koji's
> eln-build builtroot repo.)

Thanks, that worked and allowed me to debug in the mock --shell.
It was indeed that "Support for debuginfod is not compiled into GDB." message.
gdb-add-index prints that to stderr and any output to stderr makes the testcase FAIL.

I think it is a gdb bug:
https://sourceware.org/bugzilla/show_bug.cgi?id=29316

But it is easy to workaround:

--- a/testsuite/dwz.tests/gdb-add-index.sh
+++ b/testsuite/dwz.tests/gdb-add-index.sh
@@ -1,6 +1,8 @@
 cp $execs/hello 1
 
-gdb-add-index 1
+# Redirect gdb-add-index stderr to stdout.
+# https://sourceware.org/bugzilla/show_bug.cgi?id=29316
+gdb-add-index 1 2>&1
 
 readelf -S 1 | grep -q '\.gdb_index'
 
So I build a new package with that fix.
dwz-0.14-6.fc37


Note You need to log in before you can comment on or make changes to this bug.