Bug 2086863 - readelf is trying to reach debuginfod remote server?
Summary: readelf is trying to reach debuginfod remote server?
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: binutils
Version: 36
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Nick Clifton
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-05-16 17:16 UTC by Laurent Bigonville
Modified: 2022-09-15 01:28 UTC (History)
9 users (show)

Fixed In Version: binutils-2.37-30.fc36 binutils-2.37-35.fc36 binutils-2.37-25.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-08-31 10:46:09 UTC
Type: Bug


Attachments (Terms of Use)

Description Laurent Bigonville 2022-05-16 17:16:22 UTC
Description of problem:

When trying to build a RPM on a machine, check-rpath is hanging when running readelf on a ELF shared library

Digging deeper with strace, it seems that readelf is trying to reach the IP 38.165.60.21 on port 443

The problem is that this is blocked by a firewall


Version-Release number of selected component (if applicable):
2.37-27

How reproducible:
Always behind the firewall, work on an other network

Steps to Reproduce:
1. Run readelf executable with port 443 packets dropped by the network

Actual results:
readelf blocks (forever?)

Expected results:
readelf outputs the information about the ELF file and/or should not try to reach the debuginfo service

Additional info:

Comment 1 Nick Clifton 2022-05-17 12:05:07 UTC
Hi Laurent,

  Attempting to contact the debuginfod server is the new default behaviour for readelf (unless configured differently when the readelf binary was built).  There is however a command line option to disable this behaviour: -wE or --debug-dump=do-not-use-debuginfod if you prefer the long version.  If this option is added to the check-rpaths-worker script, then the problem should go away.  As such I would like to reassign this BZ to the "rpm" component.  Do you agree ?

Cheers
  Nick

Comment 2 Laurent Bigonville 2022-05-17 12:31:37 UTC
Hello Nick,

That indeed can be a solution, but I'm also wondering why there is no timeout if the debug info cannot be fetched

So maybe something needs to be changed here too?

Comment 3 Nick Clifton 2022-05-17 13:14:13 UTC
(In reply to Laurent Bigonville from comment #2)

> That indeed can be a solution, but I'm also wondering why there is no
> timeout if the debug info cannot be fetched
> 
> So maybe something needs to be changed here too?

The debuginfod client library does not provide a means for an application (eg readelf) to set a timeout, but it does support a DEBUGINFOD_TIMEOUT environment variable.  So this is something else that check-rpaths-worker could use.

Comment 4 amerey 2022-05-18 19:34:47 UTC
(In reply to Laurent Bigonville from comment #2)
> Hello Nick,
> 
> That indeed can be a solution, but I'm also wondering why there is no
> timeout if the debug info cannot be fetched
> 
> So maybe something needs to be changed here too?

In addition to manually setting the DEBUGINFOD_TIMEOUT environment variable like Nick pointed out, there should be a default timeout of 90 seconds.

Comment 5 Nick Clifton 2022-05-19 11:09:54 UTC
(In reply to amerey from comment #4)

> In addition to manually setting the DEBUGINFOD_TIMEOUT environment variable
> like Nick pointed out, there should be a default timeout of 90 seconds.

There is a default timeout and it is 90 seconds.   Since you say that readelf is hanging, something strange must be going on.  I am going to set a needinfo request for Mark Weilaard, the debuginfod expert.  Maybe he can shed some more light on the situation....

Comment 6 Mark Wielaard 2022-05-19 11:19:37 UTC
38.145.60.21 is debuginfod.fedoraproject.org which likely means you have DEBUGINFOD_URLS=https://debuginfod.fedoraproject.org/ set (which is the default).

The best way to debug debuginfod issues is by setting export DEBUGINFOD_VERBOSE=1
Set that and rerun readelf to see what it is trying to do.

Other useful environment variables are (see man 7 debuginfod-client-config):

       $DEBUGINFOD_RETRY_LIMIT
              This environment variable governs the default limit of retry at‐
              tempts.  If  a  query  failed with errno other than ENOENT, will
              initiate several attempts within the limit.

       $DEBUGINFOD_TIMEOUT
              This environment variable governs the download commencing  time‐
              out for each debuginfod HTTP connection.  A server that fails to
              provide at least 100K  of  data  within  this  many  seconds  is
              skipped. The default is 90 seconds.  (Zero or negative means "no
              timeout".)

       $DEBUGINFOD_MAXTIME
              This environment variable dictates how long the client will wait
              to complete the download a file found on a server in seconds. It
              is best used to ensure that a file is downloaded quickly  or  be
              rejected. The default is 0 (infinite time).

       $DEBUGINFOD_MAXSIZE
              This environment variable dictates the maximum size of a file to
              download in bytes. This is best used if the user would  like  to
              ensure  only  small files are downloaded. A value of 0 causes no
              consideration for size, and the client may attempt to download a
              file of any size.  The default is 0 (infinite size).

Comment 7 Florian Weimer 2022-05-19 11:20:09 UTC
Would it also make sense to restrict debuginfod client activity until readelf actually needs debugging information?

$ strace -econnect readelf -hW /bin/true
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Position-Independent Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x1960
  Start of program headers:          64 (bytes into file)
  Start of section headers:          26520 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         13
  Size of section headers:           64 (bytes)
  Number of section headers:         31
  Section header string table index: 30
connect(9, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("38.145.60.21")}, 16) = -1 EINPROGRESS (Operation now in progress)
connect(10, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("38.145.60.20")}, 16) = -1 EINPROGRESS (Operation now in progress)
+++ exited with 0 +++

Comment 8 Nick Clifton 2022-05-19 12:03:36 UTC
(In reply to Florian Weimer from comment #7)
> Would it also make sense to restrict debuginfod client activity until
> readelf actually needs debugging information?

Probably.  Is it safe to assume that any separate debug info file will only contain debug information + (possibly) symbol tables + (possibly) string tables ?  If so, then we could avoid loading those files if we are not dumping that type of information.

Comment 9 Nick Clifton 2022-05-21 13:12:35 UTC
Ok - I have applied a patch upstream which stops readelf and objdump from being so enthusiastic about following links to separate debug info files.  More specifically it stops the program from following the links when the dump option(s) that are active do not require any debugging information.

I have backport the patch to rawhide, and I am in the process of porting it to F36 and F35.

Fixed in binutils-2.38-11.fc37

Comment 10 Nick Clifton 2022-05-21 18:56:18 UTC
Now also fixed in:
binutils-2.37-30.fc36
binutils-2.37-19.fc35

Comment 11 Fedora Update System 2022-05-21 18:57:31 UTC
FEDORA-2022-d0c6475359 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-d0c6475359

Comment 12 Fedora Update System 2022-05-21 18:57:32 UTC
FEDORA-2022-ffad9aca7a has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-ffad9aca7a

Comment 13 Fedora Update System 2022-05-22 01:43:11 UTC
FEDORA-2022-d0c6475359 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-d0c6475359`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-d0c6475359

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 14 Fedora Update System 2022-05-22 02:14:21 UTC
FEDORA-2022-ffad9aca7a has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-ffad9aca7a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-ffad9aca7a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 15 Fedora Update System 2022-06-11 01:14:43 UTC
FEDORA-2022-57015a1d06 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-57015a1d06`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-57015a1d06

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 16 Fedora Update System 2022-06-11 01:31:27 UTC
FEDORA-2022-4300e4c941 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-4300e4c941`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-4300e4c941

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 17 Fedora Update System 2022-08-06 02:33:05 UTC
FEDORA-2022-6bb4a92d4f has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-6bb4a92d4f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-6bb4a92d4f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 18 Fedora Update System 2022-08-06 02:50:40 UTC
FEDORA-2022-7a51ade8e5 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-7a51ade8e5`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-7a51ade8e5

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 19 Fedora Update System 2022-08-09 01:33:33 UTC
FEDORA-2022-e74fd85484 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-e74fd85484`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-e74fd85484

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 20 Fedora Update System 2022-08-09 02:52:41 UTC
FEDORA-2022-c60ef417af has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-c60ef417af`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-c60ef417af

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 21 Fedora Update System 2022-08-12 01:41:19 UTC
FEDORA-2022-da31238a14 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-da31238a14`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-da31238a14

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 22 Fedora Update System 2022-08-12 02:31:03 UTC
FEDORA-2022-a46f1b2ba2 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-a46f1b2ba2`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-a46f1b2ba2

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 23 Fedora Update System 2022-08-31 09:48:27 UTC
FEDORA-2022-e5a363e490 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2022-e5a363e490`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-e5a363e490

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 24 Fedora Update System 2022-08-31 10:46:09 UTC
FEDORA-2022-a46f1b2ba2 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 25 Fedora Update System 2022-09-15 01:28:33 UTC
FEDORA-2022-e5a363e490 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.