Bug 1591225 - dnf tries to contact mirror manager only once
Summary: dnf tries to contact mirror manager only once
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 28
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1593033 1630304 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-14 10:31 UTC by clime
Modified: 2019-05-28 19:03 UTC (History)
17 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2019-05-28 19:03:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
dnf.librepo.log (42.29 KB, text/plain)
2018-06-14 10:31 UTC, clime
no flags Details
Also attaching dnf.log (4.75 KB, text/plain)
2018-06-14 10:38 UTC, clime
no flags Details

Description clime 2018-06-14 10:31:25 UTC
Created attachment 1451287 [details]
dnf.librepo.log

Description of problem:

In Copr builds and even in personal use, I can find the following error occasionally:

Error: Error downloading packages:
  Status code: 503 for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f26&arch=x86_64

Meaning that mirror manager had a quick temporary outage. The problem is that this problem occurs for Copr builds relatively often.

There was already some investigation done in the thread here:

https://pagure.io/fedora-infrastructure/issue/6957

...pointing out that dnf code could be fixed to actually try contacting mirror manager more than only once.

Would this be possible to fix/implement? It could quite significantly improve success rate of Copr builds and also a little improve the user experience.

Version-Release number of selected component (if applicable):
Version      : 2.7.5
Release      : 12.fc28

How reproducible:
Occasionally when mirrors.fedoraproject.org drops out. Frequently seen in Copr builds.

Steps to Reproduce:
1. dnf install ...

Actual results:
Error: Error downloading packages:
  Status code: 503 for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f26&arch=x86_64


Expected results:
A packages/packages are successfully fetched and installed.

Comment 1 clime 2018-06-14 10:38:26 UTC
Created attachment 1451288 [details]
Also attaching dnf.log

Please, let me know if more logs than dnf.log and dnf.librepo.log are needed.

Comment 3 Adrian Reber 2018-06-20 12:03:24 UTC
*** Bug 1593033 has been marked as a duplicate of this bug. ***

Comment 4 Adrian Reber 2018-07-02 08:47:02 UTC
Just for completeness I am mentioning another fedora-infrastructure ticket which mentions similar problems: https://pagure.io/fedora-infrastructure/issue/6706

Comment 5 clime 2018-07-18 13:50:09 UTC
Today again, I encountered this issue during my normal personal use.

Comment 6 Jonathan Wakely 2018-07-18 13:56:30 UTC
I'm seeing Copr failures that seem to be related to this e.g.

https://copr.fedorainfracloud.org/coprs/jwakely/autotools-gcc/build/778626/

Two of the five builds failed for unreproducable reasons, which seem likely to be transient network outages.


In https://copr-be.cloud.fedoraproject.org/results/jwakely/autotools-gcc/fedora-27-x86_64/00778626-automake-gcc-compat/builder-live.log I see:

Start: clean chroot
INFO: unmounting tmpfs.
INFO: unmounting tmpfs.
Finish: clean chroot
ERROR: Command failed: 
 # /usr/bin/systemd-nspawn -q -M 0f00dacdde52474a90f666351de706da -D /var/lib/mock/778626-fedora-27-x86_64-bootstrap-1531919712.809825/root -a --setenv=TERM=vt100 --setenv=SHELL=/bin/bash --setenv=HOME=/builddir --setenv=HOSTNAME=mock --setenv=PATH=/usr/bin:/bin:/usr/sbin:/sbin --setenv=PROMPT_COMMAND=printf "\033]0;<mock-chroot>\007" --setenv=PS1=<mock-chroot> \s-\v\$  --setenv=LANG=en_US.UTF-8 --setenv=LC_MESSAGES=C.UTF-8 --setenv=LD_PRELOAD=/var/tmp/tmp.mock.500_hike/$LIB/nosync.so /usr/bin/dnf builddep --installroot /var/lib/mock/778626-fedora-27-x86_64-1531919712.809825/root/ --releasever 27 --disableplugin=local --setopt=deltarpm=False /var/lib/mock/778626-fedora-27-x86_64-1531919712.809825/root//builddir/build/SRPMS/automake-gcc-compat-1.11.6-4.fc27.src.rpm
Error: Failed to synchronize cache for repo 'fedora'


Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild", line 129, in main
    action(args, config)
  File "/usr/bin/copr-rpmbuild", line 229, in build_rpm
    builder.run()
  File "/usr/lib/python3.6/site-packages/copr_rpmbuild/builders/mock.py", line 47, in run
    self.produce_rpm(srpm, configdir, self.resultdir)
  File "/usr/lib/python3.6/site-packages/copr_rpmbuild/builders/mock.py", line 122, in produce_rpm
    raise RuntimeError("Build failed")
RuntimeError: Build failed



https://copr-be.cloud.fedoraproject.org/results/jwakely/autotools-gcc/fedora-27-x86_64/00778626-automake-gcc-compat/root.log.gz shows:

DEBUG package_manager.py:118:  ['/usr/bin/dnf', '--installroot', '/var/lib/mock/778626-fedora-28-x86_64-bootstrap-1531919803.046647/root/', '--releasever', '28', '--disableplugin=local', '--setopt=deltarpm=False', 'install', 'dnf', 'dnf-plugins-core', 'distribution-gpg-keys']
DEBUG util.py:651:  child environment: None
DEBUG util.py:577:  Executing command: ['/usr/bin/dnf', '--installroot', '/var/lib/mock/778626-fedora-28-x86_64-bootstrap-1531919803.046647/root/', '--releasever', '28', '--disableplugin=local', '--setopt=deltarpm=False', 'install', 'dnf', 'dnf-plugins-core', 'distribution-gpg-keys'] with env {'TERM': 'vt100', 'SHELL': '/bin/bash', 'HOME': '/builddir', 'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'PROMPT_COMMAND': 'printf "\\033]0;<mock-chroot>\\007"', 'PS1': '<mock-chroot> \\s-\\v\\$ ', 'LANG': 'en_US.UTF-8', 'LC_MESSAGES': 'C.UTF-8', 'LD_PRELOAD': '/var/tmp/tmp.mock.s5fxnam0/$LIB/nosync.so'} and shell False
DEBUG util.py:491:  'Copr repository'                                40 kB/s | 3.2 kB     00:00    
DEBUG util.py:489:  BUILDSTDERR: Error: Failed to synchronize cache for repo 'fedora'
DEBUG util.py:632:  Child return code was: 1

Comment 7 Jonathan Wakely 2018-07-18 13:57:03 UTC
And for the f28 failure:

In https://copr-be.cloud.fedoraproject.org/results/jwakely/autotools-gcc/fedora-28-x86_64/00778626-automake-gcc-compat/builder-live.log I see:


Start: clean chroot
INFO: unmounting tmpfs.
Finish: clean chroot
ERROR: Command failed: 
 # /usr/bin/dnf --installroot /var/lib/mock/778626-fedora-28-x86_64-bootstrap-1531919803.046647/root/ --releasever 28 --disableplugin=local --setopt=deltarpm=False install dnf dnf-plugins-core distribution-gpg-keys
'Copr repository'                                40 kB/s | 3.2 kB     00:00    
Error: Failed to synchronize cache for repo 'fedora'


Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild", line 129, in main
    action(args, config)
  File "/usr/bin/copr-rpmbuild", line 229, in build_rpm
    builder.run()
  File "/usr/lib/python3.6/site-packages/copr_rpmbuild/builders/mock.py", line 44, in run
    self.produce_srpm(spec, self.sourcedir, configdir, self.resultdir)
  File "/usr/lib/python3.6/site-packages/copr_rpmbuild/builders/mock.py", line 95, in produce_srpm
    raise RuntimeError("Build failed")
RuntimeError: Build failed


https://copr-be.cloud.fedoraproject.org/results/jwakely/autotools-gcc/fedora-28-x86_64/00778626-automake-gcc-compat/root.log shows:

DEBUG package_manager.py:118:  ['/usr/bin/dnf', '--installroot', '/var/lib/mock/778626-fedora-28-x86_64-bootstrap-1531919803.046647/root/', '--releasever', '28', '--disableplugin=local', '--setopt=deltarpm=False', 'install', 'dnf', 'dnf-plugins-core', 'distribution-gpg-keys']
DEBUG util.py:651:  child environment: None
DEBUG util.py:577:  Executing command: ['/usr/bin/dnf', '--installroot', '/var/lib/mock/778626-fedora-28-x86_64-bootstrap-1531919803.046647/root/', '--releasever', '28', '--disableplugin=local', '--setopt=deltarpm=False', 'install', 'dnf', 'dnf-plugins-core', 'distribution-gpg-keys'] with env {'TERM': 'vt100', 'SHELL': '/bin/bash', 'HOME': '/builddir', 'HOSTNAME': 'mock', 'PATH': '/usr/bin:/bin:/usr/sbin:/sbin', 'PROMPT_COMMAND': 'printf "\\033]0;<mock-chroot>\\007"', 'PS1': '<mock-chroot> \\s-\\v\\$ ', 'LANG': 'en_US.UTF-8', 'LC_MESSAGES': 'C.UTF-8', 'LD_PRELOAD': '/var/tmp/tmp.mock.s5fxnam0/$LIB/nosync.so'} and shell False
DEBUG util.py:491:  'Copr repository'                                40 kB/s | 3.2 kB     00:00    
DEBUG util.py:489:  BUILDSTDERR: Error: Failed to synchronize cache for repo 'fedora'
DEBUG util.py:632:  Child return code was: 1

Comment 8 Kamil Páral 2018-07-20 11:58:19 UTC
I can confirm this is a frequent issue for us in Taskotron. We execute thousands tasks a day, and we see quite frequently DNF errors caused by MirrorManager returning some 5xx error. Also, I see this frequently as a reason why a COPR build failed.

Comment 9 Sandro Bonazzola 2018-07-20 13:41:05 UTC
I've seen often as well.

Comment 10 Martin Kyral 2018-09-19 08:35:37 UTC
*** Bug 1630304 has been marked as a duplicate of this bug. ***

Comment 11 Adrian Reber 2018-11-15 17:46:23 UTC
*** Bug 1593033 has been marked as a duplicate of this bug. ***

Comment 12 Ben England 2018-11-15 18:34:32 UTC
status is POST, is there a fix being tested?  didn't see it here.

Comment 13 Dan Kenigsberg 2019-01-24 15:42:54 UTC
(In reply to Ben England from comment #12)
> status is POST, is there a fix being tested?  didn't see it here.

apparently it is the PR linked from comment 2

Comment 14 Gal Ben Haim 2019-01-24 16:32:18 UTC
Any updates on the fix?
I see that the u/s PR is blocked.

Comment 15 Austin DeFrancesco 2019-01-28 16:36:17 UTC
Hey Gal I found a solution by entering into terminal the following: sudo dnf clean all

Note that you must be using sudo privilege! 

Hope that helps.

Comment 16 Jaroslav Mracek 2019-02-28 09:24:59 UTC
The proposed patch solves the issue only partially. The proper patch should be in librepo.

Comment 17 Ben Cotton 2019-05-02 19:50:11 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 18 Jonathan Wakely 2019-05-15 18:19:16 UTC
This is still present in later releases, right?

Comment 19 Ben Cotton 2019-05-28 19:03:24 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.