Bug 1514754

Summary: Offline update fails due to cache synchronization failure for “updates” repo
Product: [Fedora] Fedora Reporter: Martin Ueding <mu>
Component: dnf-plugins-extrasAssignee: rpm-software-management
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 26CC: alexvillacislasso, brice.taillefer4, dmach, extras-orphan, ignatenko, jkadlcik, jmracek, John_Sauter, mblaha, misc, paul, rpm-software-management, vmukhame
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-11-30 09:51:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martin Ueding 2017-11-18 10:29:45 UTC
# Description of problem

I would like to upgrade my Fedora 26 installation to 27. The installation
medium used was 22, I think. Previous updates worked well.

Besides the official Fedora repositories I use RPM Fusion, a Spotify Copr
repository and my Open Build Service home repository.

My system uses UEFI and has `/boot` as one partition and `/` and `/home` in a
LUKS encrypted LVM.

# Version-Release number of selected component (if applicable)


# How reproducible

Always.

# Steps to Reproduce

1. Run `dnf system-upgrade download --releasever=27 --allowerasing --refresh`
2. Run `dnf system-upgrade reboot`

# Actual results

The system reboots and I am asked for the LUKS password. Then the messages say
that it attempts to start the offline update. Shortly afterwards, something
fails and the system reboots.

This is the relevant portion of the system log:

    systemd[1]: Reached target System Initialization.
    systemd[1]: Starting System Upgrade using DNF...
    systemd[1]: Started Updates device firmware whilst offline.
    audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fwupd-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    systemd[1]: Listening on D-Bus System Message Bus Socket.
    systemd[1]: Starting Update the operating system whilst offline...
    pk-offline-update[1022]: another framework set up the trigger
    systemd[1]: Started Update the operating system whilst offline.
    audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fwupd-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    systemd[1]: Received SIGRTMIN+20 from PID 387 (plymouthd).
    audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-rfkill comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    kernel: kauditd_printk_skb: 70 callbacks suppressed
    kernel: audit: type=1131 audit(1510838539.176:94): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-rfkill comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
    dnf[1020]: Fehler: Zwischenspeicher für Paketquelle »updates« konnte nicht synchronisiert werden
    systemd[1]: dnf-system-upgrade.service: Main process exited, code=exited, status=1/FAILURE
    systemd[1]: Failed to start System Upgrade using DNF.
    audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-system-upgrade comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
    systemd[1]: dnf-system-upgrade.service: Unit entered failed state.
    systemd[1]: dnf-system-upgrade.service: Failed with result 'exit-code'.
    systemd[1]: Rebooting: service failed
    kernel: audit: type=1130 audit(1510838543.517:95): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-system-upgrade comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'

The most interesting line seems to be this one here:

    dnf[1020]: Fehler: Zwischenspeicher für Paketquelle »updates« konnte nicht synchronisiert werden

That is German for

> Error: failed to synchonize cache for repo »updates«

This seems strange, because it does not have an issue downloading the repo
metadata just before I started the upgrade.

# Expected results

Smooth upgrade to Fedora 27.

# Additional info

This problem seems to occur for other people as well:

- https://ask.fedoraproject.org/en/question/113668/error-failed-to-synchronize-cache-for-repo-updatesbs-after-installing-fedora-27/

- https://ask.fedoraproject.org/en/question/113708/cache-for-updates-cannot-be-synchronized-in-f26-to-f27-offline-upgrade/

- https://ask.fedoraproject.org/en/question/113728/problem-reboot-to-upgrade-from-26-to-27/

Comment 1 Paul van Maaren 2017-11-19 14:03:15 UTC
I have got the same, the exact message in English is:

dnf[1726]: Error: Failed to synchronize cache for repo 'updates'

Comment 2 Brice Taillefer 2017-11-19 20:33:24 UTC
The exact same and I'm french. 

I'm willing to help on this bug but I never developped on a linux system.

I can try whatever is suggested though.

Comment 3 Paul van Maaren 2017-11-20 03:10:25 UTC
In /var/log//dnf.log I found messages like:

2017-11-19T13:42:27Z DEBUG Cannot download 'http://linuxdownload.adobe.com/linux/x86_64/': Cannot download repomd.xml: Curl error (5): Couldn't resolve proxy name for http://linuxdownload.adobe.com/linux/x86_64/repodata/repomd.xml [Could not resolve proxy: proxy.internaldomain.org].

So I disabled all third paty repos, and reran:

dnf -v system-upgrade download --releasever=27
dnf system-upgrade reboot

After that it worked. 

On this system I had changed /etc/dnf/dnf.conf:
metadata_expire=7200
#metadata_expire=86400

I am wondering if it has something to do with that as I am using a very slow ADSL line...

Comment 4 Martin Ueding 2017-11-20 08:58:31 UTC
I doubt that your internet connection is to blame. I tried the upgrade with my 16 MBit/s ADSL2 connection at home and the 100 MBit/s university access. Every time I got the same error.

Trying without third-party repositories is an idea, though.

Comment 5 Paul van Maaren 2017-11-20 13:10:56 UTC
I am not sure how metadata_expire exactly works, but the total download took more than 2 hours (7200 s) on my link. I also noticed messages like:

2017-11-19T09:13:21Z DEBUG virtualbox: using metadata from Wed 18 Oct 2017 16:02:06 BST.
2017-11-19T10:31:57Z DDEBUG repo: downloading from remote: virtualbox, _Handle: metalnk: None, mlist: None, urls ['http://download.virtualbox.org/virtualbox/rpm/fedora/27/x86_64'].
2017-11-19T10:31:57Z DEBUG Cannot download 'http://download.virtualbox.org/virtualbox/rpm/fedora/27/x86_64': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried.
2017-11-19T10:31:57Z WARNING Failed to synchronize cache for repo 'virtualbox', disabling.

Not sure if this is related though, as I upgraded another system with virtualbox without any problems. Looking through my proxy logs I can see that yesterday virtualbox.org gave a 404, but this morning a 200 when repomd.xml was requested.

Comment 6 Martin Ueding 2017-11-20 13:18:19 UTC
I have tried with third party repositories disabled, that did not change anything.

Comment 7 Paul van Maaren 2017-11-20 13:35:54 UTC
Seems we have different issues... Have you seen these?

https://bugzilla.redhat.com/show_bug.cgi?id=1513111
https://github.com/rpm-software-management/dnf-plugin-system-upgrade/issues/58

Comment 8 Brice Taillefer 2017-11-20 19:31:09 UTC
I disabled third party repos ans it worked. I was having the exact same log as  Martin Ueding.

What I did : 
Dnf repolist
Dnf --disablerepo [repo without stars]
Dnf repolist (check if only star repo)

Dnf update

Thème follow what's described in fedora magazine article for upgrading from 26 to 27. 

Worked like a charm.

Comment 9 Brice Taillefer 2017-11-20 21:25:32 UTC
Can't edit, here's the code (was on phone sorry)

sudo dnf repolist
sudo dnf config-manager --set-disabled [repo name without stars]
sudo dnf repolist      /!\(check if only star repo)/!\
sudo dnf update

Comment 10 Michael S. 2017-11-24 19:27:15 UTC
Can one of the reporter try to set  "check_config_file_age = false" in the main section of /etc/dnf/dnf.conf and try the upgrade ?

I did debug the problem as explained on https://bugzilla.redhat.com/show_bug.cgi?id=1514877 but as I did found another work around, I can't test easily now :/

Comment 11 Martin Ueding 2017-11-24 19:57:08 UTC
I have just added this to the configuration and then ran:

    dnf system-upgrade download --releasever=27 --allowerasing --refresh -y

    dnf system-upgrade reboot

It still did not work, the logs the same failure to update cache for “updates”.

Comment 12 Marek Blaha 2017-11-30 09:51:41 UTC

*** This bug has been marked as a duplicate of bug 1492036 ***