Bug 1044086 - Fedup 19->20 fails
Fedup 19->20 fails
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: fedup (Show other bugs)
19
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Will Woods
Fedora Extras Quality Assurance
:
: 1044076 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-12-17 14:02 EST by Andy Lutomirski
Modified: 2014-04-21 22:26 EDT (History)
12 users (show)

See Also:
Fixed In Version: fedup-0.8.0-3
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-20 17:57:45 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
journalctl output for failed fedup boot (100.08 KB, text/x-log)
2013-12-17 14:02 EST, Andy Lutomirski
no flags Details

  None (edit)
Description Andy Lutomirski 2013-12-17 14:02:09 EST
Created attachment 837871 [details]
journalctl output for failed fedup boot

fedup --network 20 succeeds.  I reboot into the upgrade GRUB option, enter my passphrase, and the system reboots a few seconds later.  No upgrade happens.

I suspect that this is the cause:

Dec 17 10:50:08 antithesis upgrade-prep.sh[892]: moving mounts into /system-upgrade-root
Dec 17 10:50:08 antithesis upgrade-prep.sh[892]: mount: /system-upgrade-root is not mountpoint or bad option
Dec 17 10:50:08 antithesis upgrade-prep.sh[892]: In some cases useful info is found in syslog - try
Dec 17 10:50:08 antithesis upgrade-prep.sh[892]: dmesg | tail or so.
Dec 17 10:50:08 antithesis upgrade-prep.sh[892]: mount: mount point /system-upgrade-root/sysroot does not exist
Dec 17 10:50:08 antithesis systemd[1]: upgrade-prep.service: main process exited, code=exited, status=1/FAILURE

Complete logs attached.
Comment 1 Andy Lutomirski 2013-12-17 14:09:49 EST
The fact /system-upgrade-root/sysroot does not exist is suspicious.  (/system-upgrade-root only contains 'mnt'.)
Comment 2 Andy Lutomirski 2013-12-17 14:19:54 EST
Creating /system-upgrade-root/sysroot gets to:

Not switching root: /system-upgrade-root does not seem to be an OS tree. /etc/os-release is missing.

and then the upgrade hangs.  (I'm not entirely sure that the hang is caused by that error.)
Comment 3 Will Woods 2013-12-17 16:21:27 EST
Right - it's hanging because the upgrade image hasn't been copied to /system-upgrade-root, for some reason.

Try fedup-0.8.0 (currently in updates-testing) - does the upgrade start if you use that?
Comment 4 Andy Lutomirski 2013-12-17 16:46:10 EST
Grr.  fedup-0.8.0 is re-downloading the world.  Is that fixable?

I'll let you know what the ultimate effect is when that finishes.
Comment 5 Will Woods 2013-12-17 17:23:17 EST
It shouldn't redownload anything, unless it changed. Maybe you got F20 Beta on your first run, and now it's actually downloading the real F20?
Comment 6 Andy Lutomirski 2013-12-17 17:36:24 EST
AFAICT it's really downloading rpms that already exist.

I now have files in /var/lib/fedora-upgrade that are symlinked to /system-upgrade, and new files are being downloaded to /var/tmp/system-upgrade/updates/packages.
Comment 7 Volker Braun 2013-12-17 17:53:45 EST
Can confirm it is redownloading everything.

1) start with F19 release
2) fedup --network 20
3) fail after reboot
4) yum --enablerepo=updates-testing update fedup
5) fedup --network 20

downloads all 2000+ rpms again.
Comment 8 Andy Lutomirski 2013-12-17 18:08:37 EST
FWIW, hardlinking the rpms from .../fedora-upgrade/ to .../system-upgrade reduced the number of downloads needed to 660.  I don't know why (maybe F20 beta -> final).
Comment 9 Andy Lutomirski 2013-12-17 20:11:11 EST
fedup-0.8.0 fails:

Downloading failed: GPG key retrieval failed: [Errno 14] curl#37 - "Couldn't open file /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-x86_64"

This may be related to:
warning: /var/tmp/system-upgrade/fedora/packages/libSM-devel-1.2.1-6.fc20.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 246110c1: NOKEY
Comment 10 Andy Lutomirski 2013-12-17 20:20:54 EST
Fixed by rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-primary

Sigh.  Presumably that's not actually fedup's fault.
Comment 11 Andy Lutomirski 2013-12-17 21:42:47 EST
That last issue was arguably my fault for failing to deal with .rpmnew files in /etc/yum.repos.d.  The update ended up working.

I'm marking this ON_QA.
Comment 12 Volker Braun 2013-12-18 05:17:16 EST
I also had to import keys before it would work (after update to fedup-0.8):

rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-primary
rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-rpmfusion-nonfree-fedora-20

I did not have any .rpmnew files in /etc/yum.repos.d, but I do have rpmfusion and some other repos enabled.
Comment 13 Matthew Booth 2013-12-18 08:09:46 EST
I can confirm that fedup 0.8.0 re-downloads the world.

I can also confirm that fedup 0.8.0 fails with:

Downloading failed: GPG key retrieval failed: [Errno 14] curl#37 - "Couldn't open file /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-x86_64"

This goes away with:
rpmkeys --import /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-20-primary

I'm also using rpmfusion-nonfree, but didn't seem to need the second import listed above.
Comment 14 Matthew Booth 2013-12-18 10:53:48 EST
I can confirm that an upgrade is possible with fedup 0.8.0, so this definitely resolves a critical issue.
Comment 15 Tommi Kyntola 2013-12-18 12:10:20 EST
*** Bug 1044076 has been marked as a duplicate of this bug. ***
Comment 16 Chris Murphy 2013-12-19 12:50:52 EST
It looks like fedup 0.7 and 0.8 use different locations for downloads, is why 0.8 redownloads everything. Doing this avoids that.

mv /var/tmp/fedora-upgrade /var/tmp/system-upgrade
mv /var/lib/fedora-upgrade /var/lib/system-upgrade
Comment 17 Andy Lutomirski 2013-12-20 17:57:45 EST
Volker: you most likely have the wrong value for gpgkey in /etc/yum.repos.d/whatever.repo.

I'm closing this bug.  The fix is in all relevant stable repositories, the problem is well-enough publicized, and the system-upgrade rename thing will probably never be any more fixed than it already is.
Comment 18 Bill McGonigle 2014-04-20 16:42:13 EDT
Used the current fedup as of today on a fully updated f19 and had the same experience as Matthew in comment #13.

If we're not going to fix this, then I'll add the key import step to the wiki so people don't have to wind up on this bug to get an upgrade happening.  As long as it's documented, it doesn't seem unreasonable to have to import the next version's key (though automated would be better for the users' ease-of-use).
Comment 19 Will Woods 2014-04-21 10:28:04 EDT
(In reply to Bill McGonigle from comment #18)
> Used the current fedup as of today on a fully updated f19 and had the same
> experience as Matthew in comment #13.
> 
> If we're not going to fix this, then I'll add the key import step to the
> wiki so people don't have to wind up on this bug to get an upgrade
> happening. 

Problems importing keys are unrelated to the original bug here.

That said: if fedup-0.8.0 is failing to import keys, there's two likely causes.

* If the message is "Couldn't open file /etc/pki/rpm-gpg/<keyfile>":

  You probably have ".repo.rpmnew" files in /etc/yum.repos.d.
  You need to fix your old .repo files.
  See bug 1046332.

* If the message is "Downloading failed: Didn't install any keys"

  You probably installed a *-release package with "--nogpgcheck".
  You need to trust the key(s) for those repo(s).
  See bug 1044086.

> As long as it's documented, it doesn't seem unreasonable to have
> to import the next version's key (though automated would be better for the
> users' ease-of-use).

Keys are imported automatically, as long as they're in the place that the .repo file says they'll be and they were installed by a package you trust.


A gentle reminder: please don't update closed bugs with comments unrelated to the original problem. It just makes things more confusing for everyone involved.
Comment 20 Bill McGonigle 2014-04-21 22:26:17 EDT
Ooops, thanks, Will.  Google found Matt's comment and I didn't look above it.  :P

Note You need to log in before you can comment on or make changes to this bug.