Bug 813973

I have seen it too.

How did you decide that the small boot partition is relevant to the wrong URL being used? The connection isn't immediately apparent.

Can you run preupgrade at a console and paste all the messages that appear on the console, while reproducing the issue?

Thanks!



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Created attachment 581870 [details]
preupgrade console output

I'm going to attach a series of files, first are two preupgrade sessions, the second a continuation of this one which seems to have failed due to mirror synchronisation, but I experienced the bug a couple of days before this particular session and believe it's unconnected.

Created attachment 581871 [details]
preupgrade console output

Preupgrade running successfully up to the point where it prompts to reboot. At this point I checked for the existence of a /boot/preupgrade_filler to remove, but it wasn't present. Went ahead and rebooted. I noticed the "upgrade to" item in grub wasn't the default, but not sure if that's intentional.

Created attachment 581872 [details]
dmesg available in dracut

As described in this bug the upgrade fails and kicks the user to a dracut terminal, this is the available dmesg at that point.

Created attachment 581873 [details]
initramfs tmp contents

From the dracut prompt I tried to find whatever logging information was available, this is a tar of the tmp directory stored in initramfs

BLOCKER: I don't have bug editing privileges but would like to propose this as a blocker for F17Blocker, the reasoning being that it is required by Beta requirements [1] which in turn are required to be met for release.

[1]The installer must be able to successfully complete an upgrade installation from a clean, fully updated default installation (from any official install medium) of the previous stable Fedora release, either via preupgrade or by booting to the installer manually. The upgraded system must meet all release criteria

-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

We believe we figured out why this happens: in the case of a small /boot partition, preupgrade will try to pull stage2 from the network rather than stashing it in /boot and pulling it from there.

In both cases, preupgrade uses the stage2= parameter 'wrong' (or rather, archaically: it uses it the way it existed in F15 and earlier, when you pointed directly to a stage2 file, rather than the way it should be in F17, where you should point to a repository as with repo= ). We think that in the case where stage2 is downloaded, this error is hidden, because preupgrade falls back on the (correctly-specified) repo= parameter as a location for stage2. In the low-space-in-/boot case, the error is fatal, because there's no stage2 in the repo= location to fall back on.

As far as blocker status goes, 2012-05-03 meeting, agreed it's not a blocker, because the low-space-in-/boot case is sufficiently unusual by now (we've defaulted to 500MB /boot for several releases).



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

Well, several releases in Fedora is about as many months... I can understand the decision, at least it's been considered now. It sounds like this bug is located in the preupgrade package itself anyway and therefore an F16 component not affected by the release freeze, so I'll hold onto hope it can still get fixed.

Should the version number for this bug be changed to 16 as it's reported against preupgrade-1.1.10-1.fc16?

Yes.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers

(In reply to comment #9)
> In both cases, preupgrade uses the stage2= parameter 'wrong' (or rather,
> archaically: it uses it the way it existed in F15 and earlier, when you pointed
> directly to a stage2 file, rather than the way it should be in F17, where you
> should point to a repository as with repo= ). We think that in the case where
> stage2 is downloaded, this error is hidden, because preupgrade falls back on
> the (correctly-specified) repo= parameter as a location for stage2. In the
> low-space-in-/boot case, the error is fatal, because there's no stage2 in the
> repo= location to fall back on.

I hit this issue while attempting an F16 to F17 upgrade today, on a system that was originally installed with a 200MB /boot.  As you described, anaconda/dracut treated the stage2 URL provided by preupgrade as a repo URL.

The entry added to grub2 by preupgrade:
linux /upgrade/vmlinuz preupgrade repo=hd::/var/cache/yum/preupgrade ks=hd:UUID=23b3b5f4-d5b3-4661-ad41-caa970f3ca59:/upgrade/ks.cfg stage2=http://mirror.bytemark.co.uk/fedora/linux/development/17/x86_64/os/LiveOS/squashfs.img

The errors seen after booting:
dracut: anaconda fetching installer from http://mirror.bytemark.co.uk/fedora/linux/development/17/x86_64/os/LiveOS/squashfs.img

http://mirror.bytemark.co.uk/fedora/linux/development/17/x86_64/os/LiveOS/squashfs.img/.treeinfo
curl: (22) The requested URL returned error: 404

dracut Warning: can't find installer mainimage path in .treeinfo

http://mirror.bytemark.co.uk/fedora/linux/development/17/x86_64/os/LiveOS/squashfs.img/LiveOS/squashfs.img
curl: (22) The requested URL returned error: 404

The URL straight to the stage2 image ended up being treated as a repo URL, as the .treeinfo and path to squashfs.img were appended again.  To fix, edit the grub menu and remove "/LiveOS/squashfs.img" from the URL.

(clearing NEEDINFO as there are two sets of logs)

Getting this on 6 different preupgrade attempts 4 on VM's, 2 on physical machines. Some w/ 200MB /boot, but others w/ 500MB /boot.

Sadly enough removing the excess URL (LiveOS/squashfs.img) or even switching mirror to knowingly good mirror (mirrors.kernel.org/fedora) with a fixed URL switch return "Warning transient problem, retry in XXX" followed by assortment of "Couldn't resolve host: Unknown error" before dropping to shell.
Looking the at the DHCP server logs, I can see that VM's and host got a valid IP address.

In short, thus far, F17 preupgrade is 0/6.

- Gilboa

... I should add that all the machines (both virtual and physical) have multiple Ethernet controllers.

Managed to get it working by disabling the "excess" NICs and forcing dracut to use the Internet facing NIC.
Oh man.

Then that has nothing at all to do with this bug. Your problem is that dracut is picking the 'wrong' NIC to use. I think it just uses the first enumerated one, if you don't specify which to use. You can specify with the ip= parameter, see https://fedoraproject.org/wiki/Dracut/Options#Network for details. But please don't clutter up this bug any further.

Adam,

Please note that I have two issues: One directly related to this bug report (broken URL) and another which I mentioned in passing (Second post).

Never the less, sorry for the noise.

- Gilboa

Still problem with fedora 17 proper. The URL it used for grabby should be:

stage2=http://..../os/

rather than 

stage2=http://.../os/LiveOS/squashfs.img

Have to manually run grubby at the end of preupgrade to let it work.
Otherwise preupgrade fails with cannot get URL:

http://.../LiveOS/squashfs.img/LiveOS/squashfs.img

Yes. We know. Or else the bug would be closed. =)

*** Bug 826272 has been marked as a duplicate of this bug. ***

I was getting similar errors with a /boot partition of ~497 MB

While running preupgrade, I get the traceback:

Fetched treeinfo from http://mirror.optus.net/fedora/linux/releases/17/Fedora/x86_64/os//.treeinfo
treeinfo timestamp: Wed May 23 06:55:30 2012
/boot/upgrade/vmlinuz checksum OK
/boot/upgrade/initrd.img checksum OK
Traceback (most recent call last):
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 804, in <module>
    widgets = PreUpgradeGtk()
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 396, in __init__
    self._do_main()
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 278, in _do_main
    self.main_preupgrade()
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 503, in main_preupgrade
    stage2file = self.pu.retrieve_non_critical_files()
  File "/usr/lib/python2.7/site-packages/preupgrade/__init__.py", line 573, in retrieve_non_critical_files
    self._retrieve_file(self.mainimage, targetdir, reserve_space=extra_space)
  File "/usr/lib/python2.7/site-packages/preupgrade/__init__.py", line 480, in _retrieve_file
    self.instrepo._getFile(relative=fileinfo, local=local)
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 848, in _getFile
    raise Errors.NoMoreMirrorsRepoError, errstr
yum.Errors.NoMoreMirrorsRepoError: failure: LiveOS/squashfs.img from preupgrade: [Errno 256] No more mirrors to try.

That's clearly wrong twice over, since:
1. http://mirror.optus.net/fedora/linux/releases/17/Fedora/x86_64/os/LivOS/squashfs.img exists (I just downloaded it manually through FF)
2. A copy of that file was already present in /boot/upgrade

In an attempt to start the process over again, I cleared out the downloaded files:
rm -rf /boot/upgrade
rm -rf ~/preupgrade*

and disabled the Adobe flash repo (the only non-Fedora repo I have configured)

However, I'm wondering if there's a more appropriate bug report (or if I need to create a new one). As near as I can tell, while my problem appears to be due to issues with squashfs.img, it's not the specific problem of being unable to cache it in /boot and thus getting an incorrect stage2= entry.

I'm beginning to suspect there are deeper problems with this VM - its NetworkManager is failing to start on a normal boot (however, the network comes up fine after "sudo service NetworkManager restart" post-boot), which doesn't bode well for a successful upgrade.

Bowing out of the thread - sorry for the noise.

(In reply to comment #12)
> The URL straight to the stage2 image ended up being treated as a repo URL,
> as the .treeinfo and path to squashfs.img were appended again.  To fix, edit
> the grub menu and remove "/LiveOS/squashfs.img" from the URL.


I had this problem on x86_64 machines.  The suggestion in comment #12 solved the problem for me.

hughsie, could you fix this one? It's broadly simple - change the URL used when writing stage2= - but the code in preupgrade is a bit too complex for me to grok and fix this myself...

Seen

(In reply to comment #26)
> Seen

Ups, sorry. Just wanted to be CC'ed.

(In reply to comment #12)
I was able to use the workaround listed as well but wondered if it might be easier to use the anaconda command line to download the image as instead.

http://wwoods.fedorapeople.org/doc/boot-options.html#_inst_stage2

Just throwing that in there in case its easier to code one way or the other.

I don't want to be rude, but after running into multiple preupgrade errors, I'm facing this error right now. As I initially installed Fedora 6 on my machine, my boot partition is only 233MB in size. 

The first thing that really annoys me is that the "2012-05-03 meeting" decides that a "low-space-in-/boot case is sufficiently unusual by now". WTF guys? It's not about shrinking the squashfs.img to 150MB in size! It's about to get the damn URL right into my grub.conf! This is a really quick and small hotfix.

But okay, lets say it WOULD BE "sufficiently unusual" (which, looking at this bug, apparently is not the case) then stop preupgrade with a meaningful error message (that my /boot is too small) instead messing arround with my bootloader and sending me into an Dracut emergency shell (and sent me into the datacenter to fix my machine).

As concatenating a wrong URL and putting it into the bootloader config is clearly a bug I would expect that this behaviour got fixed quickly.

BUT, doing a "yum update; preupgrade" on 2012-07-20 and running into this kind of problem is just ridiculous! Dominic Cleal has postet a simple fix for this problem on 2012-05-12 (just removing the LiveOS/squashfs.img). And I gave Fedora nearly two month since the release of F17 to get it right. (because I had troubles with earlier preupgrades right after the release dates that were fixed a few weeks later)

So what is really annoying is not that there are errors in Fedora. It's not about the bugs, its not about manually fixing not booting machines. 
It's about attitude. People outside are tring to use your stuff. And preupgrade is a tool to easily upgrade machines WITHOUT having physical access to machines (to put in DVDs or mess arround with a local keyboard). So having a /boot that is too small is not the easiest thing to fix remotely, unless you have a spare disc or spare space after you partition. This renders preupgrade virtually unusable for systems with a "too small" /boot.

The disturbing implication here is that the number of users who have been using & upgrading Fedora for many many releases is very small.  If that's true, then Fedora has some very serious retention problems.  I have a number of systems that I've been upgrading since around FC3.  If the attitude is that keeping the users who have been using Fedora that long is unimportant, that's sad.

Thats exactly what I was trying to say, just i fewer and better words :)

Sascha: the blocker meeting determines whether bugs are serious enough to warrant delaying the release. It's _not_ about evaluating whether they ought to be fixed or not. The fact that the fix is easy doesn't come into the evaluation at all, because it's just not relevant: the question is how bad the problem is, not how easy it is to fix. A bug being rejected as a blocker doesn't mean it won't get fixed; it just means we won't delay the release if it isn't.

"The disturbing implication here is that the number of users who have been using & upgrading Fedora for many many releases is very small.  If that's true, then Fedora has some very serious retention problems.  I have a number of systems that I've been upgrading since around FC3.  If the attitude is that keeping the users who have been using Fedora that long is unimportant, that's sad."

I don't think your logic holds. You seem to be implicitly assuming only two possible scenarios: people keep upgrading a single Fedora install, or they stop using Fedora. In my experience, most people switch hardware and re-install reasonably frequently - certainly more often, in many cases, than 'every five releases'.

Your logic seems to ignore the fact that one of the reasons why people reinstall reasonably frequently is due to the poor and/or painful experience of upgrading.

Thanks for clarification Adam. I see why this bug would not be considered as blocker for F17.

But as you can see, yesterday I still ran into this bug. So for me as an enduser it is still not fixed. I think for the time between filing a bug and resolving it for end users, the effort to fix this bug can be brought into consideration. And three month are preposterous for this kind of bug (severity and effort to fix).

And as Lonni, I'm using the same installation since FC3 or FC4. (even though I changed the hardware twice) Thats the power of Linux. You can just put you old harddisc into a new machine and it just runs right away. Sometimes you have to fix some network stuff, but in most caeses it just works like a charm.

So concluding that "certainly more often, in many cases, than 'every five releases'" users do a complete reinstall is a hazardous assumption. Or can you provide any statistics (like Smolt) that can proof this as a fact.

https://bugzilla.redhat.com/show_bug.cgi?id=826971 is a duplicate of this bug, isn't it?

 Adam Williamson

I am also upgrading machines with small /boot partitions, for me and for my customers, and still running into endless problems with preupgrade, not fixing immediatly this specific and simple bug is a real shame.

Your assumption that "most people switch hardware and re-install reasonably frequently - certainly more often, in many cases, than 'every five releases'." is definatly absolutly terribly false, especially in a production environment.

It might be true in the geek/developers world, but not int the real users world.

I 100% agree with Sascha Zorn.
This should have been a blocker and the bug should have been fixed within days if not hours.

Please understand, you develop this nice software, but I am the evangelist and distributor in the field, and my customers are your users.
If you want to keep Fedora as a geek only distro, your on the right track and understanding.

Alain.

@Alain, blocker or not, the bug is not in F17 itself and could have been fixed after the release freeze without changing anything in the F17 media. It could even be fixed now. Looking at the workaround (just correct the URL in the grub menu), suggests this should be quite an easy fix to preupgrade.

I have had this problem quite a few times upgrading various machines (from X->17). I would try preupgrade, and the machines on which I hit this error I would simply boot from the installation DVD and upgrade using that.

But now that I've seen the fix in comment #12 I can do all my upgrading with Preupgrade. Hooray!

Will someone put the fix in preupgrade?

Comment #12 doesn't contain a fix. It contains a hand-editing workaround. If someone writes a patch I can apply it to firstboot, but comment #12 is not that. I already asked hughsie to fix it, see comment #25, but I guess he's busy.

(In reply to comment #30)

> I have a number of systems that I've been upgrading since around FC3.


Same here, clearly these have ~100MB /boot partitions, and since preupgrade allows upgrading them, the partition is going to stay that size for the next upgrade, and the next ... it doesn't matter how many releases ago the /boot size increased to 500MB, my partitions don't grow.

Anyway, glad a workaround was mentioned, sad that preupgrade hasn't been fixed.

This bug really annoyed me, so I just cloned the git repository and wrote a bug fix myself.

I can confirm that it works in a VM from F16 -> F17. If this introduces any trouble in an pre F16 environment someone has to put a version check around the fix. (But I guess the update sites are similar on all Fedora versions)

Could someone please talk to an developer to put this patch into the repository?


From 19aac40855bf4e36b188a0478dc9a57db84ca0b0 Mon Sep 17 00:00:00 2001
From: Sascha Zorn <Sascha.Zorn>
Date: Tue, 23 Oct 2012 15:48:36 +0200
Subject: [PATCH] fix #813973

stage2 url points to squashfs.img directly, but should point to "os" folder
---
 preupgrade-cli.py |    2 +-
 preupgrade-gtk.py |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/preupgrade-cli.py b/preupgrade-cli.py
index f6a4dfa..a48950f 100755
--- a/preupgrade-cli.py
+++ b/preupgrade-cli.py
@@ -234,7 +234,7 @@ class PreUpgradeCli(PreUpgrade, YumUtilBase, YumBaseCli):
                 # FIXME: sleep here?
                 self.errorprint(message)
                 # We fall back to getting stage2 from instrepo
-                stage2url = os.path.join(self.instrepo.urls[0], self.mainimage)
+                stage2url = self.instrepo.urls[0]
                 extra_args += " stage2=" + stage2url
 
 
diff --git a/preupgrade-gtk.py b/preupgrade-gtk.py
index 20d116a..7781a71 100755
--- a/preupgrade-gtk.py
+++ b/preupgrade-gtk.py
@@ -517,7 +517,7 @@ class PreUpgradeGtk(PreUpgradeController):
                 if response == gtk.RESPONSE_NO:
                     self.exit_instance(exitcode=1)
                 # Fallback: let anaconda stage1 download stage2 from instrepo
-                stage2url = os.path.join(self.pu.instrepo.urls[0], self.pu.mainimage)
+                stage2url = self.pu.instrepo.urls[0]
                 extra_args += " stage2=" + stage2url
 
         # Step 3: Find out what packages we need to download
-- 
1.7.7.6

(In reply to comment #41)
> This bug really annoyed me, so I just cloned the git repository and wrote a
> bug fix myself.
> 
> I can confirm that it works in a VM from F16 -> F17. If this introduces any
> trouble in an pre F16 environment someone has to put a version check around
> the fix. (But I guess the update sites are similar on all Fedora versions)
> 
> Could someone please talk to an developer to put this patch into the
> repository?

I have also confirmed that this patch fixes the issue.

This works for me.

Here is what I did for the upgrade to be successful

1. Changed the preupgrade-cli.py and preupgrade-gtk.py (as Sascha mentioned in comment 41).
2. Re-ran preugrade (Fedora 17)

Problemo again: Reboot was taking automatically to fedora 16
3. Used grub to navigate to "Update Fedora 17 (Beefy Miracle)"

Problemo again: After squashfs.img download Ananconda tried to start and screen brightness became 0
4. Edited the grub command for the update to add "acpi_backlight=vendor acpi_osi=Linux"

Problemo again: Anaconda started but got an error "Cannot retrieve repository metadata (repomd.xml)"
5. Edited to add a new repository.

Upgrade started perfectly

Sascha's patch in Comment 41 worked for me on a F16 system (I only tested the gtk version though).  7+ months since a workaround has been identified (Comment 12) and 2+ months since a patch has been proposed yet neither have made their way into the repos (as of yesterday) for preupgrade -- so just a friendly bump.

i was planning to do an f17 update, but I'm way too busy with f18 work. sorry.

This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

I'm just a lowly user and have *a lot* to learn before I can make any substantial contributions here, but I have to *strongly* echo the comments made in https://bugzilla.redhat.com/show_bug.cgi?id=813973#c29,  https://bugzilla.redhat.com/show_bug.cgi?id=813973#c30 and https://bugzilla.redhat.com/show_bug.cgi?id=813973#c36. Here it is a year and a half later, and I just ran into this bug, upgrading from FC15 to FC17.

I can appreciate that this is a community effort and people are doing this out of the goodness of their heart, but if Fedora is to be truly accepted as a distribution that anyone can use, the developers of Fedora need to seriously take into account the comments made above. To blow them off as insignificant, unimportant or "too busy" will only drive users away.

Alex,

Both Fedora 15, 16 and soon, 17 are EOL'ed - meaning they are no longer supported.
Pre-upgrade itself has also been replaced by a new tool called fedup.
As such, there's little use in fixing this bug.

- Gilboa

Ran into this trying to upgrade from F-15->F-17 and F-16->F-17.  The systems were originally installed as F-12.  I'd skip F-17, but you have to go through F-17 to use fedup.  It's too bad this didn't get fixed before F-17 EOL (or perhaps F-16 EOL, since I'm trying to upgrade to F-17.)

I do understand that this release is no longer supported.