Bug 958586

Created attachment 742403 [details]
dmesg log

I'm also facing similar problem. In my case /boot partition is failing for dependency. /boot is an actual partition, not a logical one. Rest of the facts are ALL same as per this post. FC18 upgrade is stopped. Please provide a solution soon.

Messages look like :

[ TIME ] Timed out waiting for device ...
[DEPEND] Dependency failed for /boot
[DEPEND] Dependency failed for Local File Systems.

Created attachment 743200 [details]
Partition details showing my /boot partition

Partition details showing my /boot partition which is getting failed for dependency as mentioned in my post.

Could you attach:
  /etc/fstab
  output of `sudo blkid`
  /proc/mounts
  /proc/cmdline
  /var/log/fedup.log
  /var/log/upgrade.log

My apologies, my initial report should say /boot rather than /home, as per kingsukm.

Files are attached as requested. However, there are no logfiles for fedup.log or upgrade.log, so those are obviously not attached. I have however attached the output of fedup when I initially ran it. Hope this is of some use.

Created attachment 743388 [details]
Output of blkid

Created attachment 743389 [details]
My fstab

Created attachment 743390 [details]
/proc/cmdline

Created attachment 743391 [details]
/proc/mounts

Created attachment 743392 [details]
fedup debug log

This afternoon I began an upgrade from Fedora 17 to Fedora 18 (x86_64) using the latest fedup from updates_testing on my Toshiba laptop and have found this exact same problem.  The upgrade proceeds through several steps in the first few seconds and manages to mount my /home and /usr/local partitions but then it times out waiting for device.  The last three messages I get are identical to those reported above by kingsukm.

Last night I used this comand[CODE]fedup --clean
[/CODE] 

To remove all that fedup did until this bug is fixed. And here come the wird thing, now when I boot FC it go into the same emergency thing and need to input "control d" and it complit the boot ok. It was booting ok before this comand. What is hapenig? 
If I boot an old kernel it boot ok. I suspect fedup is messing with the grub configuration.

(In reply to comment #12)
> Last night I used this comand[CODE]fedup --clean
> [/CODE] 
> 
> To remove all that fedup did until this bug is fixed. And here come the wird
> thing, now when I boot FC it go into the same emergency thing and need to
> input "control d" and it complit the boot ok. It was booting ok before this
> comand. What is hapenig? 
> If I boot an old kernel it boot ok. I suspect fedup is messing with the grub
> configuration.

There was an upgrade parameter in the grub configuration file left by fedup, removed and booting ok now.

I see this as well, but for me it really is /home that fails to mount.  My setup is even simpler, no raid, no extra things in /etc/fstab.  Entering the emergency console I can effortlessly mount /home by hand.

I can add the attachements if desired, but not from continuing from the fedup process.  For some reason my networking doesn't work when it continues from the failed fedup process.

Created attachment 743912 [details]
All except fedup/upgrade logs

I can't reproduce this on my test systems so far. Frustrating.

I'm trying to get some internal QA folks involved in testing so we can sort this out.

Created attachment 745289 [details]
This is the fstab file from my laptop

I am attaching here my /etc/fstab file.  Sorry not to send it sooner.  And here is the output from blkid:

[root@euros ~]# blkid 
/dev/sda1: LABEL="System" UUID="EA2C923A2C9201AD" TYPE="ntfs" 
/dev/sda2: LABEL="TI106163W0C" UUID="2A36B16B36B138A1" TYPE="ntfs" 
/dev/sda3: UUID="08e2a915-d90d-445f-8fc8-6092225b691e" TYPE="ext4" 
/dev/sda5: UUID="1pdnur-14Mn-MSzg-x79u-Vxt4-DTue-Ivd4fO" TYPE="LVM2_member" 
/dev/mapper/vg_euros-LogVol03: UUID="aaa50eb6-fcfb-4970-80f7-545d86506078" TYPE="swap" 
/dev/mapper/vg_euros-LogVol00: UUID="3c40e653-9a5f-4b29-bca4-92d4ee785b0a" TYPE="ext4" 
/dev/mapper/vg_euros-LogVol02: UUID="f9e37f43-b175-47ef-bd25-da52ec5a645a" TYPE="ext4" 
/dev/mapper/vg_euros-LogVol01: UUID="c079be65-f630-4d8c-873f-495e49b2b85b" TYPE="ext4"

I have just tried using fedup-cli on another computer and have had the exact same problem as on my Toshiba laptop.  I want to add a few more details here in the hope that they might provide clues to solve this issue.  The version of fedup, I used today is 0.7.3-4.fc17 downloaded this morning from the updates-testing repo.  The full command that I used to start the upgrade was:

[root@eos ~]# fedup-cli fedup-cli --network 18 --debuglog fedupdebug.log --instrepo http://dl.fedoraproject.org/pub/fedora/linux/releases/18/Fedora/x86_64/os/

(all on one line, of course) while logged in as root.  The log from fedup-cli did not show any errors.  I will attach my /etc/fstab file.

The computer is a Dell Precision T 3500 with a four core Xeon processor.  The output from blkid is:

[root@eos ~]# blkid
/dev/sda1: SEC_TYPE="msdos" LABEL="DellUtility" UUID="07D9-0B04" TYPE="vfat" 
/dev/sda3: LABEL="/boot" UUID="c41a2b5f-96e9-4558-8ae5-d5baa6f7b129" TYPE="ext3" 
/dev/sda5: UUID="7bLDPs-86Uv-WZpG-aNla-hCzl-KIOW-V3X1n7" TYPE="LVM2_member" 
/dev/mapper/VolGroup00-LogVol01: TYPE="swap" 
/dev/mapper/VolGroup00-LogVol04: UUID="9012709f-8b45-4573-bd10-23f2f610488c" TYPE="ext4" 
/dev/mapper/VolGroup00-LogVol00: UUID="03da4a20-965d-40d1-83a0-e110bfffd320" SEC_TYPE="ext2" TYPE="ext3" 
/dev/mapper/VolGroup00-LogVol02: UUID="7970dfde-b122-4a25-ac34-a0efa43f0188" TYPE="ext3" 
/dev/mapper/VolGroup00-LogVol03: UUID="3d87cbb1-d2f2-4095-bf15-a6e093e433d1" TYPE="ext3"

Created attachment 747369 [details]
/etc/fstab from my Dell Precision T3500 running Fedora 17

Perhaps I should also add that I think both my Dell T3500 and my Toshiba Protege both are EFI-based computers.  And, of course, both are running up-to-date Fedora 17 operating systems.

*** Bug 960306 has been marked as a duplicate of this bug. ***

*** Bug 959576 has been marked as a duplicate of this bug. ***

*** Bug 903998 has been marked as a duplicate of this bug. ***

If you're upgrading from Fedora 17 using fedup-0.7.3-4.fc17, this should fix your problem:

  http://koji.fedoraproject.org/koji/buildinfo?buildID=419714

Can someone please try that fedup package and tell me if it fixes the problem?

(If anyone encountered this problem while using something *other* than fedup-0.7.3-4.fc17, please file a new bug.)

from bug #959576

"I also have device timeouts on by _lvm2 monitor_ on /dev/sda1 /boot (ext4), which is strange, the fact that /dev/sda1 is not an lvm volume, just /dev/sda2 ."

To me, seems the fault is of lvm2 monitor service, can't we disable it somehow ?

I have a clone of a F16 custom vm, but is a complicated vm because in only 8 gb of HD , so I could take some time to test it

I have used the new build of fedup just now on my Toshiba Portege that had trouble before.  After starting the upgrade with fedup-cli, I rebooted when all the new packages had been downloaded.  This time the upgrae on reboot seemed to go well.  There was no time-out while waiting to mount /boot.  The upgrade seemed to progress fine and then when it rebooted itself it couldn't quite complete the task.  The boot fails to go into graphical mode and a list of progress points are listed on the screen in text mode.  The last few to list are:

[OK] Started Sendmail Mail Transport Client.
[OK] Started NFS Remote Quota Server.
[OK] Started NFS ID-name mapping daemon.
[OK] Started NFS Mount Daemon.
     Starting Manage, install and Generate Color Profiles...

That's where it ends and the boot process stops altogether.  The kernel is running and I can use Ctrl-Alt F2 to get to a terminal and login there.

When rebooting the system I see that Fedora 18 is not a choice for booting in Grub.  Only the last few Fedora 17 kernels are there to install.  When logged in on a terminal I can see that an F18 version of grub2 has been installed.  However, the grub2-efi package is still the F17 version.

So what to do now?

I installed fedup-0.7.3-5.fc17.noarch, reran fedup which updated a number of FC18 packages since the last attempt, rebooted ... and failed in exactly the same way as before; Timeout mounting the 2nd logical volume  (/home in my case) (cf. Bug #959576)

from bug #959576

I also have device timeouts when starts _lvm2 monitor.service_ on /dev/sda1 /boot (ext4), which is strange, the fact that /dev/sda1 is not an lvm volume, just /dev/sda2 .

To me, seems the fault is of lvm2 monitor service, can't we disable it somehow ?
and Ivor not mention what service timed out . neither others  
in my case, I think, is the first partition which is not a lvm

I think I'm in a similar situation as William. Never had an f18 entry added to grub after the upgrade seemed to finish.

I think I figured out that part of my problem now is that, as I stated in Comment 18, I invoked fedup-cli with a --instrepo argument that pointed at one of the Fedora release 18 repositories.  Since my Fedora 17 had been kept up to date, many of the rpm packages on my Fedora 17 installation were newer than those in the Fedora 18 repository.  I say this because as I grep through the results of 'rpm -qa' after the yesterday's failed upgrade, I see lots of f17 packages along with many f18 packages.  All of the yum* packages are f18, except for yum-langpack.  All of the kernel* packages are f17 and most *gcc* packages are f17.  Also, 'yum repolist' only lists f17 repos.

I used the --instrepo argument because I have never, after at least a dozen attempts on three different computers over several weeks, been able to get fedup-cli to do anything without that argument.  Without the argument it always stops immediately saying it can find no repositories and that --instrepo should be used.  How do I avoid that problem?

I used the new version of fedup linked by Will in comment 24, and the upgrade proceeded past the point it stopped last time. However, after rebooting the PC now just endlessly reboots whenever you switch it on, never reaching the login screen. So the upgrade clearly failed. I don't know whether that's as a result of the fix being incomplete or introducing a new bug, or whether that's a bug that existed anyway in the upgrade process.

If the upgrade starts, then this bug is fixed and you have a different problem.

There's two possibilities:


1) All of the packages were installed but your system doesn't work

(There might be some leftover .fc17 packages - sometimes there isn't a newer .fc18 version of a package; this is not a fedup bug.)

If everything got installed, but your system doesn't work, this is a problem with one (or more) of the packages that were installed.

fedup doesn't control the contents of the packages or package scripts, so I can't fix those. You should examine the contents of /var/log/upgrade.log and/or /var/log/upgrade.journal (using journalctl -D) and see if there's anything in there about package script/install errors.

Also check `package-cleanup --problems` and/or the system logs for missing dependencies or other errors, and file bugs against the offending components.


2) A few (but not all) of the packages were installed

fedup-dracut's "system-upgrade-fedora" program might have crashed; examine /var/log/upgrade.* to see if there's any clues about where/why it crashed, and file a new bug against fedup-dracut.

In both cases you should use the *target* system version in the bug report (i.e. if you're upgrading to F18, it's a F18 bug.


*This* bug concerns the sudden drop to emergency mode, which (AFAICT) is fixed. For other problems, file new bugs as described above.

*** Bug 960878 has been marked as a duplicate of this bug. ***

I noticed this in fedupdebug.log after using the newer version of fedup (after a clean).

[    43.528] (II) fedup.yum:setup_repos() can't find valid repo metadata for fedora-geary
[    43.532] (II) fedup.yum:setup_repos() can't find valid repo metadata for google-chrome
[    43.536] (II) fedup.yum:setup_repos() can't find valid repo metadata for google-talkplugin
[    73.576] (II) fedup.yum:setup_repos() can't find valid repo metadata for updates
[    73.576] (DD) fedup.yum:setup_repos() repos.cache=0
[    73.576] (II) fedup:setup_downloader() disabled repos: fedora-geary google-chrome google-talkplugin updates


The one that is of interest is updates. I suspect this is why I do not see any kernel packages listed in fedupdebug.log or upgrade.log; hence, no f18 kernel entry to grub and system in a strange state with some f18 and some f17 packages.

I'll let someone else decide if a bug needs to be opened on this, as I have moved on and manually updated using yum through rescue disk.

Well as I no longer get thrown into emergency mode, I guess technically this bug is fixed. I now have an unbootable system, but that's a bug for another day.

I ran into this on a VM that started life as an F16 install (GPT), upgraded
via preupgrade to F17.
The updated package in comment 24 got me past the problem, so this fixes the
issue for me.

Just wanted to report I had the exact problem as described in the post and Comment 2 (failed at finding /boot).
I used the updated version of fedup linked in Comment 24, and it worked!
I guess this is unrelated: I had some worrying messages during the actual upgrade (after reboot), i.e. some packages failed to install (very few out of ~2700), but on next reboot the system started into F18 ok, and either these packages were not critical, or not upgrading them wasn't...

Hi, fedup-0.7.3-4.fc17 also fix bug #959576 , on upgrade F17 -> F19 
thanks,

Confirmed that installing fedup-0.7.3-5.fc17 and trying again properly mounted home and the upgrade proceeded normally from there onwards.  Thanks muchly

Just ran into this problem and I too can confirm that fedup-0.7.3-5.fc17 fixes it.

*** Bug 904650 has been marked as a duplicate of this bug. ***

Had a similar problem with one filesystem (/home) being xfs and I too can confirm that fedup-0.7.3-5.fc17 fixes it. THANKS!

(In reply to Ivor Durham from comment #27)
> I installed fedup-0.7.3-5.fc17.noarch, reran fedup which updated a number of
> FC18 packages since the last attempt, rebooted ... and failed in exactly the
> same way as before; Timeout mounting the 2nd logical volume  (/home in my
> case) (cf. Bug #959576)

For my first attempt to get past this problem with the new version (0.7.3.5, see above), I first used "fedup --resetbootloader" to be able to boot my original system normally and then "fedup-cli --network 18 --debuglog fedupdebug.log" again and it still failed. Tonight I used "fedup --clean" and then repeated the setup with fedup, hoping to ensure a clean slate. This still did *not* solve the problem for me. Right before it drops into Emergency Mode I still get:

[ TIME ] Timed out waiting for device dev-mapper-vg_clowder\x2dlv_home.device
[ DEPEND ] Dependency failed for /home
[ DEPEND ] Dependency failed for Local File Systems
Welcome to Emergency Mode ...

Is there anything to be read into the appearance of "\x2d" instead of plain "-" in the timeout message while the "-" appears as itself earlier in the same string?

I can now report on the problem I encountered after getting past this fedup bug.  As reported above in Comment 26, once I got past the time-out problem I was able to upgrade my installation to Fedora 18, but then the ensuing boot did not complete.  A few days ago I tried the new fedup version on another computer and had exactly the same problem.  On that computer I was able to take Will Woods advice and used 'package-cleanup --problems' and eventually discovered that a newer version of harfbuzz (one packaged for fc20) had been installed during the upgrade.  That package wasn't working properly because the version of the libicu libraries that it wanted weren't installed.  I was able to use 'yum downgrade harfbuzz.x86_64' to resolve the problem.  The error of installing the wrong harfbuzz package was reported a couple of weeks ago as bug 873082.  Thanks to Will Woods for his suggestions of what to do to help solve this second problem.

(In reply to Will Woods from comment #24)
> If you're upgrading from Fedora 17 using fedup-0.7.3-4.fc17, this should fix
> your problem:
> 
>   http://koji.fedoraproject.org/koji/buildinfo?buildID=419714
> 
> Can someone please try that fedup package and tell me if it fixes the
> problem?
> 
> (If anyone encountered this problem while using something *other* than
> fedup-0.7.3-4.fc17, please file a new bug.)

I've tested with fedup-0.7.3-5.fc17 and I still get dropped to the the Emergency Mode after timing out waiting for /home

I have a Dell Optiplex 9010 with an Intel 82801 SATA RAID Controller in.

(In reply to kingsukm from comment #3)
> Created attachment 743200 [details]
> Partition details showing my /boot partition
> 
> Partition details showing my /boot partition which is getting failed for
> dependency as mentioned in my post.

Confirmed that installing fedup-0.7.3-5.fc17 and trying again properly mounted home and the upgrade proceeded normally from there onwards. Thanks a lot for fixing the FedUp tool. Just for interest, what changes were made in this new one which fixes the problem ?

http://pkgs.fedoraproject.org/cgit/fedup.git/commit/?h=f17&id=476a14563ba3ca9717fb018bf419d89ed5eeaf8a

+# F17 needs these to start properly
+Wants=udev.service udev-trigger.service
+After=udev.service udev-trigger.service

Hi,

When fedup-0.7.3-5.fc17 will be pushed to update-testing repositories ?

Regards.

Will hasn't submitted it as an update yet - Will? If he doesn't soon, I will.

fedup-0.7.3-5.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/fedup-0.7.3-5.fc17

Package fedup-0.7.3-5.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing fedup-0.7.3-5.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-10433/fedup-0.7.3-5.fc17
then log in and leave karma (feedback).

fedup-0.7.3-5.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

As noted in comment 45, the problem persists for me.

As noted in comment 27 and comment 43 this problem is not resolved for me by fedup-0.7.3.5

fedup-0.7.3-4 is still in the F18 repository, so if you're trying to use fedup to go from F18->F19 like I did last night, you'll run into this problem.

I can fix it by doing 'lvchange -aay /vg_***/lv_home' when i get dumped into the emergency prompt. Exiting with ctrl-d to resume the boot process gets me back up and running, but it doesn't stick and I'm back to where I was next time I reboot.

Eugene: um, this bug did not affect F18 in any of our testing...sounds like you're seeing something else? Can you file it separately?

(In reply to Adam Williamson from comment #56)
> Eugene: um, this bug did not affect F18 in any of our testing...sounds like
> you're seeing something else? Can you file it separately?

sure thing

Did you upgrade from F17 to F18 previously? wwoods is theorizing that you may run into the problem you did with a system that didn't start out as F18, but as something older. I'll try and find a minute to test that path today.

The machine I was upgrading went from F17 to F18 via preupgrade. One of the disks is set up as an LVM disk with two volumes, / and /home. The rest of the disks in the system are non-LVM ext4 formatted partitions.

Like the OP, I ran fedup, it downloaded many packages, rebooted and the many packages were installed. Upon rebooting, I get the same Time out/Dependency failed error messages and get dumped into the emergency mode prompt. Only one of the LVM volumes (/) is visible at this point. The /home volume shows up when I run lvs, but is listed as inactive.

Another machine I upgraded from F17 to F18 using fedup went without a hitch. That one also has an LVM disk but with only one volume.

when i get back home to the machine i had the problem with, I'll file a new report with as much info as i can extract.

Filed a new bug at https://bugzilla.redhat.com/show_bug.cgi?id=979723

Comment 52 said "If problems still persist, please make note of it in this bug report" which we did in comments 53 and 54. Given that fedup is now our only upgrade option and FC17 is approaching the end of its maintenance life in about a month, how are we supposed to proceed? Is our only option to start over with a fresh installation and go through the pain of reconstructing our systems?

Will, what do you want Ivor to do?

The machine with which I have had problems going from 17 to 18 using fedup, has just worked fine going from 17 direct to 19 using fedup. Not sure if Ivor is in a position to try that though.

I can confirm that going from 17 to 19 completes the upgrade process. It failed to reboot automatically at the end, but the last message was about reaching a system reboot target. After a manual reboot, the biggest problems were that the system failed to start X because conflicting driver versions had been installed and Apache failed to start because of configuration syntax errors due to changes in Apache. I've corrected the Apache configuration problems and seem to have at least a headless system working, which is the most important. The display issue is most likely unique to my system; the NVIDIA graphics card is no longer supported by the latest NVIDIA driver, but I don't yet know why multiple versions of the NVIDIA driver, kernel mod etc. got installed.

Just ran into the exact same issue (the device corresponding to /boot is missing), while trying (first time) fedup upgrade from 17 to 19. 
In my case the missing device referenced by the symlink in /dev/disk/by-uuid looks like this in "normal" mode:

lrwxrwxrwx 1 root root 10 Aug 12 10:23 183a3237-75f0-4d45-9a7e-47ea9bc91043 -> ../../dm-2

But while rebooting and trying the fedup upgrade, the same link points to /dev/sda2 - which is the right partition containing /boot, but the device file is missing - so the link is broken. Dropping to a rescue shell and running partprobe would make /dev/sda2 (and a couple of others) re-appear. 

Fedup version is the latest for F17 - fedup-0.7.3-5.fc17

Should I gather some more debugging info?