+++ This bug was initially created as a clone of Bug #896010 +++ Description of problem: This was reported by 'nyloc' on #fedora-qa, I reproduced it. I installed clean Fedora 17 on this disk layout: vda1: /boot vda2: / (encrypted) vda3: swap (encrypted) vdb1: /data (encrypted) All standard partitions, no LVM. The password is the same for both systems. After I finished running fedup-cli and rebooted, the system asks for disk password and then hangs. nyloc reported he received a maintenance shell after a timeout, I didn't, but that probably doesn't matter much. This is the System Upgrade boot line from grub: linux /vmlinuz-fedup root=/dev/mapper/luks-f1cab7e0-fae8-4a00-94b9-c5d3976d6031 ro rd.luks.uuid=luks-80aa76ad-d5a3-4bfa-b8cd-7f70fe8379a0 rd.lvm=0 rd.dm=0 SYSFONT=True rd.luks.uuid=luks-f1cab7e0-fae8-4a00-94b9-c5d3976d6031 KEYTABLE=us rd.md=0 LANG=en_US.UTF-8 rhgb quiet upgrade systemd.unit=system-upgrade.target plymouth.splash=fedup It looks fine. This is the System Upgrade boot output: > dracut-cmdline[50]: Warning: Kernel command line option 'KEYTABLE' is deprecated, use 'vconsole.keymap' instead. > dracut-cmdline[50]: Warning: Kernel command line option 'SYSFONT' is deprecated, use 'vconsole.font' instead. > [ 3.114503] systemd-cryptsetup-generator[265]: Failed to create unit file: File exists > [ OK ] Found device /dev/disk/by-uuid/80aa76ad-d5a3-4bfa-b8cd-7f70fe8379a0. > Starting Cryptography Setup for luks-80aa76ad-d5a3-4...7f70fe8379a0... > [ OK ] Found device /dev/disk/by-uuid/f1cab7e0-fae8-4a00-94b9-c5d3976d6031. > Starting Cryptography Setup for luks-f1cab7e0-fae8-4...c5d3976d6031... > Expecting device dev-mapper-luks\x2d80aa76ad\x2dd5a3...379a0.device... > Expecting device dev-mapper-luks\x2df1cab7e0\x2dfae8...d6031.device... > [ OK ] Started Show Plymouth Boot Screen. > Starting Forward Password Requests to Plymouth... > [ OK ] Started Forward Password Requests to Plymouth. > > Please enter passphrase for disk luks-f1cab7e0-fae8-4a00-94b9-c5d3976d6031!:********** > [ OK ] Started Cryptography Setup for luks-f1cab7e0-fae8-4a...9-c5d3976d6031. > [ OK ] Found device /dev/mapper/luks-f1cab7e0-fae8-4a00-94b9-c5d3976d6031. > [ OK ] Started Cryptography Setup for luks-80aa76ad-d5a3-4b...d-7f70fe8379a0. > [ OK ] Reached target Local File Systems. > [ OK ] Reached target Encrypted Volumes. > [ OK ] Reached target System Initialization. > [ OK ] Reached target Basic System. <------ HANGS HERE Boot with rd.debug is attached. System reboots if Ctrl+Alt+Del is hit, so it's not dead. Version-Release number of selected component (if applicable): fedup-dracut of unknown version taken from development/18 tree. (Hey Will, why it is not mentioned anywhere, like in --debuglog?) How reproducible: with nyloc and my confirmation, seems like every time Steps to Reproduce: 1. follow description --- Additional comment from Kamil Páral on 2013-01-16 14:15:59 CET --- Created attachment 679569 [details] fedup.log --- Additional comment from Kamil Páral on 2013-01-16 14:16:03 CET --- Created attachment 679570 [details] grub.cfg --- Additional comment from Kamil Páral on 2013-01-16 14:16:09 CET --- Created attachment 679571 [details] boot.debug.log --- Additional comment from Kamil Páral on 2013-01-18 10:55:21 CET --- There are probably several issues combined in this report. Adding selinux=0 didn't help me (nothing changed) with the original problem described in comment 0. --- Additional comment from Kamil Páral on 2013-02-01 10:04:31 CET --- fedup-0.7.3-0.git20130128.fc17 doesn't fix the problem mentioned in comment 0. --- Additional comment from Will Woods on 2013-02-11 20:44:25 CET --- If adding 'enforcing=0' and removing 'rhgb' doesn't fix your problem, please file a new bug so it doesn't get lost!
The dracut log isn't going to be helpful here; the problem happens when we're back in the old (F17) system and trying to mount disks. You could add 'rd.upgrade.debugshell' to the boot args and use the shell on VT2 to figure out what's going on with the stuck system. I'm guessing it's waiting for the LUKS partition to be unlocked; since that's stuck, you could just wait a few minutes and you should get an emergency shell from systemd. Either way, can you get the syslog/journalctl output from the system when it's stuck and attach that?
> I'm guessing it's > waiting for the LUKS partition to be unlocked; since that's stuck, you could > just wait a few minutes and you should get an emergency shell from systemd. No, there is no timeout happening, no emergency shell. The system is waiting for something, but it is not completely stuck, Ctrl+Alt+Del works. I used rd.upgrade.debugshell, but I don't how to access any logs. There are no commands available, 'ls' is not there, 'less' is not there, nothing. 'help' prints out some internal bash commands, but those doesn't help get the job done. Can you please try to reproduce the problem yourself? It should be 100% reproducible using the same disk layout. Or I can upload the whole VM image somewhere, if you prefer.
(In reply to comment #2) > I used rd.upgrade.debugshell, but I don't how to access any logs. There are > no commands available, 'ls' is not there, 'less' is not there, nothing. > 'help' prints out some internal bash commands, but those doesn't help get > the job done. The shell is in the wrong root. Exit and it will restart in your current root.
I set up a VM with the following partitions: /dev/vda1: /boot /dev/vda2: swap (encrypted) /dev/vda3: / (encrypted) /dev/vdb1: /home (encrypted) I installed fedup-0.7.3-0.git20130128.fc17.noarch and ran it. It completed successfully. The 'fedup' boot entry had "enforcing=0" in it already; I removed "rhgb" *and* "plymouth.splash=fedup". The upgrade is now at 56% complete, so I consider it working. Can you try removing "rhgb" *and* "plymouth.splash=fedup" and see if that works? If so, this is very likely a dupe of bug 894242.
When I remove rhgb and plymouth.splash=fedup there is no change, still stuck. If I add rd.upgrade.debugshell and exit the shell once, I really appear in a real filesystem and can use real system commands, as you said. The filesystem is read only and the last line in /var/log/messages is from Feb 1st. That is probably the last day I ran the original F17 system. Also judging from its contents (and contents of other log files), there are not actual, but really from Feb 1st. I also found /system-upgrade-root/var/log, but that doesn't contain any files. journalctl says "Failed to iterate through journal: No such file or directory". Looking as lsblk output, vda2 and vda3 was decrypted, but vdb1 was not. An interesting thing is that if I try to start sshd, a prompt is printed out for password for vdb1 (but I can't input anything) and after a second vdb1 gets decrypted. See screenshot. Once that happens, two additional lines are printed on tty1 from systemd-fsck about vda1 and vdb1 being clean. I think you should have mirrored my disk layout precisely. Ping me on IRC to help me debug it.
Created attachment 698821 [details] tty1 "stuck"
Created attachment 698822 [details] lsblk output before and after sshd start
Will, I tried completely the same disk layout as you have in comment 4, the same fedup version, I removed the same grub options, and it is stuck! Now tell me, what the hell is wrong with me?! :-) I was working with F17 x86_64 Desktop Live install, no updates. Maybe you were working with a different architecture, or minimal install? We have to find the difference.
Your system isn't updated? That's very likely the problem. Try updating systemd; if that doesn't work, try updating selinux-policy; if neither of those works, try a full update. If updating makes the upgrade work, we should file that as a new bug. If that turns out to be the case, could you note the version(s) of each that don't work? I should probably make fedup request those package versions.
Great call, Will. Systemd update fixed the problem. Systemd before: systemd-44-8.fc17.x86_64 Systemd after: systemd-44-23.fc17.x86_64 I wonder, wouldn't it be better if fedup insisted that there must be no system updates available before it sets up the system for upgrade? This time it was systemd, next time it might be something else.
fedup-0.7.3-1.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/fedup-0.7.3-1.fc17
Package fedup-0.7.3-1.fc17: * should fix your issue, * was pushed to the Fedora 17 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing fedup-0.7.3-1.fc17' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2013-3956/fedup-0.7.3-1.fc17 then log in and leave karma (feedback).
fedup-0.7.3-3.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/fedup-0.7.3-3.fc18
fedup-0.7.3-3.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/fedup-0.7.3-3.fc17
fedup-0.7.3-4.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/fedup-0.7.3-4.fc17
fedup-0.7.3-4.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/fedup-0.7.3-4.fc18
fedup-0.7.3-4.fc19 has been submitted as an update for Fedora 19. https://admin.fedoraproject.org/updates/fedup-0.7.3-4.fc19
fedup-0.7.3-4.fc18 has been pushed to the Fedora 18 stable repository. If problems still persist, please make note of it in this bug report.
Unfortunately, fedup 0.7.3-4 didn't work for me with a fully updated system. Wouldn't boot into the anaconda installer. $ rpm -q fedup systemd fedup-0.7.3-4.fc17.noarch systemd-44-24.fc17.x86_64 systemd-44-24.fc17.i686 $ sudo blkid /dev/sda2: UUID="d8227018-346d-47b5-a74e-4eebbab677a1" TYPE="ext4" /dev/sda3: UUID="59778677-f7e9-4d4d-9bbd-9289542a67bc" TYPE="crypto_LUKS" /dev/sda4: UUID="d6f87459-69cf-4646-9a95-bf82d074ad3f" TYPE="crypto_LUKS" /dev/sda5: UUID="6787bfb0-f594-4270-a539-0af35470c8bc" TYPE="crypto_LUKS" /dev/mapper/luks-59778677-f7e9-4d4d-9bbd-9289542a67bc: UUID="ead53e81-5780-46c6-ae26-079e181977ed" TYPE="ext4" /dev/mapper/luks-d6f87459-69cf-4646-9a95-bf82d074ad3f: UUID="be7adbf3-b64f-43af-ba11-ebe211ee9ca8" TYPE="ext4" /dev/mapper/luks-6787bfb0-f594-4270-a539-0af35470c8bc: UUID="35c3d5f9-5651-4371-8527-541d3d668486" TYPE="swap" I'm sorry this is thin on the info, but I'd be happy to provide more upon request.
It's not supposed to boot into anaconda. It boots into the upgrade tool, which basically just shows a progress meter (or log text) while it upgrades your system.
Please excuse my ignorance with fedup upgrades, I usually do clean installs. It doesn't start the upgrade tool. Booting without rhgb, after I enter my passphrase, get the Welcome to Fedora 17, starting services, noted after Mounting Media Directory: Expecting device dev-disk-by\x2duuid-d8227018\x2d346d\x2d47b5\x2da74e\x2d4eebbab677a1.device... Expecting device dev-disk-by\x2duuid-d6f87459\x2d69cf\x2d4646\x2d9a95\x2dbf82d074ad3f.device... Expecting device dev-mapper-luks\x2dd6f87459\x2d69cf\x2d4646\x2d9a95\x2dbf82d074ad3f.device... Long pause (~1min) after the first line below, then [transcribing from photo] Started Initialize storage subsystems (RAID, LVM, etc.) Timed out waiting for device dev-mapper-luks\0x2dd6f87459\x2d4646\x2d9a95\x2dbf82d074ad3f.device. Dependency failed for Cryptography Setup for luks-d6f87459-69cf-4646-9a95-bf82d074ad3f Dependency failed for Encrypted Volumes. Dependency failed for /home. Dependency failed for Local File Systems. Welcome to emergency mode. Use "systemctl default" or ^D to enter default mode. Re-open or start new bug?
Not sure if I should reopen this BZ, but latest fedup shows the same issue as previous releases: On a fully updated F17 the luks prompt is never shown and after a (long) wait, the emergency prompt is shown. Should I reopen this BZ? - Gilboa
fedup-0.7.3-5.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/fedup-0.7.3-5.fc17
fedup-0.7.3-4.fc19 has been pushed to the Fedora 19 stable repository. If problems still persist, please make note of it in this bug report.
fedup-0.7.3-5.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.