Bug 1192030 - Workstation LiveISO refuses to boot on Hyper-V Generation 2 VM
Summary: Workstation LiveISO refuses to boot on Hyper-V Generation 2 VM
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: livecd-tools
Version: 23
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Brian Lane
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedFreezeException
Depends On:
Blocks: F23FinalFreezeException
TreeView+ depends on / blocked
 
Reported: 2015-02-12 13:02 UTC by Sitsofe Wheeler
Modified: 2017-02-27 07:57 UTC (History)
19 users (show)

Fixed In Version: livecd-tools-23.2-3.fc23
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-19 21:08:31 UTC
Type: Bug


Attachments (Terms of Use)
Generation 1 VM dmesg (42.24 KB, text/plain)
2015-10-10 05:13 UTC, Sitsofe Wheeler
no flags Details
Generation 1 filtered lsmod (282 bytes, text/plain)
2015-10-10 05:13 UTC, Sitsofe Wheeler
no flags Details

Description Sitsofe Wheeler 2015-02-12 13:02:10 UTC
Description of problem:
The Fedora 21 Workstation ISO boots but does not respond to the keyboard and bails out with an error message when run within a Generation 2 (no legacy hardware) VM.

Version-Release number of selected component (if applicable):
lorax-21.30.1.fc21

How reproducible:
Reproducible every time.

Steps to Reproduce:
1. Create a Generation 2 VM on Hyper-V 2012 R2.
2. Attach Fedora-Live-Workstation-x86_64-21-5.iso as a DVD.
3. Ensure that Secure boot has been turned off (File -> Settings | Firmware | Enable Secure Boot) to avoid hitting Bug 1097772.
4. Start the VM.

Actual results:
Boot to go through the usual plymouth splash screen and to show Anaconda.

Expected results:
Boot get stuck on plymouth splash screen.

Additional info:
Spun off from Bug 1097772. This doesn't seem to affect the Server DVD. https://bugzilla.redhat.com/show_bug.cgi?id=1097772#c7 mentions the keyboard driver isn't present and inspection shows that Fedora 21's current dracut doesn't try to force the hyperv-keyboard driver into initramfs (but upstream has a fix for this http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/modules.d/90kernel-modules/module-setup.sh?id=7a3e1a0e4a86681fab3baf9662f2d40733876ea2 ).

Comment 1 Sitsofe Wheeler 2015-06-05 14:23:33 UTC
Sadly this problem seems to have carried over into Fedora 22 even though Bug 1097772 has been fixed...

Updating Fedora version.

Comment 2 Sitsofe Wheeler 2015-06-05 14:32:26 UTC
Sorry - referenced the wrong bug - it is #1192035 which was fixed.

Comment 3 Brian Lane 2015-08-13 18:36:47 UTC
Please retry with the Fedora 23 Alpha images.

Also note that lorax is used for the boot.iso but not for the live iso's, if they don't work then a bug with spin-kickstarts or livecd-tools needs to be opened.

Comment 4 Sitsofe Wheeler 2015-08-13 20:42:23 UTC
This problem (On Hyper-V Gen2 booting fails after a few minutes, keyboard doesn't work) still exists with the current Fedora 23 Alpha Workstation ISO. Reassigning this issue to livecd-tools.

Comment 5 Sitsofe Wheeler 2015-10-09 18:37:05 UTC
This is still here on Fedora-Live-Workstation-x86_64-23_Beta-1.iso ...

Comment 6 Brian Lane 2015-10-09 20:27:32 UTC
I'm not sure what's going on here. hyperv-keyboard is present in the rootfs of the workstation. Attach the output from these commands as individual text/plain attachments:

dmesg
lsmod | grep hyper
lsmod | grep hv

Comment 7 Sitsofe Wheeler 2015-10-10 05:13:12 UTC
Created attachment 1081508 [details]
Generation 1 VM dmesg

Comment 8 Sitsofe Wheeler 2015-10-10 05:13:54 UTC
Created attachment 1081509 [details]
Generation 1 filtered lsmod

Comment 9 Sitsofe Wheeler 2015-10-10 05:31:00 UTC
The Hyper-V SCSI bus would be required to see the DVD drive so at least hv_storvsc and all its dependencies would be needed. If hyperv-keyboard is only present in the rootfs of workstation and not the initrd if something goes wrong during the initial boot it will be very hard for the user to debug (Kys: if only Generation 2 VM's supported a serial port for debugging :-).

Comment 10 Sitsofe Wheeler 2015-10-10 06:04:08 UTC
As mentioned in comment #1 the server DVD ISO boots to a menu but so does /linux/development/23/x86_64/os/images/boot.iso (which I gather is the net install ISO).

/mnt/loop2/isolinux/initrd0.img is from a mounted Fedora-Live-Workstation-x86_64-23_Beta-1.iso: 
# lsinitrd /mnt/loop2/isolinux/initrd0.img | grep 'hyper\|hv'
-rw-r--r--   1 root     root        12055 Jun 17 14:30 usr/lib/kbd/keymaps/legacy/i386/qwerty/hypermap.m4
drwxr-xr-x   2 root     root            0 Sep 15 09:59 usr/lib/modules/4.2.0-300.fc23.x86_64/kernel/drivers/hv
-rw-r--r--   1 root     root        17532 Sep  4 14:17 usr/lib/modules/4.2.0-300.fc23.x86_64/kernel/drivers/hv/hv_vmbus.ko.xz
drwxr-xr-x   2 root     root            0 Sep 15 09:59 usr/lib/modules/4.2.0-300.fc23.x86_64/kernel/drivers/net/hyperv
-rw-r--r--   1 root     root        16216 Sep  4 14:17 usr/lib/modules/4.2.0-300.fc23.x86_64/kernel/drivers/net/hyperv/hv_netvsc.ko.xz
-rw-r--r--   1 root     root         5820 Sep  4 14:17 usr/lib/modules/4.2.0-300.fc23.x86_64/kernel/drivers/video/fbdev/hyperv_fb.ko.xz

So neither hv_storvsc nor hyperv_keyboard are present.

Doing the same for a mounted boot.iso:
# lsinitrd /mnt/loop3/isolinux/initrd.img | grep 'hyper\|hv'
-rw-r--r--   1 root     root        12055 Jun 17 14:30 usr/lib/kbd/keymaps/legacy/i386/qwerty/hypermap.m4
-rw-r--r--   1 root     root         3608 Oct  5 16:36 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/hid/hid-hyperv.ko.xz
drwxr-xr-x   2 root     root            0 Oct  9 11:36 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/hv
-rw-r--r--   1 root     root        17512 Oct  5 16:38 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/hv/hv_vmbus.ko.xz
-rw-r--r--   1 root     root         3100 Oct  5 16:38 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/input/serio/hyperv-keyboard.ko.xz
drwxr-xr-x   2 root     root            0 Oct  9 11:36 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/net/hyperv
-rw-r--r--   1 root     root        16176 Oct  5 16:38 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/net/hyperv/hv_netvsc.ko.xz
-rw-r--r--   1 root     root         8208 Oct  5 16:36 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/scsi/hv_storvsc.ko.xz
-rw-r--r--   1 root     root         5824 Oct  5 16:37 usr/lib/modules/4.2.3-300.fc23.x86_64/kernel/drivers/video/fbdev/hyperv_fb.ko.xz

Comment 11 K. Y. Srinivasan 2015-10-10 22:55:39 UTC
Sitsofe,

I think there is way to enable serial logging on the gen 2 vms. I am adding vivek who might be able to help.

Comment 12 Alex Ng 2015-10-12 17:34:26 UTC
Sitsofe,

You'll find instructions for enabling serial ports on Gen 2 VMs here:
http://blogs.technet.com/b/jhoward/archive/2013/10/31/hyper-v-generation-2-virtual-machines-part-5.aspx

Essentially, you'll need to use the following PowerShell command to enable it.

For example, this command enables COM port 1 (mapped to ttyS0 on Linux guest).

Set-VMComPort -VMName "MyVM" -Path \\.\pipe\name -Number 1

Comment 13 Sitsofe Wheeler 2015-10-12 18:02:39 UTC
Kys, Alex: Fantastic - that's what I was after. Thank you!

Comment 14 Brian Lane 2015-10-12 18:07:58 UTC
For some reason dracut isn't installing all the modules when it is running from inside  livecd-creator. I ran a livecd build, watched the install_root, and livecd-creator is writing its 02livecd.conf file to /etc/dracut.conf.d/ with hostonly="no" in it as expected. There were no other config files that might have overridden it. The /usr/lib/modules/* directory did have them, so they are available, just not getting picked up. The initrd also reported that the cmdline used was just '-f' so it isn't being overridden there.

This might be related to commit 180e9d78516fb4b2ee5baef44521007a860d4dd2 which changed how modules are installed. But hyperv-keyboard is under serio and hid-hyperv is under hid so the only thing I can think of is hostonly=no is somehow being ignored.

Comment 15 Harald Hoyer 2015-10-13 12:41:45 UTC
(In reply to Brian Lane from comment #14)
> For some reason dracut isn't installing all the modules when it is running
> from inside  livecd-creator. I ran a livecd build, watched the install_root,
> and livecd-creator is writing its 02livecd.conf file to /etc/dracut.conf.d/
> with hostonly="no" in it as expected. There were no other config files that
> might have overridden it. The /usr/lib/modules/* directory did have them, so
> they are available, just not getting picked up. The initrd also reported
> that the cmdline used was just '-f' so it isn't being overridden there.
> 
> This might be related to commit 180e9d78516fb4b2ee5baef44521007a860d4dd2
> which changed how modules are installed. But hyperv-keyboard is under serio
> and hid-hyperv is under hid so the only thing I can think of is hostonly=no
> is somehow being ignored.

 can you add "--debug" to the dracut parameters and show me the stderr output?

Comment 16 Brian Lane 2015-10-13 15:30:23 UTC
Here's the log:

https://bcl.fedorapeople.org/logs/1192030-dracut.log

But what it amount to is that livecd-creator has been using drivers+= in the config, so this overrides all normal driver selection in dracut. If we want to extend the normal set of drivers we should be using add_drivers+= instead.

Comment 17 Fedora Blocker Bugs Application 2015-10-13 16:10:06 UTC
Proposed as a Freeze Exception for 23-final by Fedora user bcl using the blocker tracking app because:

 Would be nice if the Live iso's would work on HyperV

Comment 18 Brian Lane 2015-10-13 18:23:42 UTC
Patch: https://github.com/rhinstaller/livecd-tools/pull/16

Comment 19 Adam Williamson 2015-10-15 22:07:34 UTC
Discussed at 2015-10-15 freeze exception review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-10-15/f23-blocker-review.2015-10-15-20.11.log.txt . Accepted as a freeze exception issue: it would indeed be good if the live images worked on HyperV, yeah.

Comment 20 Fedora Update System 2015-10-15 23:19:06 UTC
livecd-tools-23.2-3.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-cb70de44bb

Comment 21 Sitsofe Wheeler 2015-10-16 19:26:46 UTC
I've just successfully booted Fedora-Live-Workstation-x86_64-23-20151016.iso inside a Hyper-V Generation 2 VM. Outside a kernel whinge about

PCI: Fatal: No config space access function found
ACPI: SCI (ACPI GSI 9) not registered

this ISO works fine.

Comment 22 Adam Williamson 2015-10-16 19:33:16 UTC
Thanks for testing! Could you file a +1 karma on the update?

Comment 23 Fedora Update System 2015-10-16 19:51:22 UTC
livecd-tools-23.2-3.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update livecd-tools'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-cb70de44bb

Comment 24 Sitsofe Wheeler 2015-10-16 20:52:23 UTC
awilliam (is the way your username is displayed in Bugzilla intentional):
Thank you and Brian for fixing! I actually tried 5 times to +1 the update over on bodhi but every time it would sit there for 5 minutes before turning down my request to add a comment...

Comment 25 Adam Williamson 2015-10-16 21:59:33 UTC
Sorry about that! Bodhi 2.0 teething troubles, I guess. I thought they'd fixed it, but I've seen before that when an update is 'locked' (to prevent the maintainer pushing/unpushing/whatever while a repo compose is running), you can't post comments, even though that should be allowed. Might be the same thing.

No, the way my name's displayed isn't intentional, I think it got messed up when I was fiddling with accounts (for a while I had separate RH and Fedora accounts, then I changed my mind and merged them again) and I don't know how to fix it :)

Comment 26 Fedora Update System 2015-10-19 21:08:26 UTC
livecd-tools-23.2-3.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.