Created attachment 958690 [details] Hacky workaround from Alexander Graf Description of problem: There was a kernel change [1] which ended in disabling virtualization on APM Mustang when kernel with 64KB pages is used (which is Fedora default). Hacky workaround was created by Alexander Graf from SUSE (attached) 1. https://git.kernel.org/cgit/linux/kernel/git/kvmarm/kvmarm.git/commit/virt/kvm/arm/vgic.c?id=63afbe7a0ac184ef8485dac4914e87b211b5bfaa Version-Release number of selected component (if applicable): 3.18-rc How reproducible: always Steps to Reproduce: 1. boot kernel on Mustang Actual results: [ 0.343796] kvm [1]: GICV size 0x2000 not a multiple of page size 0x10000 [ 0.343802] kvm [1]: error: no compatible GIC info found [ 0.343909] kvm [1]: error initializing Hyp mode: -6 Expected results: [ 0.343799] kvm [1]: interrupt-controller@780c0000 IRQ25 [ 0.343903] kvm [1]: timer IRQ30 [ 0.343912] kvm [1]: Hyp mode initialized successfully Additional info:
IRC log from #kvm-arm from 17 November 2014 (CET timestamps) 09:29 < hrw> hi 09:30 < hrw> can someone boot 64KB pages kernel on Juno or Mustang and get kvm? 09:30 < hrw> [ 0.343796] kvm [1]: GICV size 0x2000 not a multiple of page size 0x10000 09:30 < hrw> [ 0.343802] kvm [1]: error: no compatible GIC info found 09:30 < hrw> [ 0.343909] kvm [1]: error initializing Hyp mode: -6 09:30 < hrw> is what I got on 3.18-rc4 64K from fedora/rawhide 09:32 < hrw> bbiab 10:27 < suihkulokki> hrw: using 3.18-rc5 and mustang I get the same error with 64KB pages - 4KB pages work ok 10:31 < chazy> hrw: suihkulokki: see this commit: https://git.kernel.org/cgit/linux/kernel/git/kvmarm/kvmarm.git/commit/virt/kvm/arm/vgic.c?id=63afbe7a0ac184ef8485dac4914e87b211b5bfaa 10:32 < chazy> iirc, you can fix the DT to report things properly so that this check will pass and everything works (at least on Seattle), but for APM mustang I did think carefully about the proper fix. 10:33 < chazy> if you want to hack on it, and don’t care about stability etc., you can simply remove those two PAGE_ALIGNED checks (now moved to vgic-v2.c) and I believe things will work, but beware, you’re venturing into danger land. 10:37 < suihkulokki> chazy: I think hrw and fedora would be more interested in fixing the dtb 10:39 < chazy> suihkulokki: I realize that, but I think they would have to work with APM on that one. Not sure what the long term plan for juno was on this. 12:41 < agraf> chazy: I have a local patch that relaxes this check slightly 12:42 < agraf> chazy: http://paste.debian.net/132128/ 12:42 < agraf> chazy: need to make this a kernel module option that defaults to off 12:42 < agraf> chazy: but you get the idea ;) 12:43 < agraf> chazy: the problem is that on juno qemu needs to align the vgic at the same offset as real hardware has it 12:43 < hrw> chazy: seattle has gicv3 iirc 12:44 < hrw> suihkulokki: at RH we tend to switch for acpi asap. now probably only pci is not fully tested 12:44 < agraf> hrw: nope, it’s v2+msi 12:45 < hrw> ah, right. Thunder is v3 12:45 < agraf> yup, thunder is the first v3 hw you can get fwiw 12:48 < hrw> chazy: I can also go to 3.17 kernel and not have that check ;D
3.18.0-0.rc6.git0.1.fc22.aarch64 is affected too
FYI, we think this will be resolved by a firmware update. We are working with APM to achieve this.
Fresh install of Fedora 21 also lacks nationalization support: [root@pinkiepie virt]# virsh create hrw-f21.xml błąd: Utworzenie domeny z hrw-f21.xml nie powiodło się błąd: unsupported configuration: Domain requires KVM, but it is not available. Check that virtualisation is enabled in the host BIOS, and host configuration is setup to load the kvm modules. [root@pinkiepie virt]# dmesg|grep -i kvm [ 0.303959] kvm [1]: GICV size 0x2000 not a multiple of page size 0x10000 [ 0.303965] kvm [1]: error: no compatible GIC info found [ 0.304070] kvm [1]: error initializing Hyp mode: -6 [root@pinkiepie virt]# uname -a Linux pinkiepie 3.17.4-301.fc21.aarch64 #1 SMP Sun Nov 30 20:41:43 UTC 2014 aarch64 aarch64 aarch64 GNU/Linux
This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle. Changing version to '22'. More information and reason for this action is here: https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22
Created attachment 1009121 [details] DeviceTreeBlob with fixed gicv entry Grab fixed DTB, put it in /boot/ directory and add one line into GRUB2 config: devicetree /boot/mustang.dtb Then kernel will boot and will have KVM support (tested with 4.0-rc5).
Can confirm that KVM is working again with that DTB file on kernel 4.0.0-0.rc6.git0.1.fc23.aarch64
Still broken in Rawhide (without workarounds).
Richard: issue is in firmware not in kernel. As long as we do not have fixed firmware available bug will happen.
So .. Anyone give me a clue exactly how to implement the workaround? I dropped the mustang.dtb file into /boot and added: devicetree /boot/mustang.dtb to the end of /boot/grub2/grub.cfg, and rebooted, but I still see the same error: $ dmesg | grep -i kvm [ 0.725483] kvm [1]: GICV size 0x2000 not a multiple of page size 0x10000 [ 0.725576] kvm [1]: error initializing Hyp mode: -6
I also tried appending the devicetree line to /boot/efi/EFI/fedora/grub.cfg but that also doesn't appear to make any difference.
The answer is you have to edit /boot/efi/EFI/fedora/grub.cfg, find the latest kernel section in that file, and add the devicetree line to that kernel section. You end up with: ... linux /vmlinuz-4.0.0-0.rc6.git2.1.fc23.aarch64 root=UUID=9d1bb62f-5d50-458e-be91-36af494ccfa0 ro console=ttyS0,115200 earlyprintk=uart8250-32bit,0x1c020000 LANG=en_GB.UTF-8 initrd /initramfs-4.0.0-0.rc6.git2.1.fc23.aarch64.img devicetree /mustang.dtb } This fixed KVM for me. However it also broke stable network addressing. It seems that the first NIC now gets a randomized MAC address each time.
Have to check and reorder Ethernet devices.
Created attachment 1012161 [details] DeviceTreeBlob with fixed gicv entry and 1GbE as eth0 This DTB has 1GbE as eth0 (was eth2 in previous DTB), working USB and KVM: 13:46 hrw@pinkiepie-rawhide:~$ dmesg|grep kvm -i [ 0.871608] kvm [1]: interrupt-controller@780c0000 IRQ5 [ 0.877167] kvm [1]: timer IRQ3 [ 0.880426] kvm [1]: Hyp mode initialized successfully 13:47 hrw@pinkiepie-rawhide:~$ lsusb Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 003 Device 002: ID 125f:de7a A-DATA Technology Co., Ltd. Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 1234:2088 Brain Actuated Technologies Bus 001 Device 002: ID 05e3:0606 Genesys Logic, Inc. USB 2.0 Hub / D-Link DUB-H4 USB 2.0 Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub 13:47 hrw@pinkiepie-rawhide:~$ ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.200.76 netmask 255.255.255.0 broadcast 192.168.200.255 inet6 fdd2:2a70:985d::c0a prefixlen 128 scopeid 0x0<global> inet6 fdd2:2a70:985d:0:201:73ff:fe02:b73 prefixlen 64 scopeid 0x0<global> inet6 fe80::201:73ff:fe02:b73 prefixlen 64 scopeid 0x20<link> ether 00:01:73:02:0b:73 txqueuelen 1000 (Ethernet) RX packets 223 bytes 25152 (24.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 198 bytes 47959 (46.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
*** Bug 1208768 has been marked as a duplicate of this bug. ***
Version 1.1.0 firmware fixed this issue on Mustang. KVM is properly initialised at boot time.
I can confirm also that 1.1.0 firmware fixes the problem, and has a stable MAC address. I suggest we can just close this bug unless anyone objects?
As bug was not in a kernel but in DTB provided by firmware and latest official firmware (available to all users of affected platform) fixed problem I am closing this bug.
*** Bug 1263499 has been marked as a duplicate of this bug. ***
Changing the page size from 64KB (Fedora's default) to 4KB seems to fix this issue (firmware version 1.1.0-rh-0.15), hoping new firmware .20 from APM handles the 64KB kernel as well.
Itaru: 4K kernels are something which is not supported. You should/have to upgrade your firmware.