Bug 1084576 - Nested KVM on AMD doesn't work
Summary: Nested KVM on AMD doesn't work
Keywords:
Status: CLOSED DUPLICATE of bug 870071
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 20
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Libvirt Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-04-04 18:13 UTC by L.L.Robinson
Modified: 2014-04-17 16:17 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-04-17 16:17:36 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
output of cat /proc/cpuinfo on L- (1.78 KB, text/plain)
2014-04-04 20:01 UTC, L.L.Robinson
no flags Details
ouput of cpuid on L0 (25.50 KB, text/plain)
2014-04-04 20:01 UTC, L.L.Robinson
no flags Details
output of lsmod on L0 (2.76 KB, text/plain)
2014-04-04 20:02 UTC, L.L.Robinson
no flags Details
output of virsh dumpxml (5.95 KB, text/plain)
2014-04-04 20:02 UTC, L.L.Robinson
no flags Details
output of cpuid on L1 (27.17 KB, text/plain)
2014-04-04 20:03 UTC, L.L.Robinson
no flags Details
output of lsmod on L1 (2.99 KB, text/plain)
2014-04-04 20:03 UTC, L.L.Robinson
no flags Details
output of virsh dumpxml (5.31 KB, text/xml)
2014-04-04 20:04 UTC, L.L.Robinson
no flags Details
output of virsh capabilities on L0 (5.95 KB, text/plain)
2014-04-04 20:04 UTC, L.L.Robinson
no flags Details
output of cat /proc/cpuinfo on L1 (1.50 KB, text/plain)
2014-04-04 20:05 UTC, L.L.Robinson
no flags Details
output of virsh capabilities on L1 (3.95 KB, text/plain)
2014-04-04 20:05 UTC, L.L.Robinson
no flags Details

Description L.L.Robinson 2014-04-04 18:13:00 UTC
Description of problem: Setting up a KVM Hypervisor with virt-manager doesn't automatically create guests with nested KVM capabilities.


Version-Release number of selected component (if applicable): 
[root@haystack ~]# yum info libvirt
Loaded plugins: rhnplugin
This system is receiving updates from RHN Classic or Red Hat Satellite.
Installed Packages
Name        : libvirt
Arch        : x86_64
Version     : 1.1.3.4
Release     : 3.fc20
Size        : 0.0  
Repo        : installed
From repo   : updates


How reproducible: Always


Steps to Reproduce:
1. Install minimal install Fedora 20 on bare metal, yum install libvirt and kvm, install all updates, setup bridged networking interface, start libvirtd and connect remotely with virt-manager (I'll call this machine L0)
2.  Create virtual machine on L0, ensure Processor is set to "Copy Host Configuration" and install minimal install Fedora 20 on virtual machine, yum install libvirt and kvm, install all updates, start libvirtd and connect remotely with virt-manager (I'll call this machine L1)
3. start to create virtual machine on L1, 

Actual results:

observe message "Warning: KVM is not available. This may mean KVM package is not installed or the KVM kernel modules are not loaded. Your virtual machines may perform poorly."

Expected results:

Machine L1 to have KVM capability


Additional info:
attachements to follow

Comment 1 L.L.Robinson 2014-04-04 20:01:19 UTC
Created attachment 882838 [details]
output of cat /proc/cpuinfo on L-

Comment 2 L.L.Robinson 2014-04-04 20:01:48 UTC
Created attachment 882840 [details]
ouput of cpuid on L0

Comment 3 L.L.Robinson 2014-04-04 20:02:17 UTC
Created attachment 882841 [details]
output of lsmod on L0

Comment 4 L.L.Robinson 2014-04-04 20:02:42 UTC
Created attachment 882842 [details]
output of virsh dumpxml

Comment 5 L.L.Robinson 2014-04-04 20:03:02 UTC
Created attachment 882843 [details]
output of cpuid on L1

Comment 6 L.L.Robinson 2014-04-04 20:03:26 UTC
Created attachment 882844 [details]
output of lsmod on L1

Comment 7 L.L.Robinson 2014-04-04 20:04:15 UTC
Created attachment 882845 [details]
output of virsh dumpxml

Comment 8 L.L.Robinson 2014-04-04 20:04:38 UTC
Created attachment 882846 [details]
output of virsh capabilities on L0

Comment 9 L.L.Robinson 2014-04-04 20:05:02 UTC
Created attachment 882847 [details]
output of cat /proc/cpuinfo on L1

Comment 10 L.L.Robinson 2014-04-04 20:05:34 UTC
Created attachment 882848 [details]
output of virsh capabilities on L1

Comment 11 L.L.Robinson 2014-04-04 20:08:23 UTC
the output of cat  /sys/module/kvm_amd/parameters/nested is 1

Comment 12 Cole Robinson 2014-04-08 15:54:29 UTC
kernel version on host and guest?
can you check dmesg | grep kvm for any errors in the guest and host? might need to try and manually trigger a modprobe kvm or modprobe kvm_amd in the guest before checking dmesg

Comment 13 L.L.Robinson 2014-04-17 15:02:49 UTC
output of dmesg | grep kvm on L0

[    7.597020] kvm: Nested Virtualization enabled
[    7.597039] kvm: Nested Paging enabled

Kernel of L0
[root@haystack ~]# uname -a
Linux haystack.clarkconnect.lan 3.13.8-200.fc20.x86_64 #1 SMP Tue Apr 1 03:35:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Comment 14 L.L.Robinson 2014-04-17 15:08:08 UTC
on L1

[root@localhost ~]# dmesg |grep kvm
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, primary cpu clock
[    0.000000] kvm-stealtime: cpu 0, msr 7fc0e000
[    0.001000] kvm-clock: cpu 1, msr 0:7ffda041, secondary cpu clock
[    0.033499] kvm-stealtime: cpu 1, msr 7fd0e000
[    0.176626] Switched to clocksource kvm-clock
[    0.637837] systemd[1]: Detected virtualization 'kvm'.
[    5.968107] kvm: no hardware support


[root@localhost ~]# uname -a
Linux localhost.localdomain 3.13.8-200.fc20.x86_64 #1 SMP Tue Apr 1 03:35:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Comment 15 L.L.Robinson 2014-04-17 15:10:30 UTC
ouput of lsmod on L1

 [root@localhost ~]# lsmod |grep kvm
kvm                   444314  0 

result of modprobe kvm_amd on L1

[root@localhost ~]# modprobe kvm_amd
modprobe: ERROR: could not insert 'kvm_amd': Operation not supported

Comment 16 L.L.Robinson 2014-04-17 15:34:53 UTC
Ok I have a workaround from using information from this site here.

http://lost-and-found-narihiro.blogspot.co.uk/2013/08/kvm-nested-how-to-confirm-nested-works.html

The issue is that using VirtManager and selecting the "Copy Cpu Configuration" doesn't do what I'd expect it to do.

by editing the xml for the L1 guest and changing the cpu section to 

  <cpu mode='host-passthrough'>
 </cpu>

then the output of dmesg on L1 is this
[root@localhost ~]# dmesg |grep kvm
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, primary cpu clock
[    0.000000] kvm-stealtime: cpu 0, msr 7fc0e000
[    0.002000] kvm-clock: cpu 1, msr 0:7ffda041, secondary cpu clock
[    0.035042] kvm-stealtime: cpu 1, msr 7fd0e000
[    0.177545] Switched to clocksource kvm-clock
[    0.612321] systemd[1]: Detected virtualization 'kvm'.
[    2.685521] kvm: Nested Virtualization enabled
[    2.685529] kvm: Nested Paging enabled
[root@localhost ~]#

Comment 17 Cole Robinson 2014-04-17 16:17:36 UTC
Ah okay. This is just another manifestation of 870071, duping to that.

Basically, the 'copy host cpu' feature in virt-manager is the equivalent of mode='host-model' in libvirt, which has issues correctly modeling a CPU. In many cases things work fine (it works for nested SVM for me), but in some cases libvirt gets things quite wrong. However we can't use host-passthrough because it's not generally safe either (in your case it should be fine).

This bug sums up some of the remaining issues:

https://bugzilla.redhat.com/show_bug.cgi?id=1055002

*** This bug has been marked as a duplicate of bug 870071 ***


Note You need to log in before you can comment on or make changes to this bug.