Bug 1084576 - Nested KVM on AMD doesn't work
Nested KVM on AMD doesn't work
Status: CLOSED DUPLICATE of bug 870071
Product: Fedora
Classification: Fedora
Component: libvirt (Show other bugs)
20
x86_64 Linux
unspecified Severity unspecified
: ---
: ---
Assigned To: Libvirt Maintainers
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2014-04-04 14:13 EDT by L.L.Robinson
Modified: 2014-04-17 12:17 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-04-17 12:17:36 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
output of cat /proc/cpuinfo on L- (1.78 KB, text/plain)
2014-04-04 16:01 EDT, L.L.Robinson
no flags Details
ouput of cpuid on L0 (25.50 KB, text/plain)
2014-04-04 16:01 EDT, L.L.Robinson
no flags Details
output of lsmod on L0 (2.76 KB, text/plain)
2014-04-04 16:02 EDT, L.L.Robinson
no flags Details
output of virsh dumpxml (5.95 KB, text/plain)
2014-04-04 16:02 EDT, L.L.Robinson
no flags Details
output of cpuid on L1 (27.17 KB, text/plain)
2014-04-04 16:03 EDT, L.L.Robinson
no flags Details
output of lsmod on L1 (2.99 KB, text/plain)
2014-04-04 16:03 EDT, L.L.Robinson
no flags Details
output of virsh dumpxml (5.31 KB, text/xml)
2014-04-04 16:04 EDT, L.L.Robinson
no flags Details
output of virsh capabilities on L0 (5.95 KB, text/plain)
2014-04-04 16:04 EDT, L.L.Robinson
no flags Details
output of cat /proc/cpuinfo on L1 (1.50 KB, text/plain)
2014-04-04 16:05 EDT, L.L.Robinson
no flags Details
output of virsh capabilities on L1 (3.95 KB, text/plain)
2014-04-04 16:05 EDT, L.L.Robinson
no flags Details

  None (edit)
Description L.L.Robinson 2014-04-04 14:13:00 EDT
Description of problem: Setting up a KVM Hypervisor with virt-manager doesn't automatically create guests with nested KVM capabilities.


Version-Release number of selected component (if applicable): 
[root@haystack ~]# yum info libvirt
Loaded plugins: rhnplugin
This system is receiving updates from RHN Classic or Red Hat Satellite.
Installed Packages
Name        : libvirt
Arch        : x86_64
Version     : 1.1.3.4
Release     : 3.fc20
Size        : 0.0  
Repo        : installed
From repo   : updates


How reproducible: Always


Steps to Reproduce:
1. Install minimal install Fedora 20 on bare metal, yum install libvirt and kvm, install all updates, setup bridged networking interface, start libvirtd and connect remotely with virt-manager (I'll call this machine L0)
2.  Create virtual machine on L0, ensure Processor is set to "Copy Host Configuration" and install minimal install Fedora 20 on virtual machine, yum install libvirt and kvm, install all updates, start libvirtd and connect remotely with virt-manager (I'll call this machine L1)
3. start to create virtual machine on L1, 

Actual results:

observe message "Warning: KVM is not available. This may mean KVM package is not installed or the KVM kernel modules are not loaded. Your virtual machines may perform poorly."

Expected results:

Machine L1 to have KVM capability


Additional info:
attachements to follow
Comment 1 L.L.Robinson 2014-04-04 16:01:19 EDT
Created attachment 882838 [details]
output of cat /proc/cpuinfo on L-
Comment 2 L.L.Robinson 2014-04-04 16:01:48 EDT
Created attachment 882840 [details]
ouput of cpuid on L0
Comment 3 L.L.Robinson 2014-04-04 16:02:17 EDT
Created attachment 882841 [details]
output of lsmod on L0
Comment 4 L.L.Robinson 2014-04-04 16:02:42 EDT
Created attachment 882842 [details]
output of virsh dumpxml
Comment 5 L.L.Robinson 2014-04-04 16:03:02 EDT
Created attachment 882843 [details]
output of cpuid on L1
Comment 6 L.L.Robinson 2014-04-04 16:03:26 EDT
Created attachment 882844 [details]
output of lsmod on L1
Comment 7 L.L.Robinson 2014-04-04 16:04:15 EDT
Created attachment 882845 [details]
output of virsh dumpxml
Comment 8 L.L.Robinson 2014-04-04 16:04:38 EDT
Created attachment 882846 [details]
output of virsh capabilities on L0
Comment 9 L.L.Robinson 2014-04-04 16:05:02 EDT
Created attachment 882847 [details]
output of cat /proc/cpuinfo on L1
Comment 10 L.L.Robinson 2014-04-04 16:05:34 EDT
Created attachment 882848 [details]
output of virsh capabilities on L1
Comment 11 L.L.Robinson 2014-04-04 16:08:23 EDT
the output of cat  /sys/module/kvm_amd/parameters/nested is 1
Comment 12 Cole Robinson 2014-04-08 11:54:29 EDT
kernel version on host and guest?
can you check dmesg | grep kvm for any errors in the guest and host? might need to try and manually trigger a modprobe kvm or modprobe kvm_amd in the guest before checking dmesg
Comment 13 L.L.Robinson 2014-04-17 11:02:49 EDT
output of dmesg | grep kvm on L0

[    7.597020] kvm: Nested Virtualization enabled
[    7.597039] kvm: Nested Paging enabled

Kernel of L0
[root@haystack ~]# uname -a
Linux haystack.clarkconnect.lan 3.13.8-200.fc20.x86_64 #1 SMP Tue Apr 1 03:35:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Comment 14 L.L.Robinson 2014-04-17 11:08:08 EDT
on L1

[root@localhost ~]# dmesg |grep kvm
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, primary cpu clock
[    0.000000] kvm-stealtime: cpu 0, msr 7fc0e000
[    0.001000] kvm-clock: cpu 1, msr 0:7ffda041, secondary cpu clock
[    0.033499] kvm-stealtime: cpu 1, msr 7fd0e000
[    0.176626] Switched to clocksource kvm-clock
[    0.637837] systemd[1]: Detected virtualization 'kvm'.
[    5.968107] kvm: no hardware support


[root@localhost ~]# uname -a
Linux localhost.localdomain 3.13.8-200.fc20.x86_64 #1 SMP Tue Apr 1 03:35:46 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Comment 15 L.L.Robinson 2014-04-17 11:10:30 EDT
ouput of lsmod on L1

 [root@localhost ~]# lsmod |grep kvm
kvm                   444314  0 

result of modprobe kvm_amd on L1

[root@localhost ~]# modprobe kvm_amd
modprobe: ERROR: could not insert 'kvm_amd': Operation not supported
Comment 16 L.L.Robinson 2014-04-17 11:34:53 EDT
Ok I have a workaround from using information from this site here.

http://lost-and-found-narihiro.blogspot.co.uk/2013/08/kvm-nested-how-to-confirm-nested-works.html

The issue is that using VirtManager and selecting the "Copy Cpu Configuration" doesn't do what I'd expect it to do.

by editing the xml for the L1 guest and changing the cpu section to 

  <cpu mode='host-passthrough'>
 </cpu>

then the output of dmesg on L1 is this
[root@localhost ~]# dmesg |grep kvm
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, boot clock
[    0.000000] kvm-clock: cpu 0, msr 0:7ffda001, primary cpu clock
[    0.000000] kvm-stealtime: cpu 0, msr 7fc0e000
[    0.002000] kvm-clock: cpu 1, msr 0:7ffda041, secondary cpu clock
[    0.035042] kvm-stealtime: cpu 1, msr 7fd0e000
[    0.177545] Switched to clocksource kvm-clock
[    0.612321] systemd[1]: Detected virtualization 'kvm'.
[    2.685521] kvm: Nested Virtualization enabled
[    2.685529] kvm: Nested Paging enabled
[root@localhost ~]#
Comment 17 Cole Robinson 2014-04-17 12:17:36 EDT
Ah okay. This is just another manifestation of 870071, duping to that.

Basically, the 'copy host cpu' feature in virt-manager is the equivalent of mode='host-model' in libvirt, which has issues correctly modeling a CPU. In many cases things work fine (it works for nested SVM for me), but in some cases libvirt gets things quite wrong. However we can't use host-passthrough because it's not generally safe either (in your case it should be fine).

This bug sums up some of the remaining issues:

https://bugzilla.redhat.com/show_bug.cgi?id=1055002

*** This bug has been marked as a duplicate of bug 870071 ***

Note You need to log in before you can comment on or make changes to this bug.