Description of problem: I get this message when trying to start a guest: Error starting domain: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 (label charserial0) Could not access KVM kernel module: Permission denied failed to initialize KVM: Permission denied Details: Error starting domain: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 (label charserial0) Could not access KVM kernel module: Permission denied failed to initialize KVM: Permission denied Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in cb_wrapper callback(asyncjob, *args, **kwargs) File "/usr/share/virt-manager/virtManager/asyncjob.py", line 117, in tmpcb callback(*args, **kwargs) File "/usr/share/virt-manager/virtManager/domain.py", line 1160, in startup self._backend.create() File "/usr/lib64/python2.7/site-packages/libvirt.py", line 692, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 (label charserial0) Could not access KVM kernel module: Permission denied failed to initialize KVM: Permission denied Version-Release number of selected component (if applicable): libvirt-daemon-1.0.4-1.fc19.x86_64 qemu-kvm-1.4.0-11.fc19.x86_64 kernel-3.9.0-0.rc5.git1.301.fc19.x86_64 How reproducible: Happens every time.
Does selinux=permissive make any difference? What's ls -l /dev/kvm ?
I haven't seen this happen since I reported it.
Please re-open. I'm seeing this on F19 as of today (bringing up virtualization after re-install) % ls -l /dev/kvm crw-rw-rw-+ 1 root kvm 10, 232 Jul 2 12:41 /dev/kvm I'll follow up on the issue...
The SElinux trick does not change the outcome. % setenforce 0 % getenforce Permissive But issue remains.
FWIW, the following did the trick so far: # rmmod kvm_intel # rmmod kvm # modprobe kvm # modprobe kvm_intel
What is ls -l /dev/kvm for the working setup?
Sadly: % ls -l /dev/kvm crw-rw-rw-+ 1 root kvm 10, 232 Jul 2 15:05 /dev/kvm
Does this reproduce on reboot or anything? if not we are kind of stuck for finding the culprit
Does this happen only the first time after installing virt packages maybe?
Exact, this only seems to happen the first time after installing the @virtualization group of packages. Subsequent boots look fine.
Yeah the confusing thing though is that I've tried to reproduce before and never have.
After I have installed virt-manager, libvirt, qemu-kvm, I have also seen this issue. Workaround from comment #5 made the trick. Now I have: # stat /dev/kvm File: ‘/dev/kvm’ Size: 0 Blocks: 0 IO Block: 4096 character special file Device: 5h/5d Inode: 204923 Links: 1 Device type: a,e8 Access: (0666/crw-rw-rw-) Uid: ( 0/ root) Gid: ( 36/ kvm) Context: system_u:object_r:kvm_device_t:s0 Access: 2013-08-19 10:01:14.496898749 +0200 Modify: 2013-08-19 10:01:14.496898749 +0200 Change: 2013-08-19 10:01:14.497898746 +0200 Birth: - Unfortunately I do not have this before reloading these KVM related modules.
Just in case someone is looking for more data, this looks like a dup of Bug #911640, should be ok with 3.9 and 3.10 kernels.
For what it's worth, I just had the same problem on Fedora 20 (3.13.7-200.fc20.x86_64) after the initial installation. rmmod & insmod got it running.
I can reproduce this on a fresh install of Fedora 20. excerpt of: cat /proc/cpuinfo processor : 23 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU E5645 @ 2.40GHz stepping : 2 microcode : 0x15 cpu MHz : 1596.000 cache size : 12288 KB physical id : 1 siblings : 12 core id : 10 cpu cores : 6 apicid : 53 initial apicid : 53 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm arat epb dtherm tpr_shadow vnmi flexpriority ept vpid bogomips : 4799.88 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: uname -a Linux <fqdn> 3.14.2-200.fc20.x86_64 #1 SMP Mon Apr 28 14:40:57 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Let me know if you need other info. I'm reopening for now unless anyone has objections. HTH
Moving to qemu per info in Bug 958860 (that one is qemu-kvm).
Please provide: ls -l /dev/kvm getfacl /dev/kvm dmesg | grep -i kvm Does a reboot fix it?
(In reply to Cole Robinson from comment #17) > Please provide: > > ls -l /dev/kvm > getfacl /dev/kvm > dmesg | grep -i kvm Don't have these because the machine is down right now. Can run them later this week. Do you still want these on the machine now that it's "fixed" (by the rmmod/modprobe) ? > > Does a reboot fix it? It was fixed by: # rmmod kvm_intel # rmmod kvm # modprobe kvm # modprobe kvm_intel (from comment 5)
(In reply to James from comment #18) > (In reply to Cole Robinson from comment #17) > > Please provide: > > > > ls -l /dev/kvm > > getfacl /dev/kvm > > dmesg | grep -i kvm > > Don't have these because the machine is down right now. Can run them later > this week. Do you still want these on the machine now that it's "fixed" (by > the rmmod/modprobe) ? Nope, don't bother. But another question: can you describe how you installed the system? Were virt packages installed during OS installation, or yum install'ed afterwards? Was the updates repo enabled during anaconda installation?
Sorry, setting correct needinfo. James, please see comment #19
From Bug 958860 comment #11 from Miroslav: This is caused by incorrect rules file location, udev rules has to be in /usr/lib/udev not in /usr/lib64/udev I though the patch already exists since it is fixed in Bug 958860, but not back-ported to Fedora, that's why I re-assigned it with that info instead of marking it as a duplicate. Shouldn't that be enough? If not then I suggest asking Miroslav for further info.
That equivalent fix is already in fedora. But it is in fact not enough: /dev/kvm is created by udev with root:root permissions, even before kvm is installed. When kvm is installed and the relaxed permissions are added by the udev rule, they aren't automatically triggered just on rules installation, so the spec file needs to invoke the proper udevadm trigger command. But apparently that _still_ isn't enough, since it doesn't reset /dev/kvm ACLs or something, I forget the details. However that should also be fixed in Fedora. If there is still an issue, it's something new or undiscovered
(In reply to Cole Robinson from comment #22) Oh, thanks for the explanation and sorry for my misunderstanding.
(In reply to Cole Robinson from comment #19) > (In reply to James from comment #18) > > (In reply to Cole Robinson from comment #17) > > > Please provide: > > > > > > ls -l /dev/kvm > > > getfacl /dev/kvm > > > dmesg | grep -i kvm > > > > Don't have these because the machine is down right now. Can run them later > > this week. Do you still want these on the machine now that it's "fixed" (by > > the rmmod/modprobe) ? > > Nope, don't bother. > > But another question: can you describe how you installed the system? Were > virt packages installed during OS installation, or yum install'ed > afterwards? Was the updates repo enabled during anaconda installation? Virt packages were installed after os installation with yum. I don't know about what happened with anaconda. Machine was built via satellite, but not my satellite.
Well I've tried again and I can't reproduce, these issues are finicky and really the only thing I can think of is to poke at a machine while the issue is present. So just closing this as INSUFFICIENT_DATA. If someone reproduces later please reopen and provide explicit steps for how the machine was provisioned.
Yep it is still bugging me on f20: Could not access KVM kernel module: No such file or directory failed to initialize KVM: No such file or directory LAP-02-1755_16:03:14_Thu Oct 23:~ # lsmod | grep kvm kvm 452677 0 :/usr/lib/udev/rules.d # cat 80-kvm.rules KERNEL=="kvm", GROUP="kvm", MODE="0666" O heck: 16:04:52_Thu Oct 23:~ # ls -al /dev/kvm ls: cannot access /dev/kvm: No such file or directory do I have to run mknod by hand??
Is modprobe kvm_intel or modprobe kvm_amd working for you ? Do you get messages in dmesg after running these commands? Virtualization extensions (needed by kvm) tend to be disabled in the BIOS, so you need to enable them in the BIOS firrst.
LAP-02-1755_16:26:20_Thu Oct 23:~> sudo modprobe kvm_intel modprobe: ERROR: could not insert 'kvm_intel': Operation not supported bchowdhury@LAP-02-1755_16:26:30_Thu Oct 23:~> sudo modprobe kvm_amd modprobe: ERROR: could not insert 'kvm_amd': Operation not supported Heck! [21203.361826] kvm: disabled by bios [21223.198821] kvm: disabled by bios [21524.429381] kvm: disabled by bios [23848.046950] kvm: disabled by bios @ Christophe Fergeau my mistake..I should have been more vigilent...I shall be careful next time steal your time.. Thanks a bunch......
Still an issue... In this case, F21. Install Fedora 21. Used if for months... sudo yum install @virtualization sudo systemctl start libvirtd sudo systemctl status libvirtd (everything *appeared* to work fine) sudo virt-manager New instance... And then I get that error described in the initial description. Rebooted. All works. Something is up with the kvm module load. Have a nice day. :)
Several comments have been asking for additional information, in particular https://bugzilla.redhat.com/show_bug.cgi?id=950436#c17
I just ran into this on CentOS 7 ... reloading the kernel modules fixed the permissions for me. This was after a clean install of CentOS and then doing: # yum install libvirt qemu-kvm qemu-img virt-install bridge-utils Got the same permission denied errors as others and then found this bug. Here are the command output requested before: [root@hypervisor15 log]# uname -a Linux hypervisor15 3.10.0-123.20.1.el7.x86_64 #1 SMP Thu Jan 29 18:05:33 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux [root@hypervisor15 log]# cat /etc/redhat-release CentOS Linux release 7.0.1406 (Core) [root@hypervisor15 log]# ls -l /dev/kvm crw-------. 1 root root 10, 232 Feb 20 22:40 /dev/kvm [root@hypervisor15 log]# getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: root user::rw- group::--- other::--- [root@hypervisor15 log]# dmesg | grep -i kvm [root@hypervisor15 log]# rmmod kvm_intel [root@hypervisor15 log]# rmmod kvm [root@hypervisor15 log]# modprobe kvm [root@hypervisor15 log]# modprobe kvm_intel [root@hypervisor15 log]# ls -l /dev/kvm crw-rw-rw-. 1 root kvm 10, 232 Feb 21 04:07 /dev/kvm [root@hypervisor15 log]# getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: kvm user::rw- group::rw- other::rw- [root@hypervisor15 log]# dmesg | grep -i kvm
Seeing this on centos 7 also with internal qemu-rhev package. Machine wasn't rebooted since installing qemu-rhev [root@c7 ~(keystone_admin)]# uname -a Linux c7 3.10.0-123.8.1.el7.x86_64 #1 SMP Mon Sep 22 19:06:58 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux [root@c7 ~(keystone_admin)]# rpm -qa qemu* libvirt qemu-img-rhev-1.5.3-60.el7_0.12.x86_64 libvirt-1.1.1-29.el7_0.7.x86_64 qemu-kvm-rhev-1.5.3-60.el7_0.12.x86_64 qemu-kvm-common-rhev-1.5.3-60.el7_0.12.x86_64 Notice the lack of group perms from: # getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: kvm user::rw- user:padraig:rw- group::--- mask::rw- other::rw- Now resetting the module, restores group perms: [root@c7 ~(keystone_admin)]# ls -l /dev/kvm crw-rw-rw-+ 1 root kvm 10, 232 Feb 23 14:34 /dev/kvm [root@c7 ~(keystone_admin)]# dmesg | grep -i kvm [root@c7 ~(keystone_admin)]# rmmod kvm_intel [root@c7 ~(keystone_admin)]# rmmod kvm [root@c7 ~(keystone_admin)]# modprobe kvm [root@c7 ~(keystone_admin)]# modprobe kvm_intel [root@c7 ~(keystone_admin)]# ls -l /dev/kvm crw-rw-rw-+ 1 root kvm 10, 232 Feb 24 15:12 /dev/kvm [root@c7 ~(keystone_admin)]# getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: kvm user::rw- user:padraig:rw- group::rw- mask::rw- other::rw-
Same issue with CentOS 7, after installing DevStack (OpenStack). [root@lg858 qemu]# rpm -qa |grep qemu qemu-common-2.0.0-1.el7.3.x86_64 qemu-img-1.5.3-60.el7_0.11.x86_64 ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch qemu-kvm-common-1.5.3-60.el7_0.11.x86_64 qemu-system-x86-2.0.0-1.el7.3.x86_64 libvirt-daemon-driver-qemu-1.1.1-29.el7_0.7.x86_64 [root@lg858 qemu]# getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: root user::rw- group::--- other::--- root@lg858 qemu]# rmmod kvm_intel [root@lg858 qemu]# rmmod kvm [root@lg858 qemu]# modprobe kvm [root@lg858 qemu]# modprobe kvm_intel [root@lg858 qemu]# getfacl /dev/kvm getfacl: Removing leading '/' from absolute path names # file: dev/kvm # owner: root # group: root user::rw- group::--- other::--- [root@lg858 qemu]# ls -l /dev/kvm crw------- 1 root root 10, 232 Mar 29 18:30 /dev/kvm I was under the impression libvirt needs the dev to be part of the 'root' group: # The group for QEMU processes run by the system instance. It can be # specified in a similar way to user. #group = "root" Or is it just for the qemu processes? I'm quite sure the maching HAS been rebooted since installation.
I just tried to reproduce with f21, f22, and centos 7.1. After installing qemu-kvm the /dev/kvm permissions are the expected 666. So I really don't know what the issue could be. Maybe this was fixed in 7.1 since the above centos reports sound like 7.0 packages. And Comment #33 seems like it's referencing way outdates EPEL packages as well. Comment #29 says this is still an issue on F21... but again I still can't reproduce so I'm not sure what more I can do. This bug has gotten pretty long and convoluted. I realize it's probably one of the top google hits for this issue, but it's reached its limit of usefulness. If anyone can still reproduce, please do the following: - Document the exact steps to reproduce, starting with how you provisioned the machine. - Please make a _second_ attempt to reproduce, installing a new machine/VM from scratch if need be. Even if you can't reproduce on the second attempt, it will be good to know whether this is deterministic. I have repeatedly been unable to reproduce so either I'm doing something wrong, or it's setup/hw dependent, or it's completely random, but so far I'm unclear on which category it is :) - File a bug with the relevant distro. So if you are using centos or RHEL, please file a bug there and not in the Fedora tracker. If you hit this on Fedora again, please open a _new_ bug report
I'm actually going to try one more thing in the spec file, just forcing 'chmod 666 /dev/kvm' on the initial package install. It's less than ideal because it duplicates 80-kvm.rules permissions, but it also eliminates udevadm and setfacl from the equation which maybe are sporadically failing
qemu-2.3.0-1.fc22 has been submitted as an update for Fedora 22. https://admin.fedoraproject.org/updates/qemu-2.3.0-1.fc22
Package qemu-2.3.0-1.fc22: * should fix your issue, * was pushed to the Fedora 22 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing qemu-2.3.0-1.fc22' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2015-7131/qemu-2.3.0-1.fc22 then log in and leave karma (feedback).
qemu-2.3.0-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.
Still present in RHEL 7.2 with qemu-2.0.0
(In reply to David Jones from comment #39) > Still present in RHEL 7.2 with qemu-2.0.0 There is no qemu-2.0.0 in RHEL 7.2. This bug is for Fedora and likely nothing to do with your problem. If you have an issue with supported packages, then please contact Red Hat's support.