Bug 860658

Summary: nova boot fails because /dev/kvm is 0600 instead of 0666
Product: [Fedora] Fedora Reporter: Sandro Mathys <sandro>
Component: qemuAssignee: Fedora Virtualization Maintainers <virt-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 18CC: akscram, alexander.sakhnov, amit.shah, apevec, apevec, asalkeld, berrange, bfilippov, breu, cfergeau, crobinso, dwmw2, itamar, Jan.van.Eldik, jonathansteffan, jose.castro.leon, knoel, markmc, matt_domsch, mlvov, pbonzini, p, rbryant, rjones, rkukura, scottt.tw, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 863374 (view as bug list) Environment:
Last Closed: 2012-09-28 11:48:09 EDT Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Description Flags
scheduler.log none

Description Sandro Mathys 2012-09-26 08:22:16 EDT
Description of problem:
Following the test day instructions (with the quantum option) on F18 bare metal, nova boot ends with an ERROR state.

Version-Release number of selected component (if applicable):

How reproducible:
Not sure, I think I've seen it twice (out of 2) and others suggested they've seen the same.

Steps to Reproduce:
Actual results:

Expected results:

Additional info:
The logs (which I'll attach in a minute) suggested issues with libvirt/kvm. In #fedora-openstack, derekh suggested permissions on /dev/kvm could be wrong. It turned out they were 0600 while they should probably be 0666 (as on F17) instead. chmod 0666 /dev/kvm therefore fixed the problem. It's been suggested that for others a reboot (instead of the chmod) fixed things as well.

Not sure whether this should be considered a bug in udev (or ...) instead but I figure that change happened for a reason.

Also, it was suggested I run sudo su - qemu -s /bin/sh -c virt-host-validate:
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking for device /dev/kvm                                         : FAIL (Check that the 'kvm-intel' or 'kvm-amd' modules are loaded & the BIOS has enabled virtualization)
  QEMU: Checking for device /dev/vhost-net                                   : WARN (Load the 'vhost_net' module to improve performance of virtio networking)
  QEMU: Checking for device /dev/net/tun                                     : PASS
   LXC: Checking for Linux >= 2.6.26                                         : PASS
...but kvm and kvm-intel were loaded alright. After the chmod that test PASSes alright.
Comment 1 Sandro Mathys 2012-09-26 08:24:07 EDT
Created attachment 617531 [details]
Comment 2 Sandro Mathys 2012-09-26 08:25:34 EDT
Created attachment 617534 [details]
Comment 3 Sandro Mathys 2012-09-26 08:32:32 EDT
Note regarding the logs: what I did was roughly:
- nova boot 4 systems
- restart scheduler (as suggested in similar bugs)
- nova boot 4 systems
- chmod 0666 /dev/kvm
- nova boot 2 systems
Comment 4 Mark McLoughlin 2012-09-26 08:39:45 EDT
Cole - have you seen reports of bad /dev/kvm permissions in F18?
Comment 5 Sandro Mathys 2012-09-26 09:37:21 EDT
/usr/lib/udev/rules.d/80-kvm.rules:KERNEL=="kvm", GROUP="kvm", MODE="0666"

Figure that explains why reboot helps. But the first time kvm is loaded (libvirtd is installed and started) it seems to get 0600 nevertheless - is /dev/kvm not created by udev in that situation?
Comment 6 Cole Robinson 2012-09-26 21:24:56 EDT
Mark, haven't seen any bug reports go by about that besides this one. Is this consistently reproducible or a one off?
Comment 7 Sandro Mathys 2012-09-27 08:15:23 EDT
Even on a fresh F18 Alpha minimal install (no updates, no openstack):
crw-------. 1 root root 10, 232 Sep 27 08:46 /dev/kvm
Comment 8 Jan van Eldik 2012-09-28 03:13:00 EDT
Just to confirm Sandro's observation on a fresh F17 install, without
updates or openstack:

crw-------. 1 root root 10, 232 Sep 28 08:57 /dev/kvm
Comment 9 Pádraig Brady 2012-09-28 09:51:34 EDT
Moving to qemu as it seems that package on install should do:
  chmod 666 /dev/kvm
or more generally
  udevadm trigger --action=change
Comment 10 Alan Pevec 2012-09-28 10:21:24 EDT
%post in qemu-system-x86 runs /etc/sysconfig/modules/kvm.modules where kvm kernel module is modprobed. Why modprobe itself doesn't trigger udev?
Comment 11 Daniel Berrange 2012-09-28 10:36:21 EDT
The KVM modules are part of the main kernel RPM, so perhaps the module was loaded before the KVM RPM was installed. If this was the case it would have got the default permssiosn, since the udev rules would not be present yet. Then when %post ran module, nothing would be done since they were already loaded (albeit with wrong permissions).
Comment 12 Paolo Bonzini 2012-09-28 11:48:09 EDT
Fixed in qemu-1.2.0-12.fc18 and qemu-1.2.0-12.fc19.
Comment 13 Pádraig Brady 2012-09-28 11:58:49 EDT
For reference the change is:

I was a bit surprised that --action=change wasn't specified,
but I now see that that's the default operation.

Also note this was reported as an issue against F17 as well as F18.

Thanks Paolo!
Comment 14 Fedora Update System 2012-10-08 19:55:49 EDT
qemu-1.2.0-12.fc18 has been submitted as an update for Fedora 18.