Bug 829567 - RHEL 6.3 requires reboot to build rhevm/vsphere images, since qemu-kvm kernel module loading changed
RHEL 6.3 requires reboot to build rhevm/vsphere images, since qemu-kvm kernel...
Status: CLOSED DUPLICATE of bug 836498
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
6.3
Unspecified Unspecified
unspecified Severity high
: rc
: ---
Assigned To: Virtualization Maintenance
Virtualization Bugs
RHEL63
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-06 23:34 EDT by Dave Johnson
Modified: 2012-07-11 15:46 EDT (History)
13 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-07-11 13:08:22 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
'yum install qemu-kvm' output (3.89 KB, text/plain)
2012-06-13 23:59 EDT, Dave Johnson
no flags Details

  None (edit)
Description Dave Johnson 2012-06-06 23:34:46 EDT
Description of problem:
==============================
Fought this issue all day... at first it was thought that the issue was because we were mixing rhel 6.2/6.3 rpms.  Cleared that up and got our rhel 6.2 installs pulling dependencies from a rhel 6.2 repo, everything worked great, was able to build/push/deploy images to all providers.

However, as soon as I installed a RHEL 6.3 system with appropriate 6.3 repos, the problem reappeared.  Basically kvm kernel modules are not being loaded after after installing/configuring aeolus and rhevm/vsphere builds fail with "OzException: This host does not support virtualization type kvm or qemu".

Loading the appropriate kvm modules and restarting libvirtd and imagefactory did not help.  It was only after a reboot that building begain working.

Version-Release number of selected component (if applicable):
=================================================================
imagefactory-1.0.0rc11-1.el6.noarch


How reproducible:
====================
100%

Steps to Reproduce:
=======================
1.  install CFCE v1.0.1 on RHEL 6.3
2.  configure for rhevm and/or vsphere
3.  build image
  
Actual results:
===============
OzException

Expected results:
=================
built image
Comment 1 Steve Loranz 2012-06-07 17:06:08 EDT
This seems like something that should be logged against libvirt/kvm instead of imagefactory. I do not see imagefactory or Oz doing any kvm setup other than requiring libvirt as one of the rpm dependencies.
Comment 2 jrd 2012-06-08 11:35:26 EDT
Our current theory is that this has nothing to do with image factory, and is pointing to an underlying kvm or libvirt problem.  Dave?
Comment 3 Dave Allan 2012-06-08 13:03:36 EDT
This behavior really sounds like libvirt wasn't restarted after KVM was running properly.  Eric, can you work with Dave J. on this?
Comment 4 Eric Blake 2012-06-08 13:08:11 EDT
Are you able to easily reproduce the scenario?  Can you run 'virt-host-validate' both before and after each step that you take in your efforts to get things started (enabling the kvm module, restarting libvirtd, and finally rebooting), to see if it detects something obviously difference between each step?
Comment 5 Dave Johnson 2012-06-13 11:55:52 EDT
So basically this comes down to the loading of the kvm modules.  On 6.2 installs, this would happen during the install of imagefactory and its dependencies (libvirt/qemu/etc) but that doesn't seem to be the case in rhel 6.3.

The virt-host-validate command shows exactly that (on 6.3).  After installing aeolus and its components, in 6.2 the kvm modules are loaded, in 6.3 that is not the case until they are either manually loaded or a reboot is performed.

Any idea if that is expected?  We could get our aeolus-configure setup script to load these if this is now needed.
Comment 6 Eric Blake 2012-06-13 15:41:02 EDT
Fedora packaging policy is that installing an rpm must NOT start a service by default (instead, it can register the service to be auto-started on next reboot, or the user can manually start it after installation if they don't want to wait for reboot).  This is so that it is possible to install an rpm into a chroot jail, where starting a service makes no sense.  For a similar reason, I'm finding it hard to believe that installing any package should ever load a kernel module - I can see that the first time you USE a package (whether by manual start or by rebooting after the install registered the package for autostart) being the useful place to load a module.

I don't know if the change to not autoload the kvm module was intentional, or for that matter, even which package it was that in 6.2 was autoloading the kvm module.  But since I think the new 6.3 semantics of 'install does not autoload modules' seems to make sense, I think it may be an issue where aeolus-configure will need to load the kvm module rather than relying on installation doing the same trick.
Comment 10 Dave Johnson 2012-06-13 23:59:10 EDT
Created attachment 591725 [details]
'yum install qemu-kvm' output

qemu-kvm-0.12.1.2-2.209.el6.x86_64 is the one loading the kvm modules on its rpm install in rhel 6.2... for whatever reason it stopped doing this in 6.3 (which is installing qemu-kvm.x86_64 2:0.12.1.2-2.295.el6)
Comment 11 Eric Blake 2012-06-14 07:26:02 EDT
Reassigning to qemu-kvm (perhaps temporarily) to determine if the change in the rpm installation no longer loading the kernel module was intentional.  If so, we can reassign it back to imagefactory to deal with the fallout.
Comment 13 jrd 2012-06-14 09:33:57 EDT
(In reply to comment #11)
> Reassigning to qemu-kvm (perhaps temporarily) to determine if the change in
> the rpm installation no longer loading the kernel module was intentional. 
> If so, we can reassign it back to imagefactory to deal with the fallout.

Well, ok, so it sounds like where you're going with this is that if some random app (eg imagefactory) wants services like libvirtd, it's responsible for ensuring they're started?  That doesn't sound unreasonable, btw, just making sure I'm correctly parsing the intent...
Comment 14 Eric Blake 2012-06-14 09:46:21 EDT
(In reply to comment #13)

> Well, ok, so it sounds like where you're going with this is that if some
> random app (eg imagefactory) wants services like libvirtd, it's responsible
> for ensuring they're started?  That doesn't sound unreasonable, btw, just
> making sure I'm correctly parsing the intent...

Yes, if imagefactory needs to run a qemu-kvm guest and starts by installing software, then it is reasonable for imagefactory to ensure that after qemu and libvirt are installed, that the kvm kernel module loaded, and libvirtd started, all before trying to use them, since the install process alone should not be starting services or loading modules.
Comment 15 jrd 2012-06-14 13:11:01 EDT
Well, ok, but to my knowledge, imagefac doesn't believe it's installing software on the host at runtime.  IOW, we come up and try to start doing our work, and implicitly assume that the needed services are there.

Was the scenario which spawned this bz as follows?

1.  start from base image
2.  install software including imagefac, which by side-effect pulled in libvirt
3.  start imagefac

?

Even in that case, it may be that imagefac should expect to ensure, on its own, that right services are started.  But if, for example, we believe that this scenario should work without imagefac explicitly starting services...

1.  start from base image
2.  install software including imagefac, which by side-effect pulled in libvirt
3.  reboot
4.  start imagefac


... then I'd argue that this bug should be closed.  

Now, one could argue that imagefac should not be making assumptions, and even if installing code and rebooting the machine might be expected to start things, good practice says we should assume nothing, that's fine too.

Mostly just trying to get crisp about what expectations should be about services.
Comment 16 Ademar Reis 2012-06-18 17:33:41 EDT
Based on comment 15, it doesn't look like there's anything qemu-kvm can do, so I'm moving this bug back to imagefactory.
Comment 17 James Laska 2012-06-25 11:09:51 EDT
What is the next step for this issue?  Do the imagefactory SysV (and systemd) startup procedures need to check whether libvirtd is running *and* whether the kvm module(s) are loaded?
Comment 18 Thomas Oulevey 2012-06-29 03:56:08 EDT
Hi,

It seems there is a bug in qemu-kvm.

In the %post section of the spec file Redhat added a If statement that change the behavior of qemu-kvm package if qemu-kvm-agent is defined, then the sh %{_sysconfdir}/sysconfig/modules/kvm.modules script is not added to qemu-kvm package because another %post statement is added and change the logic.

Thomas.
Comment 19 Dave Johnson 2012-07-11 11:07:27 EDT
moving to qemu-kvm team based on comment 18
Comment 20 Thomas Oulevey 2012-07-11 11:14:09 EDT
BTW, private bugzilla #836498 has been opened by our support contact.
Comment 22 Ademar Reis 2012-07-11 13:08:22 EDT
Closing as dupe of Bug 836498, which has a customer portal case attached to it.

*** This bug has been marked as a duplicate of bug 836498 ***
Comment 23 Garrett Holmstrom 2012-07-11 14:44:45 EDT
Do I have any hope of being able to follow this bug in the future?  This affects Eucalyptus as well.
Comment 24 Ademar Reis 2012-07-11 15:46:28 EDT
Garret, the other bug is restricted at the moment (I'm checking if we can open the other bug to the public), but FYI, the patch is trivial:

--- qemu-kvm.spec.orig	2012-06-29 11:29:51.060386352 +0200
+++ qemu-kvm.spec	2012-06-29 11:30:25.141384511 +0200
@@ -8110,16 +8110,16 @@
 /sbin/chkconfig --add ksm
 /sbin/chkconfig --add ksmtuned
 
+# load kvm modules now, so we can make sure no reboot is needed.
+# If there's already a kvm module installed, we don't mess with it
+sh %{_sysconfdir}/sysconfig/modules/kvm.modules
+
 %if %{with guest_agent}
 %post -n qemu-guest-agent%{?pkgsuffix}
 /sbin/chkconfig --add qemu-ga
 /sbin/chkconfig qemu-ga off
 %endif # guest_agent
 
-# load kvm modules now, so we can make sure no reboot is needed.
-# If there's already a kvm module installed, we don't mess with it
-sh %{_sysconfdir}/sysconfig/modules/kvm.modules
-
 %preun
 if [ $1 -eq 0 ]; then
     /sbin/service ksmtuned stop &>/dev/null || :

Note You need to log in before you can comment on or make changes to this bug.