Bug 849984

Summary: [vdsm] Injeceted vdsm fails to start on rhevh
Product: Red Hat Enterprise Linux 6 Reporter: Kiril Nesenko <knesenko>
Component: vdsmAssignee: Dan Kenigsberg <dkenigsb>
Status: CLOSED UPSTREAM QA Contact: Ilanit Stein <istein>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 6.3CC: abaron, bazulay, dfediuck, fsimonce, iheim, lpeer, mburns, ykaul
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-09-02 10:29:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kiril Nesenko 2012-08-21 12:15:32 UTC
Description of problem:
Injeceted vdsm fails to start on rhevh

# service vdsmd start
error reading information on service libvirt-guests: No such file or directory
error reading information on service libvirtd: No such file or directory
libvirtd: unrecognized service
checking certs..
vdsm: libvirt already configured for vdsm                  [  OK  ]
Starting iscsid: 
diff: /usr/share/doc/libvirt-0.9.10/libvirtd.upstart: No such file or directory
/bin/cp: cannot stat `/usr/share/doc/libvirt-0.9.10/libvirtd.upstart': No such file or directory
vdsm: one of the dependent services did not start, error co[FAILED]

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor release 6.3 (20120813.0.rhev31.el6_3)
vdsm-4.9.6-29.0.el6_3.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Inject new vdsm to rhevh and try to start the service.
2.
3.
  
Actual results:
vdsm fails to start

Expected results:
vdsm should start

Additional info:

Comment 2 Federico Simoncelli 2012-08-21 12:42:16 UTC
The problem is that VDSM tries to update the upstart script every time it starts (vdsm/vdsmd.init.in):

464 start_libvirtd() {
[...]
434     if ! diff -q "$packaged" "$target" >/dev/null;
435     then
436         /bin/cp -p "$packaged" "$target" || return 1
437         /sbin/initctl reload-configuration
438     fi

We should add a check to verify that the file is present before proceeding to the copy.

Comment 3 Federico Simoncelli 2012-08-21 12:44:20 UTC
A patch has been proposed upstream:

commit 41521ed9eb3cd5a3e40e675137d45d9d462e3cde
Author: Federico Simoncelli <fsimonce>
Date:   Tue Aug 21 08:36:50 2012 -0400

    Check if libvirtd.upstart is present before copying
    
    If the (libvirt) packaged upstart script is present in the rpm list
    of files but is absent from the filesystem the initscript was failing
    because the cp command couldn't copy it. This is mostly interesting
    in stripped down installations as the ovirt-node.
    
    Signed-off-by: Federico Simoncelli <fsimonce>
    Change-Id: I1ba7d52d21e2a072b99ffc9055327d217c78e8fc

http://gerrit.ovirt.org/7375

Comment 4 Mike Burns 2012-08-21 12:49:40 UTC
Agree that we should have a check for whether the original file exists or not, but why haven't we seen this problem before?  RHEV-H has always provided a version of libvirtd.conf in /etc/init directly and no version in /usr/share/doc (or wherever the original one is)

Comment 5 Federico Simoncelli 2012-08-22 16:46:17 UTC
(In reply to comment #4)
> Agree that we should have a check for whether the original file exists or
> not, but why haven't we seen this problem before?  RHEV-H has always
> provided a version of libvirtd.conf in /etc/init directly and no version in
> /usr/share/doc (or wherever the original one is)

Do you modify in some way the /etc/init.d/vdsmd script?
For what I see in a rhevh host you have:

420 start() {
[...]
434     start_needed_srv
[...]

Instead of:

434     start_needed_srv && start_libvirtd

The issue was triggered by the fact that a newer version of vdsm was injected into rhevh (updating /etc/init.d/vdsmd). In any case you may want to generalize and propose upstream vdsm your changes instead of keeping them on just rhevh.

Comment 6 Itamar Heim 2012-08-27 08:08:54 UTC
this happened since vdsm was injected into an installed machine, not into the .iso.

Comment 8 Mike Burns 2012-08-28 18:37:15 UTC
check in ovirt-node repository in $GIT_REPO/recipe/rhevh6-post.ks

These are the changes that I see

# semanage is not present in the image and virt_use_nfs is on (see rhbz#642209)
# remove it from vdsmd startup script to avoid error
sed -i 's#/usr/sbin/semanage#/bin/true#' /etc/rc.d/init.d/vdsmd

# libvirtd upstart job is already configured on rhevh
sed -i 's/ && start_libvirtd$//' /etc/rc.d/init.d/vdsmd

# chkconfig results (symlinks) cannnot be peristed
sed -i 's#/sbin/chkconfig \$srv off##' /etc/rc.d/init.d/vdsmd