Bug 854763

Summary: 3.1 - vdsmd service should create directory eariler
Product: Red Hat Enterprise Linux 6 Reporter: Alon Bar-Lev <alonbl>
Component: vdsmAssignee: Douglas Schilling Landgraf <dougsland>
Status: CLOSED ERRATA QA Contact: Martin Pavlik <mpavlik>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.3CC: abaron, bazulay, gklein, iheim, ilvovsky, lpeer, mburns, mpavlik, ykaul
Target Milestone: rcKeywords: Reopened, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: vdsm-4.9.6-37.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-04 19:10:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alon Bar-Lev 2012-09-05 18:56:26 UTC
CURRENT

vdsmd creates directory layout after it initialize external components such as libvirtd.

If it fails, the layout is not created, and future upgrades fails.

EXPECTED

create the layout before communicating with external components so tasks that are dependent on vdsm will likely to succeed.

Comment 2 Douglas Schilling Landgraf 2012-09-13 04:23:43 UTC
Hello Alon,

   Looking again this scenario, it's not a real bug since /data/updates is required only for upgrades via Engine interface to upload the new Node ISO, so if the upgrade will be executed via Engine, we already have a Node system working (vdsm as well) and /data/updates created.

The other supported way to upgrade the Node is via media which doesn't use /data/updates.

I do believe, I have created this confusion with my commit message 8a872c2b6171151fba253fb6d3e198486a2457cf (vdsm-upgrade: use python scripts from ovirtnode) that shows how to upgrade the node manually (*not supported*, developer mode).

Additional note: 
Upstream moved already the creation of /data/updates to vdsm_reg/vdsm-reg-setup.in, as reference only:

commit f79fa0dd09d12a3f5f9d4e79ecbbd536da158c19
Author: Xu He Jie <xuhj.ibm.com>
Date:   Wed May 2 08:22:12 2012 +0800

    Move the creation of /data/updates out of init script

Thanks for this bugzilla, I am going to close this as notabug for now. 
Fell free to re-open it if required.

Cheers
Douglas

Comment 3 Alon Bar-Lev 2012-09-13 06:42:53 UTC
I do not follow.

If for some reason vdsm is not up during initial setup, which is what happened to us because of the fw rules.

You end up with a node that has no /data/updates, that cannot be upgraded or reinstalled to fix the above issue.

It should be fixed, it is a bug.

Comment 4 Douglas Schilling Landgraf 2012-09-13 11:27:04 UTC
Hi Alon,

 /me  just figure out that my last night theory fails since it's RHEV-H and we create the /data/updates in every boot and _not_ persist it.

I am going to work on it, asap I will provide an update here!

Thanks!
Douglas

Comment 5 Mike Burns 2012-09-13 19:27:34 UTC
/data/* doesn't need manual persisting.  It's a separate partition.

Alon,  how are you doing the upgrade?  Don't you need vdsmd running to upgrade from RHEVM?  For re-install or upgrade from media, /data/updates doesn't matter, they're not used.

Comment 6 Alon Bar-Lev 2012-09-13 19:36:43 UTC
(In reply to comment #5)
> /data/* doesn't need manual persisting.  It's a separate partition.

Right. And we seen that /data/updates is created by vdsmd.

> Alon,  how are you doing the upgrade?  Don't you need vdsmd running to
> upgrade from RHEVM?  For re-install or upgrade from media, /data/updates
> doesn't matter, they're not used.

We don't need vdsm running, that's true...
But we *DO* need it to run successfully at least once in order to create /data/updates.

Not sure why it was done in the daemon and not during installation...

I prefer to do this during installation... but if not, at least make sure the /data/updates is created before anything can go wrong in vdsmd.

Comment 7 Mike Burns 2012-09-13 19:40:20 UTC
(In reply to comment #6)
> 
> Not sure why it was done in the daemon and not during installation...
> 
> I prefer to do this during installation... but if not, at least make sure
> the /data/updates is created before anything can go wrong in vdsmd.

Because /data is a separate partition that doesn't exist during rpm installation.  It is created during RHEV-H installation.  Moving it from vdsmd to vdsm-reg would actually solve the problem.  vdsm-reg has to run successfully at least once in order to trigger in place upgrade from rhevm, so having it there would ensure that it's always run correctly.

Comment 8 Alon Bar-Lev 2012-09-13 19:42:00 UTC
This is great, anything "earlier" is good for me :)

Comment 9 Douglas Schilling Landgraf 2012-09-13 19:58:34 UTC
Thanks Mike and Alon. In that case, backport the Xu He Jie patch resolves the problem since vdsm-reg init scripts calls vdsm-reg-setup on start().

I will prepare the backport patch.

Reference only, upstream:
commit f79fa0dd09d12a3f5f9d4e79ecbbd536da158c19
Author: Xu He Jie <xuhj.ibm.com>
Date:   Wed May 2 08:22:12 2012 +0800

    Move the creation of /data/updates out of init script

Cheers
Douglas

Comment 10 Douglas Schilling Landgraf 2012-09-14 01:38:58 UTC
Backport downstream:
https://gerrit.eng.lab.tlv.redhat.com/#/c/2113/

Comment 13 Martin Pavlik 2012-11-27 16:27:42 UTC
Hi Alon,

could you provide me with some lead how to test this one?

thanks

Comment 14 Martin Pavlik 2012-11-27 16:50:46 UTC
Verified with vdsm-4.9.6-44.0.el6_3.x86_64
and RHEVH 20121121.0.el6_3

/data/updates is created before vdsmd.pid

drwxr-xr-x. 2 root root 4096 2012-11-27 15:33:19.463701894 +0000 /data/updates/

-rw-rw-r--. 1 vdsm kvm 5 2012-11-27 15:33:28.407763079 +0000 /var/run/vdsm/vdsmd.pid

Comment 16 errata-xmlrpc 2012-12-04 19:10:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1508.html