Bug 799986 - libvirtd should explicitly check for existance of configured sanlock directory before trying to register lockspace
libvirtd should explicitly check for existance of configured sanlock director...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.2
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Jiri Denemark
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-05 09:50 EST by Vlastimil Holer
Modified: 2013-02-21 02:08 EST (History)
8 users (show)

See Also:
Fixed In Version: libvirt-0.10.0-0rc1.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-02-21 02:08:32 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch adding dependency on sanlock (543 bytes, patch)
2012-03-05 09:50 EST, Vlastimil Holer
no flags Details | Diff

  None (edit)
Description Vlastimil Holer 2012-03-05 09:50:42 EST
Created attachment 567652 [details]
Patch adding dependency on sanlock

Description of problem:
If libvirtd is configured to use sanlock locking, libvirtd must be started after sanlock. Otherwise the libvirtd fails.
Attached patch for following package version ...

Version-Release number of selected component (if applicable):
libvirt-0.9.4-23.el6_2.6.x86_64
libvirt-lock-sanlock-0.9.4-23.el6_2.6.x86_64
sanlock-1.8-2.el6.x86_64
sanlock-lib-1.8-2.el6.x86_64
  
Actual results:
libvirtd daemon fails to start:
15:10:17.036: 3004: info : libvirt version: 0.9.4, package: 23.el6_2.6 (CentOS BuildSystem <http://bugs.centos.org>, 2012-03-01-10:07:12, c6b6.bsys.dev.centos.org)
15:10:17.036: 3004: error : virLockManagerSanlockSetupLockspace:241 : Unable to add lockspace /mnt/shared/sanlock/__LIBVIRT__DISKS__: No such file or directory
15:10:17.196: 3004: error : qemudLoadDriverConfig:457 : Failed to load lock manager sanlock
15:10:17.197: 3004: error : qemudStartup:566 : Missing lock manager implementation
15:10:17.198: 3004: error : virStateInitialize:849 : Initialization of QEMU state driver failed
15:10:17.274: 3004: error : daemonRunStateInit:1149 : Driver state initialization failed

Expected results:
Sanlock starts first, then libvirtd and everything succeeds.
Comment 3 Przemyslaw Sztoch 2012-03-20 12:50:52 EDT
I have to confirm this problem.

1. My proposition (tested in my environment) for init.d is:
wdmd: # chkconfig: 2345 90 10
sanlock: # chkconfig: 2345 91 09

2. sanlock have to touch "/var/lock/subsys/sanlock"
Without it, we have problem with nice&successful shutdown - sanlock is working (/etc/rc.d/rc will not kill sanlock), and gfs2 service can't unmount.

3. /etc/init.d/libvirtd have to check that shared filesystem is mounted:
if [ qemu.conf->lock_manager = "sanlock" ] then
  if [ ! -f /var/lib/libvirt/sanlock/shared ]; then
    echo "SHARED FILESYSTEM IS NOT MOUNTED or is not marked by file 'shared'!!!"
    echo "Check it and if mounted then mark by command touch shared.'
    exit 1
  fi
fi

Sorry for poor english :-)
Comment 4 Przemyslaw Sztoch 2012-04-05 08:37:44 EDT
Please, escalate this bugs!
Comment 5 Przemyslaw Sztoch 2012-06-08 18:14:28 EDT
And?!
Comment 6 Daniel Berrange 2012-06-11 10:22:34 EDT
There are multiple problems in this BZ report, related to sanlock itself

 - sanlock not using /var/lock/subsys/sanlock

   https://bugzilla.redhat.com/show_bug.cgi?id=830848

 - sanlock & wdmd initscript priority is wrong

   https://bugzilla.redhat.com/show_bug.cgi?id=830858


While I don't think the libvirtd initscript should be checking for existance of /var/lib/libvirt/sanlock itself, we could improve the lock manager startup code to check this so you get a better error message in syslog.
Comment 7 Daniel Berrange 2012-06-11 10:25:39 EDT
Leaving this BZ to track improved libvirt directory checking. The other new BZs I mentioned above will track the sanlock initscript ordering problems
Comment 9 yanbing du 2012-07-30 02:28:33 EDT
I can reproduce this problem, and it's more like a duplicate bug with bug 820173.
Comment 10 Jiri Denemark 2012-08-21 12:14:28 EDT
This issue is now fixed upstream by v0.10.0-rc0-205-g2560a51:

commit 2560a51ef3f72711852c365ecf1efd6212c0793b
Author: Jiri Denemark <jdenemar@redhat.com>
Date:   Tue Aug 21 15:27:10 2012 +0200

    sanlock: Provide better error if lockspace directory is missing
    
    Generating "Unable to add lockspace /lock/space/dir/__LIBVIRT__DISKS__:
    No such file or directory" is correct but not exactly clear. This patch
    changes the error message to "Unable to create lockspace
    /lock/space/dir/__LIBVIRT__DISKS__: parent directory does not exist or
    is not a directory".
Comment 12 Alex Jia 2012-08-23 05:14:27 EDT
I can reproduce this issue on libvirt-0.10.0-0rc0.el6 and it's okay on libvirt-0.10.0-0rc1.el6, I can get a expected error in libvirtd.log:

<snip>

2012-08-23 09:17:42.331+0000: 26264: debug : virLockManagerSanlockSetupLockspace:178 : Lockspace /var/lib/libvirt/sanlock/__LIBVIRT__DISKS__ does not yet exist
2012-08-23 09:17:42.331+0000: 26264: error : virLockManagerSanlockSetupLockspace:188 : internal error Unable to create lockspace /var/lib/libvirt/sanlock/__LIBVIRT__DISKS__: parent directory does not exist or is not a directory

</snip>

A simple verification method is to remove '/var/lib/libvirt/sanlock' if it exists then configuration basic 'lock manager' in qemu.conf and qemu-sanlock.conf and restart libvirtd service.
Comment 13 errata-xmlrpc 2013-02-21 02:08:32 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-0276.html

Note You need to log in before you can comment on or make changes to this bug.