Bug 843013

Have you changed your lvm configuration? Please, check /etc/lvm/lvm.conf and the global/locking_type setting. By default and after installation it's set to 1 (the "file-based locking"), one can use 4 (the "read-only locking") in specific situations, but it's definitely not the default one used as this locking does not allow metadata change...

Problem persists after a reboot.

...and do you have "locking_type=1" set in /etc/lvm/lvm.conf?

(In reply to comment #1)
> Have you changed your lvm configuration? Please, check /etc/lvm/lvm.conf and
> the global/locking_type setting. By default and after installation it's set
> to 1 (the "file-based locking"), one can use 4 (the "read-only locking") in
> specific situations, but it's definitely not the default one used as this
> locking does not allow metadata change...

Not knowingly, but there is an rpmnew file.  Below are
the differences.  Perhaps some rpm %post or the /run move
tries to edit the file by hand?

--- /etc/lvm/lvm.conf	2012-07-08 18:38:00.131335147 +0100
+++ /etc/lvm/lvm.conf.rpmnew	2012-07-04 10:22:02.000000000 +0100
@@ -366,7 +366,7 @@
     # Type 3 uses built-in clustered locking.
     # Type 4 uses read-only locking which forbids any operations that might 
     # change metadata.
-    locking_type = 4
+    locking_type = 1
 
     # Set to 0 to fail when a lock request cannot be satisfied immediately.
     wait_for_locks = 1
@@ -386,7 +386,7 @@
 
     # Local non-LV directory that holds file-based locks while commands are
     # in progress.  A directory like /tmp that may get wiped on reboot is OK.
-    locking_dir = "/var/lock/lvm"
+    locking_dir = "/run/lock/lvm"
 
     # Whenever there are competing read-only and read-write access requests for
     # a volume group's metadata, instead of always granting the read-only
@@ -477,7 +477,7 @@
     # The thin tools are available as part of the device-mapper-persistent-data
     # package from https://github.com/jthornber/thin-provisioning-tools.
     #
-    thin_check_executable = "/sbin/thin_check"
+    thin_check_executable = "/usr/sbin/thin_check"
 
     # String with options passed with thin_check command. By default,
     # option '-q' is for quiet output.
@@ -550,6 +550,15 @@
     #
     # volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
 
+    # If auto_activation_volume_list is defined, each LV that is to be
+    # activated is checked against the list while using the autoactivation
+    # option (--activate ay/-a ay), and if it matches, it is activated.
+    #   "vgname" and "vgname/lvname" are matched exactly.
+    #   "@tag" matches any tag set in the LV or VG.
+    #   "@*" matches if any tag defined on the host is also set in the LV or VG
+    #
+    # auto_activation_volume_list = [ "vg1", "vg2/lvol1", "@tag1", "@*" ]
+
     # If read_only_volume_list is defined, each LV that is to be activated 
     # is checked against the list, and if it matches, it as activated
     # in read-only mode.  (This overrides '--permission rw' stored in the
@@ -778,5 +787,5 @@
 
     # Full path of the dmeventd binary.
     #
-    # executable = "/sbin/dmeventd"
+    # executable = "/usr/sbin/dmeventd"
 }

Setting locking_type = 1 does fix the problem.

Changing the bug summary.

(In reply to comment #4)
> -    locking_type = 4
> +    locking_type = 1
>  

Well, I've looked at the rpms directly (http://koji.fedoraproject.org/koji/buildinfo?buildID=336465) and it's "locking_type = 1" there. So the conf must have been edited by someone (it would be a huge mistake to put that in an rpm otherwise!).

I'll double check though...

The timestamps on the files are interesting:

--- /etc/lvm/lvm.conf	2012-07-08 18:38:00.131335147 +0100

^ This date corresponds to a large yum install from Rawhide.
Specifically I installed oprofile and the kernel debuginfo,
but it also updated many other packages including systemd,
libvirt, libmount, util-linux, pulseaudio, xorg, selinux-policy
and more.

+++ /etc/lvm/lvm.conf.rpmnew	2012-07-04 10:22:02.000000000 +0100

^ This date presumably comes from the RPM file itself, so doesn't
really mean anything in the context of this error.

Yes, there's been mass rebuild in rawhide recently. This must have been changed from "1" to "4" by manually editing the lvm.conf.

Now, regarding the error message, I think it's quite adequate - it says what's wrong exactly: wrong locking used. Read-only locking is supposed to behave this way for any command trying to modify metada (thus trying to take the read-write lock).

!!!BUT!!! This might be the consequence of the recent bugs in dracut which installed everything to the root fs instead of initramfs image. Yeah, that's certainly the case here as well, I bet!

http://lists.fedoraproject.org/pipermail/devel/2012-July/169602.html

There was a bug in dracut installing everything in root instead on to ramdisk.
I guess this is just side effect of it. I think you even replied to that thread :)

Yes. That's exactly the case. Dracut uses locking_type=4 for LVM :) So I'm closing this bug then.

Yes I agree this was the dracut problem.

However the error message is obscure.  I'm an expert in most
things Linux, but I could not work out even where to begin
fixing this bug, even with extensive searching on Google
*and* examining the upstream source code.  How would anyone
else be expected to diagnose it?

All you need to do to improve LVM is to edit the bug message
to point out that the 'global_locking' value in lvm.conf may
need to be adjusted.

(In reply to comment #13)
> Yes I agree this was the dracut problem.
> 
> However the error message is obscure.  I'm an expert in most
> things Linux, but I could not work out even where to begin
> fixing this bug, even with extensive searching on Google
> *and* examining the upstream source code.  How would anyone
> else be expected to diagnose it?
> 

Looking at lvm.conf?

> All you need to do to improve LVM is to edit the bug message
> to point out that the 'global_locking' value in lvm.conf may
> need to be adjusted.

Yes, but then we'd need to add such messages for all the other settings. The lvm.conf is the first place to look at. If that conf file stayed untouched, everything would work well. It's just a bad luck that dracut changed that. If any user changes the lvm.conf by hand, he can see all the accompanying comments in the lvm.conf directly which, in case of this setting says:

  # Type 4 uses read-only locking which forbids any operations that might 
  # change metadata.

Anyway, how about rewording the error message to:

  "Read-only locking configured/set. Write locks are prohibited."

The "set" or "configured" would inform more about something that is configured (and so it will direct the user more to look at lvm.conf).

(I'd like to avoid using "lvm.conf" keyword as configuration could be overloaded by the "--config" argument that could be used for each lvm command separately)

The verbose log says a little bit more:

[0] rawhide/~ # lvcreate -l1 -vvv vg
      Setting activation/monitoring to 1
        Processing: lvcreate -l1 -vvv vg
        O_DIRECT will be used
      Setting global/locking_type to 4
      Setting global/wait_for_locks to 1
    Read-only locking selected. Only read operations permitted.
    ...

Better.  Can we get 'global_locking' into the error message too?
It will give them something to grep -r /etc with.

How about this:

  "Read-only locking configured via global/locking_type setting. Write locks are prohibited."

Yes, that is much clearer.

If you look at the Google results you'll see this would
have been much easier to diagnose:

https://encrypted.google.com/search?q=global%2Flocking_type

http://git.fedorahosted.org/git/?p=lvm2.git;a=commit;h=a05ac836412f1a9ebc6bc90fe611ff6808d6d844

Well, finally, I had to change it to what was proposed in comment #14:

  "Read-only locking type set. Write locks are prohibited."

I couldn't write "configured via <concrete_setting>" - there is also the automatic fallback from cluster and file locking to read-only locking as the last thing we do to make the command pass even if the other types of locking fail (not allowing metadata change in that case of course).