Bug 1561669

Yep - thanks for report.

I believe 'lvconvert --uncache|--splitcache' is supposed to work in cases PV with cache data & metadata are missing,  but 'vgreduce --removemissing' is not using this code piece.

It's longer outstanding problem where we try to define better consistent rules - ATM focusing on  'raid' logic - where we the consistently apply rules on other targets.

Ah, I was wondering when I wrote it; whether lvconvert --uncache would work, I hadn't actually tried it myself back then, sorry, thanks for the heads up.

I think for LVM cache what's really important is that it should always auto-detect out-of-sync problems and automatic invalidation (cache wipe) on such occasions; in addition to that being able to boot without the cache I believe

Perhaps by forcing it to activate (not the default) but a cache state should not be considered more important I think than a bootable system.

I mean if the cache automatically invalidates when gone, and automatically rebuilds when present (build anew) then you have a system that survives the cache disk being removed from the system with grace;

apart from the problem that VG operations are not possible during that time.

Perhaps automatic "ignoring the missing cache" would not always be desirable, but that could be a configuration choice.

If lvconvert --uncache works with missing cache that's okay, but recreating the cache (by manual commands) is annoying and should really be avoided I think if not necessary.

It's a shame cache disk metadata has equal "authority" as origin disk metadata, honestly if a disk is only used for cache, and in many cases it would, it wouldn't really need any metadata of its own (I mean VG config), that would reduce chances of inconsistent VG metadata when an earlier missing cache is reattached to the system after it had been vgreduce --remove-missing'd, otherwise there is conflicting metadata that can only be resolved with manual edit?

Any PV holding cache volumes should really not have any metadata (for the VG), and if you did have additional "ordinary" volumes on the PV, maybe you should just move that to a different PV (partition); at least, the PV would be unusable without the "main" disk.

I guess there is the --metadataignore option (for pvcreate) for such a use case?

Sorry for rambling. That would imply that cache PV is best created with --metadataignore y or --metadatacopies 0, I didn't know that yet.

In any case, --metadataignore y would work well with vgreduce --removemissing, of course, you would have to recreate the cache (and the PV or its inclusion in the VG) afterwards if desired.

Alternatively automatic invalidation / rebuilding would make it easier to ignore the missing cache disk for a while without removing it.

What happens if PV has --metadataignore y and is missing; are VG operations still possible?

(In reply to Xen from comment #2)
> Ah, I was wondering when I wrote it; whether lvconvert --uncache would work,
> I hadn't actually tried it myself back then, sorry, thanks for the heads up.
> 
> I think for LVM cache what's really important is that it should always
> auto-detect out-of-sync problems and automatic invalidation (cache wipe) on
> such occasions; in addition to that being able to boot without the cache I
> believe

For detection of out-of-sync cache - the cache has to by physically present.
LVM2 does not store any sort of this info in it's metadata.

There is 'assumption' when cache is in 'writethrough' mode - the cache can be dropped anytime - but if user would have manually changed mode with 'dmsetup' -  such assumption may lead to data lose.

So normally lvm2 prefer to have access it cache metadata.

In general it's hard to guess when it's wanted 'automatic' activation if PV with cache is missing (or cache got i.e. write-error or some other kind of problem)

So the lvm2 aims rather for proper tool support and admin picks what's wanted in given moment.

> Perhaps by forcing it to activate (not the default) but a cache state should
> not be considered more important I think than a bootable system.

In case PV is missing - the VG is getting inconsistent - and there is 'limited' number of actions we allow to do - it has large number of complex cases so while it's not visible on first sight there is huge impact on LVs sharing space with missing PV and each case has long list of details for solving.

> I mean if the cache automatically invalidates when gone, and automatically
> rebuilds when present (build anew) then you have a system that survives the
> cache disk being removed from the system with grace;

The simple case is -  'lvconvert --uncache' is meant to work.
Cache can by recreated easily anytime later again.

The key problem current '--uncache' code has is it's misusing 'partial' activation - this needs to be fixed (together with raid code).

> Perhaps automatic "ignoring the missing cache" would not always be
> desirable, but that could be a configuration choice.

We have already created RFE Bug 1305573   for what we call 'optional' PV.
In queue with low priority ATM.

> 
> If lvconvert --uncache works with missing cache that's okay, but recreating
> the cache (by manual commands) is annoying and should really be avoided I
> think if not necessary.

Creation of cache isn't really complicated either - especially when user prepares profile.

Also cachepool (data & metadata) can be just spitted - so can be reattached later again for caching any LV  (--splitcache)

> It's a shame cache disk metadata has equal "authority" as origin disk
> metadata, honestly if a disk is only used for cache, and in many cases it

It's NOT shame at all - it's key principle about VG consistency and safety. 
We often get 'suggestion' to support devices without any PV headers - just by addressing them like  /dev/sda  :) - this thing is however unsupportable.

While often users do think it's secure - there is no concept of device ownership and we do not intend at all to step of this 'dirty water' of plain craziness ;)

So lvm2 always requires PV UUID header as key element to keep things properly recognizable.

> I guess there is the --metadataignore option (for pvcreate) for such a use
> case?

PV has UUID (in its header) and some 'space' for metadata (metadata area).
Option  --metadataignore  makes this area ignore - but there is still a PV (just some other PV in VG has metadata for it...)

If user uses 1000PV in single VG - he may want to have only 10 metadata copies maintained - remaining 990 PV have metadata ignored - this is surely way more efficient in terms of speed of metadata update but clearly way less secure in case those 10 PVs holding metadata are lost in emergency case.... Admin has to choice best pick...