Red Hat Bugzilla – Bug 817770
Investigate ways to improve internal interfaces/caching to support more-granular metadata
Last modified: 2016-07-05 17:03:14 EDT
E.g. provide the ability to update a subset of LVs in a VG in isolation
Maybe LV-level functions/caching.
Since RHEL 6.3 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.
Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.
I think the two changes needed first are:
- remove most/all of lvmcache
- add a per-LV query interface to lvmetad
Both should get a separate BZ tickets I think. For the rest, I suggest following:
- internally, use a transaction-oriented interface
- existing code that requires multiple commits needs to be refactored
- complex commits need to be handled more atomically
- all possibly-failing *computation* needs to be done before the commit starts
- this is especially troublesome with current mirror handling code
and more so with cluster mirrors
What I suggest: the "computation" part of a transaction builds up an action log, which has both DM actions (suspend, resume, etc.) and metadata actions (write, commit, update lvmetad) appropriately interspersed. The log is played out on a commit, discarded on abort and it could also contain an undo action for each step. That would make it possible to clean up a partially aborted commit (which currently leave behind messed up state in many cases).
The idea is, that a "traditional" LVM2 VG will, upon opening a transaction, read the VG metadata (either from lvmetad, or through a globally-locked scan operation) without grabbing VG locks. The commit can then obtain appropriate locks (i.e. lock out both scanning and other commits on the same VG), check that seqno did not change and either proceed with repeating the action log or aborting the transaction. Since the computation within a transaction is side-effect free (it instead builds an action log), it can be restarted easily. Therefore, this approach improves concurrency (especially for read-intensive workloads), even without a new metadata format.
The tricky part is reworking the operations that run within a transaction. It might be possible to go incrementally about this, but it all depends on how the above works out.
This change then enables better granularity by hiding the VG itself behind an opaque concept of transaction. The transaction does *not* expose a VG structure, only per-LV and per-PV queries (lookup by name, uuid, parameter queries, segments...). LV and PV enumeration should be possible too, although used sparingly.
When the above changes are done, a LVM3 format can then implement the transaction-oriented APIs directly, on top of a transactional metadata store. Action log is still required for DM actions. However, fewer conflicts would arise with such a store, allowing further concurrency, and less reading and parsing would need to be done when opening transactions (linear in the operation size, not in the VG size).
Reminds me of one more requirement I forgot to throw into the mix: the ability to find out from any lvm command precisely what changes it would make without actually changing anything yet. (A souped-up --test that can cope with multi-step processes.)
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Since we are unable to provide this feature at this time,
it has been proposed for the next release of
Red Hat Enterprise Linux.
This is not a change suitable for RHEL6, moving to RHEl7 for consideration.