I'm opening this to track and gather any ideas for this issue... Sometimes we can see (in the scripts most notably) that the output needs to be laboriously parsed by external tools to filter out some of the output that is not actually needed or is even undesirable. Also, sometimes we need to run specific commands changing metadata or changing the activation status based on some other criteria than just PV/VG/LV name. Examples of such criteria-based selection include reporting/displaying VGs that are clustered or local, or any other type-based criteria like mirror volumes, snapshot volumes, thin volumes, RAID volumes etc. Other criteria might be the size of the volume or whatever we report today in LVM display commands in general... I think this would improve the way the scripts work with LVM commands. Also, the same filtering logic should probably be available via liblvm interface. This is definitely not a high priority feature request, but just something that would be nice to have.
This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle. Changing version to '19'. (As we did not run this process for some time, it could affect also pre-Fedora 19 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19
It may make sense to borrow the command line "selection" syntax for bug 1070381. We want to be able to select vgs from the command line by: name, uuid, tags. If a uuid or tag is used, it requires reading all vg's to match against the uuid/tag args. If a name is used, only the vg's with the given names are read. So a command with a uuid/tag arg can be much more expensive. People can intentionally select vgs only by name to avoid this expense. The problem is that people use uuids as names, and they are indistinct as command line args: we do not know if the arg is meant as one or the other. When a uuid arg is seen, the process_each_vg code has to read all vgs in case the uuid arg will match a vg's uuid, not a name. This means a person with uuids as names, who is selecting the vgs by name on the command line would unavoidably pay the expense of reading all vgs. A solution would be to know from the command line syntax if the arg is meant to match a vg's name or uuid. If we knew this up front, we could avoid unnecessarily reading all vgs. If the selection syntax were something like -o field=value, then: 1. vgs -o uuid=12-34-56-78 2. vgs -o name=12-34-56-78 The first command would read all vg's and attempt to match the uuid arg with a vg's uuid. The second command would only read the named vg. Importantly, this is not only an output filter, but selects the objects to be processed.
I've pushed a development branch 'dev-prajnoha-condout' with the code to support filtering report output (vgs, lvs, pvs, dmsetup as of now) based on conditions supplied: https://git.fedorahosted.org/cgit/lvm2.git/log/?h=dev-prajnoha-condout. This is actually rebased original version of the patchset. The original code is dated back in 2007 and was written by Jun'ichi Nomura: http://www.redhat.com/archives/dm-devel/2007-April/msg00025.html. I've added some improvements like support for comparing sizes with units... I'd still like to explore a possibility to reuse part of this logic for conditional execution as well (e.g. change all LVs that comply with the condition supplied). Still needs more testing, but the reporting part seems to work basically. Example: [1] raw/~ # pvs PV VG Fmt Attr PSize PFree /dev/sda vg1 lvm2 a-- 480.00m 440.00m /dev/sdb vg1 lvm2 a-- 96.00m 96.00m /dev/sdc vg1 lvm2 a-- 124.00m 124.00m /dev/sde vg2 lvm2 a-- 124.00m 72.00m /dev/sdf vg2 lvm2 a-- 124.00m 124.00m /dev/sdg vg2 lvm2 a-- 124.00m 124.00m /dev/sdh other lvm2 a-- 124.00m 28.00m /dev/sdi other lvm2 a-- 124.00m 124.00m /dev/vda2 fedora_raw lvm2 a-- 9.50g 0 [1] raw/~ # vgs VG #PV #LV #SN Attr VSize VFree fedora_raw 1 2 0 wz--n- 9.50g 0 other 2 2 0 wz--n- 248.00m 152.00m vg1 3 4 0 wz--n- 700.00m 660.00m vg2 3 4 0 wz--n- 372.00m 320.00m [1] raw/~ # lvs LV VG Attr LSize root fedora_raw -wi-ao---- 9.00g swap fedora_raw -wi-ao---- 512.00m one other -wi-a----- 32.00m two other -wi-a----- 64.00m lvol0 vg1 -wi-a----- 4.00m lvol1 vg1 -wi-a----- 8.00m lvol2 vg1 -wi-a----- 12.00m lvol3 vg1 -wi-a----- 16.00m lvol0 vg2 -wi-a----- 4.00m lvol1 vg2 -wi-a----- 12.00m lvol2 vg2 -wi-a----- 16.00m lvol3 vg2 -wi-a----- 20.00m [1] raw/~ # pvs --condition 'pv_size > 200m' PV VG Fmt Attr PSize PFree /dev/sda vg1 lvm2 a-- 480.00m 440.00m /dev/vda2 fedora_raw lvm2 a-- 9.50g 0 [1] raw/~ # pvs --condition '(pv_size > 200m) && (pv_size < 9g)' PV VG Fmt Attr PSize PFree /dev/sda vg1 lvm2 a-- 480.00m 440.00m [1] raw/~ # lvs --condition 'lv_name=~"lvol[023]"' LV VG Attr LSize lvol0 vg1 -wi-a----- 4.00m lvol2 vg1 -wi-a----- 12.00m lvol3 vg1 -wi-a----- 16.00m lvol0 vg2 -wi-a----- 4.00m lvol2 vg2 -wi-a----- 16.00m lvol3 vg2 -wi-a----- 20.00m [1] raw/~ # lvs --condition '(lv_name=~"lvol[023]") || (lv_size > 500m)' LV VG Attr LSize root fedora_raw -wi-ao---- 9.00g swap fedora_raw -wi-ao---- 512.00m lvol0 vg1 -wi-a----- 4.00m lvol2 vg1 -wi-a----- 12.00m lvol3 vg1 -wi-a----- 16.00m lvol0 vg2 -wi-a----- 4.00m lvol2 vg2 -wi-a----- 16.00m lvol3 vg2 -wi-a----- 20.00m [1] raw/~ # lvs --condition 'lv_attr=~"ao"' LV VG Attr LSize root fedora_raw -wi-ao---- 9.00g swap fedora_raw -wi-ao---- 512.00m
Second iteration of the patchset: https://git.fedorahosted.org/cgit/lvm2.git/log/?h=dev-prajnoha-selout - renamed --condition to -S/--select - added -S/--select to pv/vg/lvdisplay (with -C) - AND is "+" and OR is "," now - "==" is "=" now - no need to use quotes, so this works now: lvs --select name=abc,vg_name=def - added a check whether a field referenced in the selection is recognized (so "lvs --select unknown" produces: [0] raw # lvs -S bla=abc Unrecognized selection field: bla Selection syntax error at 'bla=abc' ...now I'll try to add the coverage for list matching (e.g. tag list). So that will be in next iteration I hope...
Hopefully final candidate: https://git.fedorahosted.org/cgit/lvm2.git/log/?h=dev-prajnoha-report-select
The final released version looks like this (lvm2 v2.02.107/libdevmapper v1.02.86): pvs/vgs/lvs/pvdisplay/vgdisplay/lvdisplay/lvm devtypes -S SELECTION # pvs -S help Selection operands ------------------ field - Reporting field. number - Non-negative integer value. size - Floating point value with units, 'm' unit used by default if not specified. percent - Non-negative integer with or without % suffix. string - Characters quoted by ' or " or unquoted. string list - Strings enclosed by [ ] and elements delimited by either "all items must match" or "at least one item must match" operator. regular expression - Characters quoted by ' or " or unquoted. Reserved values --------------- -1, undefined, undef, unknown - Reserved value for undefined numeric value. [number] unmanaged - Reserved value for unmanaged number of metadata copies in VG. [number] auto - Reserved value for size that is automatically calculated. [size] Selection operators ------------------- Comparison operators: =~ - Matching regular expression. [regex] !~ - Not matching regular expression. [regex] = - Equal to. [number, size, percent, string, string list] != - Not equal to. [number, size, percent, string, string_list] >= - Greater than or equal to. [number, size, percent] > - Greater than. [number, size, percent] <= - Less than or equal to. [number, size, percent] < - Less than. [number, size, percent] Logical and grouping operators: && - All fields must match , - All fields must match || - At least one field must match # - At least one field must match ! - Logical negation ( - Left parenthesis ) - Right parenthesis [ - List start ] - List end (the reserved values are specific to the tool, e.g. dmsetup doesn't define any of these reserved values) From man lvm (8): Informal grammar specification: STATEMENT = column cmp_op VALUE | STATEMENT log_op STATEMENT | (STATEMENT) | !(STATEMENT) VALUE = [VALUE log_op VALUE] For list-based types: string list. The log_op must always be of one type within the whole list value. VALUE = value For scalar types: number (integer), size (floating point number with size unit suffix), percent (floating point number with or without % suffix), string.
I'm separating this original BZ in two: - criteria-based reporting (done, this bz) - criteria-based processing (to be done, bz #1112544)
There's probably one more thing to cover - date/time fields (e.g. lv creation time). Currently, these fields are recognized as strings which means the comparison is done simply on strings. It would be nice, if we could add a separate field type "DM_REPORT_FIELD_TYPE_DATE_TIME" and then we should be able to compare the date and time formats. I envision this to be very similar to what "git" command has, e.g. 'git log --since "two weeks ago"' - this would look like in LVM: 'lvs --select time < "two weeks"'. Or maybe a new time-comparing operator for this (like since, before...). But this is not high prio at all. Let's have a look at this if there's some spare time. Maybe lots of this code can be just reused: https://github.com/git/git/blob/master/date.c. === Another extensions may include shortcuts like: 'lvs --select name=[a || b || c]' which would be equal to 'lvs --select name=a || name=b || name=c' (this should be easy to add). If found usable for some situations, maybe also transitive comparisons could be recognized: 'lvs --select lv_name=vg_name=abc' which would be equal to 'lvs --select lv_name=abc && vg_name=abc'
(In reply to Peter Rajnoha from comment #8) > There's probably one more thing to cover - date/time fields (e.g. lv > creation time). Currently, these fields are recognized as strings which > means the comparison is done simply on strings. > > It would be nice, if we could add a separate field type > "DM_REPORT_FIELD_TYPE_DATE_TIME" and then we should be able to compare the > date and time formats. I envision this to be very similar to what "git" > command has, e.g. 'git log --since "two weeks ago"' - this would look like > in LVM: 'lvs --select time < "two weeks"'. Or maybe a new time-comparing > operator for this (like since, before...). But this is not high prio at all. > Let's have a look at this if there's some spare time. Maybe lots of this > code can be just reused: https://github.com/git/git/blob/master/date.c. > ...tracked by bug #1112645 > > Another extensions may include shortcuts like: > > 'lvs --select name=[a || b || c]' which would be equal to 'lvs --select > name=a || name=b || name=c' (this should be easy to add). > > If found usable for some situations, maybe also transitive comparisons could > be recognized: > > 'lvs --select lv_name=vg_name=abc' which would be equal to 'lvs --select > lv_name=abc && vg_name=abc' ...tracked by bug #1112646