Bug 438604 - Add environment variable style reporting option to libdevmapper + dmsetup
Summary: Add environment variable style reporting option to libdevmapper + dmsetup
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: device-mapper-obsolete
Version: 10
Hardware: All
OS: Linux
low
low
Target Milestone: ---
Assignee: Alasdair Kergon
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-03-22 18:48 UTC by David Zeuthen
Modified: 2013-03-06 03:55 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-11-18 13:06:31 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
the patch from SUSE's latest srpm (3.65 KB, patch)
2008-03-22 18:48 UTC, David Zeuthen
no flags Details | Diff
udev rules to use the 'dmsetup export' functionality (918 bytes, text/plain)
2008-03-22 18:50 UTC, David Zeuthen
no flags Details

Description David Zeuthen 2008-03-22 18:48:01 UTC
(I'm filing this against lvm2 as it appears that the device-mapper package is
now a subpackage of that source package. Please reassign if this wrong. Thanks.)

In order to 

 - reliably and sanely detect device mapper devices
 - have persistent symlinks

some integration work is needed. Specifically I need this for the next version
of HAL to properly handle encrypted disks using LUKS etc. Right now we use a
lame hack where we look at major/minor numbers as well as naming. As a result
HAL can only identify the the crypto dm devices it set up itself.

It turns out SUSE already did this, will attach their patches. Please apply. Thanks.

Comment 1 David Zeuthen 2008-03-22 18:48:54 UTC
Created attachment 298852 [details]
the patch from SUSE's latest srpm

Comment 2 David Zeuthen 2008-03-22 18:50:13 UTC
Created attachment 298853 [details]
udev rules to use the 'dmsetup export' functionality

This is from SUSE's latest udev srpm. It's also in the udev upstream git
repository.

Comment 3 David Zeuthen 2008-03-22 18:53:34 UTC
FWIW the patch applies fine to the rawhide version of device-mapper.

Comment 4 David Zeuthen 2008-03-22 18:55:48 UTC
Here's an illustration of what it does

[root@zelda ~]# dmsetup export -j 253 -m 0
DM_NAME=myluks
DM_STATE=ACTIVE
DM_TABLE_STATE=LIVE
DM_OPENCOUNT=0
DM_LAST_EVENT_NR=0
DM_MAJOR=253
DM_MINOR=0
DM_TARGET_COUNT=1
DM_TARGET_TYPES=crypt

[root@zelda ~]# udevinfo -q all --name /dev/dm-0 
P: /devices/virtual/block/dm-0
N: dm-0
S: disk/by-id/dm-name-myluks
S: disk/by-uuid/47C2-19CC
S: disk/by-label/eyes\x20only
L: 2147483647
E: DM_NAME=myluks
E: DM_STATE=ACTIVE
E: DM_TABLE_STATE=LIVE
E: DM_OPENCOUNT=0
E: DM_LAST_EVENT_NR=0
E: DM_MAJOR=253
E: DM_MINOR=0
E: DM_TARGET_COUNT=1
E: DM_TARGET_TYPES=crypt
E: ID_FS_USAGE=filesystem
E: ID_FS_TYPE=vfat
E: ID_FS_VERSION=FAT32
E: ID_FS_UUID=47C2-19CC
E: ID_FS_UUID_ENC=47C2-19CC
E: ID_FS_LABEL=eyes only
E: ID_FS_LABEL_ENC=eyes\x20only
E: ID_FS_LABEL_SAFE=eyes_only



Comment 5 Alasdair Kergon 2008-03-22 21:32:50 UTC
I asked for some changes to this before I would accept it upstream, but it never
happened.  So I started to code it myself and got about half-way through before
getting diverted onto something else.

Basically the change needed doing by integrating it into the reporting
mechanism, and the output needed generalising a bit.  (The current patch does
not cope adequately with all the possible cases.)

I'll try to pick it up again...

(Don't recall whether we've already a bugzilla for this or not.)

Comment 6 David Zeuthen 2008-04-21 16:02:40 UTC
Alasdair, did you get a chance to look at this?

Comment 7 Alasdair Kergon 2008-04-21 17:01:32 UTC
See the upstream check-ins in the last couple of days.  Is that sufficient, or
is more needed?

- I still don't think I'll do one-field-per-line output without adding row
number suffices to the variable names e.g DM_NAME_0 DM_NAME_1 etc.

- I'm still not happy with the name '--prefixes' and might change it if I come
up with something better.

- I need to fix the quoting of double quotes, and consider whether to change the
quoting mechanism.

Comment 8 Bug Zapper 2008-05-14 06:47:58 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 9 David Zeuthen 2008-05-28 15:57:01 UTC
(In reply to comment #7)
> See the upstream check-ins in the last couple of days.  Is that sufficient, or
> is more needed?

I'm not well-versed in how the upstream code is organized and I'm reluctant to
replace the device-mapper packages on my system as I'm using root on LVM; is it
possible you can illustrate what it does a'la comment 4?

Also, any chance we can get something like this in Rawhide soon?

> - I still don't think I'll do one-field-per-line output without adding row
> number suffices to the variable names e.g DM_NAME_0 DM_NAME_1 etc.
> 
> - I'm still not happy with the name '--prefixes' and might change it if I come
> up with something better.
> 
> - I need to fix the quoting of double quotes, and consider whether to change the
> quoting mechanism.

Another thing I'd like to pick your brain about is exporting information about
the PV's. The patch from SUSE only exports information about created dm devices. 

Ideally, given a PV, I'd like to know what VG's and LV's it contains. I'd also
like to know how many other PV's are needed. And I'd like this stored in the
udev database. The context here is that this kind of information is necessary in
order to

 - do auto assembly without config files (think hotplug and bootup)
 - provide an UI to configure LVM

For the latter, I'm actually working on UI to do this. This is already done for
md thanks to this commit

http://neil.brown.name/git?p=mdadm;a=commitdiff;h=0d726f17e191094665a5f2af8c2564dedb990502

giving

 [root@quad ~]# udevinfo -q all --name /dev/sde1|grep MD_
 E: MD_LEVEL=raid5
 E: MD_DEVICES=5
 E: MD_UUID=ba6f635b:38e6d08c:8b716019:116beef4
 E: MD_UPDATE_TIME=1205791423
 E: MD_EVENTS=0.22

 [root@quad ~]# udevinfo -q all --name /dev/md2|grep MD_
 E: MD_LEVEL=raid5
 E: MD_DEVICES=5
 E: MD_METADATA=0.90
 E: MD_UUID=ba6f635b:38e6d08c:8b716019:116beef4

so I now have UI that looks like this

 http://people.freedesktop.org/~david/gdu-raid5.png
 http://people.freedesktop.org/~david/gdu-more-raid-progress.png

allowing users to 

 - start/stop md RAID arrays
 - add/remove components

which is pretty useful.

I'd like to do the same for LVM, the UI I have in mind looks like this

 VolGroup00
   LogVol00
   LogVol01
   Unallocated Space

where, when VolGroup00 is selected, there is UI that lists all the PV's of the
volume group. Including adding/removing PV's (removing might mean having to
transfer data from the PV to be removed). It would also include UI to start/stop
the VG. So it would make it very easy for desktop users to configure LVM.

So for this, all I think we need is the following metadata on the PV (similar to
what we have on the RAID components) similar to what pvdisplay gives us 

(this is from my laptop where /dev/dm-0 is the cleartext device for a LUKS
encrypted partition; default anaconda install with encrypted root).

# pvdisplay /dev/dm-0 
  --- Physical volume ---
  PV Name               /dev/dm-0
  VG Name               VolGroup00
  PV Size               48.83 GB / not usable 14.67 MB
  Allocatable           yes 
  PE Size (KByte)       32768
  Total PE              1562
  Free PE               1
  Allocated PE          1561
  PV UUID               hkbZnG-w3XW-qY2r-TWxG-QfGV-TlxK-raoKdL

so ideally, I'd like an --export option to pvdisplay so we can merge

 LVM2_PV_UUID
 LVM2_PV_SIZE
 LVM2_PV_VG_NAME
 LVM2_PV_TOTAL_PE
 LVM2_PV_FREE_PE
 LVM2_PV_ALLOCATED_PE

etc. into the udev database. Does that make sense?


Comment 10 Alasdair Kergon 2008-10-07 20:29:25 UTC
So, in dm we have:
  --nameprefixes --rows --unquoted
which together with -o and a field list should be sufficient I think.

The same options apply to the lvm2 reporting tools.

(We could still add built-in lists of fields commonly required together.)

Comment 11 Bug Zapper 2008-11-26 02:10:36 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 Kay Sievers 2008-12-06 03:48:09 UTC
Most of the stuff should work now:
  $ dmsetup info -c --nameprefixes --unquoted --rows --noheadings \
     -o name,uuid,suspended,readonly,major,minor,open,tables_loaded
  DM_NAME=xyz
  DM_UUID=
  DM_SUSPENDED=Active
  DM_READONLY=Writeable
  DM_MAJOR=253
  DM_MINOR=0
  DM_OPEN=0
  DM_TABLES_LOADED=Live

One thing we are missing, what the "export" did, is a comma separated
list of the table types used for the volume. There are currently
matches in udev rules for:
 ENV{DM_TARGET_TYPES}=="|*error*"
 ENV{DM_TARGET_TYPES}=="*snapshot-origin*"

Can we please get something similar? We don't care about the name or format it is provided, as long as we can get that kind of information. Thanks!

Comment 13 Dave Wysochanski 2008-12-13 01:29:15 UTC
This looks mostly done in rawhide:
# pvs --nameprefixes --rows --unquoted --noheadings -opv_uuid,pv_size,vg_name,pv_pe_count,pv_pe_alloc_count
  LVM2_PV_UUID=in3VUB-M9EB-3RKm-5R8Y-HyLQ-1YNj-VkjkUW
  LVM2_PV_SIZE=7.81G
  LVM2_VG_NAME=VolGroup00f9
  LVM2_PV_PE_COUNT=250
  LVM2_PV_PE_ALLOC_COUNT=249
# rpm -q lvm2 device-mapper
lvm2-2.02.43-1.fc11.x86_64
device-mapper-1.02.29-1.fc11.x86_64

Is all that is left DM_TARGET_TYPES for dmsetup?

Comment 14 Dave Wysochanski 2008-12-13 01:43:08 UTC
Note also that unless you use a devices/filter line, other devices may get scanned in certain cases with pvdisplay/pvs.  To avoid this you can use something like this:
# pvs --nameprefixes --rows --unquoted --noheadings -opv_name,pv_uuid,pv_size,vg_name,pv_pe_count,pv_pe_alloc_count  --config 'devices { filter = ["a/xvda/", "r/.*/"] }' 
  LVM2_PV_NAME=/dev/xvda2
  LVM2_PV_UUID=in3VUB-M9EB-3RKm-5R8Y-HyLQ-1YNj-VkjkUW
  LVM2_PV_SIZE=7.81G
  LVM2_VG_NAME=VolGroup00f9
  LVM2_PV_PE_COUNT=250
  LVM2_PV_PE_ALLOC_COUNT=249

But if you do that you may get a blank vg_name if the PV was created with "--metadatacopies 0".

So you may want to check 'pv_mda_count' and make sure it's non-zero if you get a blank vgname (or you could just check pv_pe_alloc_count == 0)

Comment 15 Kay Sievers 2008-12-13 14:17:50 UTC
(In reply to comment #13)
> This looks mostly done in rawhide:
> # pvs --nameprefixes --rows --unquoted --noheadings
> -opv_uuid,pv_size,vg_name,pv_pe_count,pv_pe_alloc_count
>   LVM2_PV_UUID=in3VUB-M9EB-3RKm-5R8Y-HyLQ-1YNj-VkjkUW
>   LVM2_PV_SIZE=7.81G
>   LVM2_VG_NAME=VolGroup00f9
>   LVM2_PV_PE_COUNT=250
>   LVM2_PV_PE_ALLOC_COUNT=249

Looks good.

> Is all that is left DM_TARGET_TYPES for dmsetup?

Yeah, something like this. We don't actually care about the name or the format, whatever fits for you, we just want to match on the "types", to take actions for certain matches.

That looks promising so far. Thanks for your help! I hope we can get the remaining issues like the "types" solved soon, so we can start using it, and come back to you how it works, and if we may need to improves some things.

Comment 16 Bug Zapper 2009-11-18 12:27:46 UTC
This message is a reminder that Fedora 10 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 10.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '10'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 10's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 10 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 17 Alasdair Kergon 2009-11-18 13:06:31 UTC
Target types information is not going to be provided at this stage as it is too easy to misuse in that form (complex logic is required matching logic we already have inside the tools creating the devices) and it's not what udev rules actually need anyway AFAICT.  (But perhaps we can export extended device classifications to assist the gui tools eventually.)

Instead information is now provided indicating which subsystem created the device - cryptsetup or multipath or lvm2, for example, and information about whether it is meant to be kept private to that subsystem or available for other parts of the system to use (which deals with the snapshot creation problem).

Closing this bugzilla as the new udev rules have hit rawhide.


Note You need to log in before you can comment on or make changes to this bug.