I think there is some misuse here.

You have to decided if you want to support/manage devices within host or guest.
It's very bad plan to allow both.

In normal case - you set the filter so the host is not seeing devices which are meant for guests - which is IMHO the best idea - since the guess can do whatever he wants with device and can setup any  VG/LV names and uuids.

So it would be really funny if the guest would set up a device in such a way, that host would suddenly start to use this device instead of it's own.

If you want to allow such usability - you would need synchronization elements to be running in such system (i.e. something like clvm)

So in principle you should not allow host to scan & work with device which is under control of guest virtual machine - it should be left private to the guest for numerous of reasons and security of host system is one of the most important one.

So similar like the request for  'lvchange --uuid' support  - I do not think lvm project has anything to do here...

No... I don't think it really matters a lot what you wrote,... at least not to the real world (and I think I know what I'm talking, since at the institute we manage thousands of VMs and as well thousands of physical machines ;-) ).


First, under _normal_ operations you're of course right, that hosts shouldn't directly see the devices from the guests...
But there are many cases when this is not the case, just some very few examples:
- recovery works when the guest doesn't e.g. boot anymore and when it's easy to just mount make the devices available to the (e.g. KVM) host.
- deployment of VMs that are based on an commonly maintained base image... where you want to e.g. create new ssh host keys, passwords, etc. before you start the VM first.
- security infrastructure, e.g. when you stop the guests, and use things like AIDE on them from the host.
- etc. pp.

And of course nothing of this necessarily means that both host and guest use the devices concurrently, which would really be a bit strange and need e.g. clvm.
Nevertheless... even such a setup might be desired by some users... and it shouldn't be upon us to decide whether it's valid or not.


And the whole problem doesn't only apply to guests vs. hosts.
We for example often have the situation, that we move virtual devices (which were created as part of the clone to either other guests... or perhaps to physical machines (i.e. dd it on a physical device).

When doing such, it can also easily happen that you have duplicate UUIDs... and when such devices are already active... you cannot easily change them.

So in principle... the request goes even further, and it would be nice to also be able to change the names of active VGs/LVs  (not sure if that was possible, though).


Cheers,
Chris.

(In reply to Christoph Anton Mitterer from comment #3)
> No... I don't think it really matters a lot what you wrote,... at least not
> to the real world (and I think I know what I'm talking, since at the
> institute we manage thousands of VMs and as well thousands of physical
> machines ;-) ).
> 

The fact the many users are using it in a wrong way will not make it right.
I'm well aware of such kind of wrong usage - since I've friends with same
view :)


> First, under _normal_ operations you're of course right, that hosts
> shouldn't directly see the devices from the guests...

Of course you have to follow this rule in all cases you mention here.
And I'll explain more reasons later.

> But there are many cases when this is not the case, just some very few
> examples:
> - recovery works when the guest doesn't e.g. boot anymore and when it's easy
> to just mount make the devices available to the (e.g. KVM) host.
> - deployment of VMs that are based on an commonly maintained base image...
> where you want to e.g. create new ssh host keys, passwords, etc. before you
> start the VM first.
> - security infrastructure, e.g. when you stop the guests, and use things
> like AIDE on them from the host.
> - etc. pp.

So you run lvm command with filter set EXACTLY to match disk mapping
of your guest.


> And of course nothing of this necessarily means that both host and guest use
> the devices concurrently, which would really be a bit strange and need e.g.
> clvm.

This ONLY works if all devices within host system included all guest system
have UNIQUE uuid.

It seems you want from LVM to resolve this uniqueness somehow automagically
for you - but it's very hard problem.


> Nevertheless... even such a setup might be desired by some users... and it
> shouldn't be upon us to decide whether it's valid or not.

Of course I understand it's desired to have 'some kind of magic' in system :)

> And the whole problem doesn't only apply to guests vs. hosts.
> We for example often have the situation, that we move virtual devices (which
> were created as part of the clone to either other guests... or perhaps to
> physical machines (i.e. dd it on a physical device).
> 
> When doing such, it can also easily happen that you have duplicate UUIDs...
> and when such devices are already active... you cannot easily change them.

You should never ever be able to activate devices with same UUID. And I'm curious how are you achieving that - since that seems like internal bug in dm/lvm. (meaning you have them in one system in its dm table)

> 
> So in principle... the request goes even further, and it would be nice to
> also be able to change the names of active VGs/LVs  (not sure if that was
> possible, though).

Here are couple reasons why the thing you want is very hard.

If you have a VG with PVs which all have unique uuids  (note PV uuid is completely unrelated to LV uuid) - and some PV will disappear.

Now some 'guest' system will change PV UUID to match the missing UUID in host.
Host lvm command will suddenly start to 'fix' missing PV from VG - and will update lvm metadata on such device. I do not want to go into tricky details,
but it's simply important to have clean borderline in your system, so such situation may not happen.  Since the guest system and the host system are using devices rather independently - the content of metadata gets pretty much worthless if there is no locking mechanism between them.

Now - obviously if you really know what you are doing - and you guest is not running while you update lvm metadata on such lvm from host - that's perfectly fine - but if you are expert on this level - then managing unique VG UUID is your least problem.

Since the device mapper is taking UUID identification as 'internal' thing which is unique during whole lifetime of active device.
(Note from man page of dmsetup:  "After a uuid has been set it cannot be changed.") DM only support rename of device. (which still keeps same uuid).

Well that sounds rather like the GNOME folks mentality, that they tell the users what should be right and what not... which is why we've ended up with GNOME shell and similar crap.

Nonetheless... I do not see a single technical or conceptual reason, why cloning VMs, and later accessing their virtual disks from the host (or other hosts shouldn't be valid.
Actually LVM itself proposes just this idea by having clvm.
Similarly, without VMs, nothing speaks against cloning/dumping disks from/to different hosts, which could also easily lead to the situation of duplicate UUIDs. Or an alternative scenario... exporting such (previously cloned) PVs via nbd.

And it's not that I ask for LVM supporting to (actively) run with PVs/VGs of the same UUID... (even though it shouldn't fail utterly and corrupt data)... all I ask is, that one has the possibility to change them (and possibly also the VG names)... while they're active.
So if one know that one might provoke a collision (e.g. by accessing cloned disks)... one can change the UUIDs/names in advance.


>You should never ever be able to activate devices with
>same UUID. And I'm curious how are you achieving that -
>since that seems like internal bug in dm/lvm. (meaning
>you have them in one system in its dm table)
I didn't mean that I was able to (or should be able to) activate VGs with the same UUID...but as said above, what I have is _for example_ PVs/VGs running in different systems, from which I know they have identical UUIDs... I know I'll stop one machine and access it's devices from the other... that would lead to collisions... so I should change the PV/VG UUIDs/names in advance.
But right now I cannot do this (unless I boot them from another system image, USB stick or that like), cause when the PV/VG/LV is active/used... the change doesn't work.



I didn't quite understand what you wrote about splitting out a PV... sounds like you meant to take a single PV from a guest and use it in the host?
That's not what I want... again... what I meant is (one example again with VMs):
- I have two guests that are running, both are clones an hence have the same UUIDs/names.
- I know I will want to access them both from the host (while they're stopped!).. or perhaps move all disks from the one VM into the other.
- This would lead to a collision.

Now I could stop VM b, boot it from a rescue image, change the UUIDs/names in VM b,... and then I could access both from the host without collision.
But that step with the rescue image is really annoying.

I'd rather to what I can do with e.g. ext-filesystems.
Logon to VM b (while it's still running):
Change the UUIDs/names of the VGs/PVs.
Then stop VM a and b... and access them without collisions.

No crude PV mixing or whatever...



Admittedly I don't know too much about the internals of the device mappers... so if that has the limitation of not being able to change the UUID while the container is in use... one would to make it possible there first.

But are you really sure that this is not possible in DM? Cause I just tried it with dm-crypt/LUKS, which (I'd guess) would also use the DM for that matters... and there it works, while the device is opened.


Cheers,
Chris.

(In reply to Christoph Anton Mitterer from comment #5)

> I didn't quite understand what you wrote about splitting out a PV... sounds
> like you meant to take a single PV from a guest and use it in the host?
> That's not what I want... again... what I meant is (one example again with
> VMs):
> - I have two guests that are running, both are clones an hence have the same
> UUIDs/names.
> - I know I will want to access them both from the host (while they're
> stopped!).. or perhaps move all disks from the one VM into the other.
> - This would lead to a collision.
> 
> Now I could stop VM b, boot it from a rescue image, change the UUIDs/names
> in VM b,... and then I could access both from the host without collision.
> But that step with the rescue image is really annoying.
> 
> I'd rather to what I can do with e.g. ext-filesystems.
> Logon to VM b (while it's still running):
> Change the UUIDs/names of the VGs/PVs.
> Then stop VM a and b... and access them without collisions.

I'm missing here the point of booting rescue image to change UUID of devices.

Your host lvm.conf is set to see only physical devices for the host.
(So usually all activate LVs are not considered for scanning on host)
So i.e.  your  device filter is set to accept only your local hw disks
and reject everything else)

Now to change the VG uuid of your guest disk you simply run this command on
your host:

vgchange --uuid --config 'devices{filter=["a|dev/vg/host12345|","r|.*|"]}' vg12345
(eventually vgrename with similar filter settings).
Accepted disk in the filter is the disk used for the guest.

Is that the direction you are looking for ?
Or is this missing something ?
How it should work ?

> I'm missing here the point of booting rescue
> image to change UUID of devices.
The point is, as I wrote several times now, that you cannot change the UUIDs on PV/VGs (neither the names), which are active and used.
E.g.:
# pvchange --uuid /dev/sda1 
  Volume group containing /dev/sda1 has active logical volumes
  0 physical volumes changed / 1 physical volume not changed
or:
# vgchange --uuid vg_system
  Volume group has active logical volumes

and if these are the ones on which your system runs on, you cannot easily make them inactive, unless you stop the system.



I guess we can continue to discuss this whole issue back and forth now forever...
What I wanted and what the original idea of this ticket was, is that it should be possible to change the UUIDs of PVs and VGs, and I guess (even if LV UUIDs are unique only within a VG) those of the LVs... and ideally also the names of VGs/LVs, while these are actively used.
I tried to give you several different scenarios where it could happen that at least PV/VG UUIDs/names collide.
The whole thing seems to be supported as well by all other containers on blockdevices I know of (filesystems, LUKS, haven't tried mdadm though)... so it seems not so uncommon that people get into situations where they want to change these while the devices are active.

Now if you say it's technically impossible... well... very bad IMHO... but then we can't do anything anyway...

If it's technically possible, then you as upstream, have to decide whether you want to implement this, or whether you consider the use-cases of users invalid.
As I've said,... that e.g. ext* supports this for both (UUID and LABEL( while the fs is mounted, shows that it's not so uncommon and quite useful.


Cheers,
Chris.

Just go back to the requirement: you have cloned or snapshotted disk images that have duplicate uuids.  For whatever reason you were not and perhaps are still not able to change them.  Normally these are only used by guests and the host doesn't see them because they are (or should be) filtered out, so there is no problem.

But occasionally you *do* want to be able to activate these LVs and at the moment this fails in all sorts of ways because of the duplicate UUIDs.

Forgetting implementation details for now, does that sum up the requirement?  You need a way to activate concurrently two LVs that *you* know are different but which LVM thinks are the same LV because the UUIDs match?

It really is necessary to understand exactly what you're trying to achieve and the reasoning behind it.

Several suggestions have been made in the past, but with some ingenuity people managed to do what they needed with the existing tools.

I would start by asking: what is preventing you from making the UUIDs unique *before* the devices concerned become active?  What is preventing you from using the existing facilities vgimportclone/pvchange/vgchange/vgrename/lvrename for that?

Once we understand better *why* the existing tools aren't going to work for you, then we can move on to discussing what changes might be possible.

To give you an idea of the sorts of things discussed before, while LVM can't change a uuid in use, it could queue up a request to change the uuid next time the object concerned is inactive, and meanwhile *pretend* it had changed it when handling input and output.

While LVM can't activate an LV with the same VG uuid + LV uuid as one that is already active, it could temporarily append something to the uuid when activating it (without changing the one on disk) to make it unique so it could be activated concurrently temporarily.

Neither of those proposals would be easy or quick to implement.

Okay let me see... =)

with respect to comment 8:
Let me point out again, that this might not only happen in the VM host/guest scenario... even though this is IMHO the most common case as cloning is just so simple there.
But it could also be that one simply dd images and for whatever reason these end up in the same running system (perhaps even accidentally)...
Or you use network block devices like nbd or drbd, and get PVs (and also VGs) of the same names and/or UUIDs into one running system.

AFAIU the following happens now:
- if any of such colliding PVs/VGs were already active/used before the colliding ones appear,... then they continue to work (in the sense you can still use your filesystems or whatever on top of them, and the files in /dev like /dev/vg_foo or /dev/mapper/..) will continue to point to the already active LVs, right?
- but the lvm tools,... might actually use any other (i.e. not the already active) PVs/VGs/LVs, as we're discussing in bug #979121.
- regardless of whether one set of the colliding PVs/VGs/LVs is already active/used or not... LVM won't let me activate them until the conflict is resolved
True so far?

So when I'm already at the point where I have PV/VG UUID/name collisions... the active ones will continue to work and I won't be able to activate the others anyway... so now I actually can change the PV/VG UUID/name collisions, of course on the 2nd set only, as this one is inactive and then I first pvchange --uuid on the respective device files, then vgimportclone on these device files (with new name and - automatically - new UUID).
True so far?


So things work basically already,... but I think it would be better for sysadmins if they were able to pro-actively avoid such a situation at all.
having duplicate PV UUIDs and VG UUID sound like a good start for really messing things up by accident (especially when things are scripted, which is often the case in automatic deployments of VMs)... not sure whether conflicting VG names could be a problem as well or whether that simply means "things stop working but nothing could ever get corrupted"...

Pro-actively avoiding such collisions means: After cloning: change at least PV and VG UUIDs and - if one already expects that one will use the different VGs (of same name) together - the VG names.

Easily doable for all the VGs one can deactivate, but a) not easily doable for those where the root fs runs from and b) possibly undesirable because one may need to stop services which are already running and using (non-system) VGs.

So this is where it all comes down to:
- Being able to change the PV/VG UUIDs while they're used.
- Ideally being able to do the same for the VG/LV names... and yes,.. for the LV names it's not necessary in terms of collision prevention (because I guess the LV names are unique only per VG)... but it would simply be nice to be able doing so... and when it would already be implemented for VGs... it couldn't be that difficult to do it for LVs as well.


with respect to comment 9:
I hope the above made it clear... in short the main motivation is pro-actively avoiding any possible UUID/name collisions, which could in the worst case cause data corruption[0].
And in order to make this easy, allowing the UUID (and ideally also name) changes while things are active/used

With respect to your question about why it can't be made in advance:
I could name different reasons:
- people forget it
- when you clone a VM while it's running... you'd have to stop it and again boot from some other medium (rescue ISO image or such)
- even if you change it before using the VM... you need to boot from some other rescue image (otherwise you again cannot change the stuff, as it would be actively used already)


But of course,... all this is a matter of "how easy is it to implement"... if it takes you enormous effort to make it possible, patching the DM or whatever... well then at some point one has to probably resign even it would be nice and help people.
I didn't expect by any means that this was so difficult (well I don't know the inner workings of LVM) and simply thought it would be similar as with filesystems and other containers where I can do this while they're mounted.


with respect to comment 10:
The queuing sounds like a good idea which would probably fulfil all of my motivation... since the changes would happen before the PVs/VGs could be used by/in another system (well because you have to shut the current one down before)
But I think few things would need to be secured/done:
a) It must be secured that the new UUIDs are written as soon as the active VGs get deactivated... and if they are not deactivated, they must be written before the kernel shuts down.
I remember that there are several situations where some container devices cannot be stopped by the kernel cleanly... like I have full root-encryption with dm-crypt... and when I stop I always see the errors that it couldn't close the LUKS device and the underlying VGs.[1]
b) How should tools like pvs/lvs/vgs [lv,pv,vg]display show notify about such a queued change? Should it show both current and next UUIDs/names in a separate field? etc. pp.
c) Does this anyhow touch clvm?
d) Possible issues with possibly existing 3rd party tools?

OTOH... if that's already so much effort... wouldn't it be possible to think about a way that makes chaning the UUIDs/names immediately possible? Could such changes in the DM be easier to do?

Sure.. if it's that complex.. but I guess time doesn't matter that much... people could somehow live without it so far... and even if I'd consider it a major improvement, one can probably live without it for a bit longer ;)

Cheers,
Chris.


[0] And this is not impossible... just imagine I have four hard disks
1: PV A for VG I
2: PV B for VG I
3: PV A for VG II
4: PV B for VG II
The PV As and PV Bs have each identical UUIDs... VG I and VG II as well
Now I guess LVM won't activate me any of these VGs, as long as it sees all PVs right?
But now my friendly coworker comes along, and removes two disks and takes by accident 2 and 3 instead of 3 and 4:
I will have now:
1: PV A for VG I
4: PV B for VG II
All PV and VG UUIDs and names fit... so I guess lvm can do nothing but activate the VGs... voilà... data corruption... 

[1] But IIRC; Milan Broz mentioned that the kernel would ensure by the DM's barriers that all data is written out correctly.

(In reply to Christoph Anton Mitterer from comment #11)
> Okay let me see... =)
> 

> AFAIU the following happens now:
> - if any of such colliding PVs/VGs were already active/used before the
> colliding ones appear,... then they continue to work (in the sense you can
> still use your filesystems or whatever on top of them, and the files in /dev
> like /dev/vg_foo or /dev/mapper/..) will continue to point to the already
> active LVs, right?
> - but the lvm tools,... might actually use any other (i.e. not the already
> active) PVs/VGs/LVs, as we're discussing in bug #979121.
> - regardless of whether one set of the colliding PVs/VGs/LVs is already
> active/used or not... LVM won't let me activate them until the conflict is
> resolved
> True so far?
 
> So when I'm already at the point where I have PV/VG UUID/name collisions...
> the active ones will continue to work and I won't be able to activate the
> others anyway... so now I actually can change the PV/VG UUID/name
> collisions, of course on the 2nd set only, as this one is inactive and then
> I first pvchange --uuid on the respective device files, then vgimportclone
> on these device files (with new name and - automatically - new UUID).
> True so far?
> 
> 
> So things work basically already,... but I think it would be better for
> sysadmins if they were able to pro-actively avoid such a situation at all.
> having duplicate PV UUIDs and VG UUID sound like a good start for really
> messing things up by accident (especially when things are scripted, which is
> often the case in automatic deployments of VMs)... not sure whether
> conflicting VG names could be a problem as well or whether that simply means
> "things stop working but nothing could ever get corrupted"...
> 
> Pro-actively avoiding such collisions means: After cloning: change at least
> PV and VG UUIDs and - if one already expects that one will use the different
> VGs (of same name) together - the VG names.
> 
> Easily doable for all the VGs one can deactivate, but a) not easily doable
> for those where the root fs runs from and b) possibly undesirable because
> one may need to stop services which are already running and using
> (non-system) VGs.

I think you already answered here yourself.

Sysadmin is supposed to ensure there are no UUID conflicts.

So if he does a cloning operation - he uses right tool for that.
If you misuse  'dd' tool - lvm2 cannot fix such user fault, but could help you to recover from this fault later.

And as you've already said - lvm will not allow to activate 2 devices with
same UUID - so I miss to see the problem you are still talking about,
that you have BOTH devices active.

If you have roorfs active - what's the problem to change UUID of the other
for this moment STILL inactive device ??

I just hope you are not talking about manipulation of metadata when this 'inactive' device is being active on another system?
(Which would be seriously wrong for many other reasons)

So why do you want to change UUID on your active rootfs - and not of inactive one?



> with respect to comment 9:
> I hope the above made it clear... in short the main motivation is
> pro-actively avoiding any possible UUID/name collisions, which could in the
> worst case cause data corruption[0].
> And in order to make this easy, allowing the UUID (and ideally also name)
> changes while things are active/used

Again - there is always one device inactive - and easy to change


> 
> With respect to your question about why it can't be made in advance:
> I could name different reasons:
> - people forget it
> - when you clone a VM while it's running... you'd have to stop it and again
> boot from some other medium (rescue ISO image or such)
> - even if you change it before using the VM... you need to boot from some
> other rescue image (otherwise you again cannot change the stuff, as it would
> be actively used already)

As already said - there is no reason for booting anything - tools can handle that with right options easily - what could be for discussion is
optional simplification here.


> But of course,... all this is a matter of "how easy is it to implement"...
> if it takes you enormous effort to make it possible, patching the DM or
> whatever... well then at some point one has to probably resign even it would
> be nice and help people.

Currently linux kernel/dm driver does not support live change of UUID,
so it's nothing lvm2 can change here.

And no - cryptsetup is not modifying live UUID (easy to check and test)
it only store changed UUID, so next activation of luks device has new UUID.
Unfortunately this is currently not easily doable for lvm - since clustered
support of such feature is somewhat harder - but as  Alasdair pointed out,
there is option to queue such change - but IMHO we still miss reason why it should be necessary.


> I didn't expect by any means that this was so difficult (well I don't know
> the inner workings of LVM) and simply thought it would be similar as with
> filesystems and other containers where I can do this while they're mounted.

The problem is - lvm metadata needs to match what is active in the system,
and if in the system is active device with UUID  'XXXXX' we cannot pretend,
that metadata believe it's actually 'YYYYY' - unless we deploy some queuing.

> But I think few things would need to be secured/done:
> a) It must be secured that the new UUIDs are written as soon as the active
> VGs get deactivated... and if they are not deactivated, they must be written
> before the kernel shuts down.

We have some different mechanism for this in lvm, where we are ensuring
consistency of all metadata and PVs.

> b) How should tools like pvs/lvs/vgs [lv,pv,vg]display show notify about
> such a queued change? Should it show both current and next UUIDs/names in a
> separate field? etc. pp.

Yep - lot's of work - and so far we miss the really good reason for it,
since it makes mainly recovery more and more complicated.

> c) Does this anyhow touch clvm?

Sure it does - and a lot.

> 
> [0] And this is not impossible... just imagine I have four hard disks
> 1: PV A for VG I
> 2: PV B for VG I
> 3: PV A for VG II
> 4: PV B for VG II
> The PV As and PV Bs have each identical UUIDs... VG I and VG II as well
> Now I guess LVM won't activate me any of these VGs, as long as it sees all
> PVs right?
> But now my friendly coworker comes along, and removes two disks and takes by
> accident 2 and 3 instead of 3 and 4:
> I will have now:
> 1: PV A for VG I
> 4: PV B for VG II
> All PV and VG UUIDs and names fit... so I guess lvm can do nothing but
> activate the VGs... voilà... data corruption... 

And how do you think the tool should know  1.) and 4.) is from different set, if the user hasn't changed UUID ??

I'm really confused here....

If you use 'dd' here (i.e. for data recovery from dying harddrive) you expect this functionality that you are able to replace  2.) with 4.)

If you plan to use drives independently you need to fix UUIDs after clone.

(In reply to Christoph Anton Mitterer from comment #7)
> > I'm missing here the point of booting rescue
> > image to change UUID of devices.
> The point is, as I wrote several times now, that you cannot change the UUIDs
> on PV/VGs (neither the names), which are active and used.
> E.g.:
> # pvchange --uuid /dev/sda1 
>   Volume group containing /dev/sda1 has active logical volumes
>   0 physical volumes changed / 1 physical volume not changed
> or:
> # vgchange --uuid vg_system
>   Volume group has active logical volumes
> 
> and if these are the ones on which your system runs on, you cannot easily
> make them inactive, unless you stop the system.

Hmm what do you mean here by  'you cannot easily make them inactive'
They have to be inactive before you want to make them active.

i.e. you run them in 'guest' - you inactivate them in guest,
and you activate them in 'host'.

As I said in comment 12 hopefully you are not trying to make them active at them
same time in guest and host.

> I guess we can continue to discuss this whole issue back and forth now
> forever...

Nope - surely not forever - but making things clear here is very important.


> What I wanted and what the original idea of this ticket was, is that it
> should be possible to change the UUIDs of PVs and VGs, and I guess (even if
> LV UUIDs are unique only within a VG) those of the LVs... and ideally also
> the names of VGs/LVs, while these are actively used.

Of course you could change the name of active LV and active VG,
just in case you have your LV already mounted with the old name,
it will appear to be mounted with this old name.

You just cannot change their UUID since this is currently kernel limitation.


> I tried to give you several different scenarios where it could happen that
> at least PV/VG UUIDs/names collide.

Well for all your scenarios there is existing solution how to fix it.

> The whole thing seems to be supported as well by all other containers on
> blockdevices I know of (filesystems, LUKS, haven't tried mdadm though)... so
> it seems not so uncommon that people get into situations where they want to
> change these while the devices are active.

BUT lvm supports change of the LV/VG name -  we do not support live change of lvm's INTERNAL uuid identifiers.


> As I've said,... that e.g. ext* supports this for both (UUID and LABEL(
> while the fs is mounted, shows that it's not so uncommon and quite useful.

OK - here is the big misunderstanding of lvm I guess.

UUID is not comparable to  i.e.  filesystem name.
Lvm  provides 'tags' - which are designed for the users - you could put there anything you need (limited in size and ascii7) - i.e. you could put there identifier for your database access.   LVM UUID on the other hand is fully under the control of LVM, and the way lvm uses this UUID in the system is 'private' to lvm - which means - we may introduce i.e. namespaces into device UUID - so the format may change in the future and users are not supposed to use these identifiers for anything persistent.

answer on comment 12:


On Tue, 2013-07-02 at 12:17 +0000, bugzilla wrote: 
> > Easily doable for all the VGs one can deactivate, but a) not easily doable
> > for those where the root fs runs from and b) possibly undesirable because
> > one may need to stop services which are already running and using
> > (non-system) VGs.
> I think you already answered here yourself.
What? o.O

> Sysadmin is supposed to ensure there are no UUID conflicts.
Well and that's why I ask for the whole feature...o.O


> So if he does a cloning operation - he uses right tool for that.
> If you misuse  'dd' tool - lvm2 cannot fix such user fault, but could help you
> to recover from this fault later.
Aha... well and you decide what's the right tool and which not?! ;-)
Guess dd have been used for cloning images since.... ever?

Anyway... coming back to the VM examples again,.. what should be the
right tool... or how should that work automatically? LVM can occur at
any of the blocklayers, so even if the VM management tools would
understand LVM (which they don't)... it wouldn't be easy if at all
possible.


> And as you've already said - lvm will not allow to activate 2 devices with
> same UUID - so I miss to see the problem you are still talking about,
> that you have BOTH devices active.
> 
> If you have roorfs active - what's the problem to change UUID of the other
> for this moment STILL inactive device ??
Well either we're misunderstanding each other ... or well I don't know
what else I should write ^^
Don't think it makes sense to continue that nitpicking and discussing
back and forth any possible detail (about which use cases/tools should
be allowed for users and which not), without really getting anything
new.

When you look at my last post, I told you the main motivation behind all
that: proactively preventing collisions... and I also gave one of the
many examples I could think of,... where such collisions could lead to
problems (I mean for that reasons you've made the UUIDs, haven't you)..


It's really as simple as that:
- People clone/dd images with LVM leading to UUID/name collisions... LVM
should handle that gracefully which it does already (well perhaps unless
that's what's being discussed in #979121).
- In order to prevent troubles, it should be easily possible to change
the UUIDs/names, before anything bad happesn
- "Easy" means I can also change it when the cloned LVM is running and
can't be deactivated (as the OS runs from it).



> I just hope you are not talking about manipulation of metadata when this
> 'inactive' device is being active on another system?
> (Which would be seriously wrong for many other reasons)
Not sure what you mean here exactly.


> So why do you want to change UUID on your active rootfs - and not of inactive
> one?
I wrote that already in my last post about at "With respect to your
question about why it can't be made in advance:"...
plus the argument that in order to make it inactive, I have too boot it
from some other system/rescue-CD/etc... which is always quite and effort
(especially when automating things).


> And no - cryptsetup is not modifying live UUID (easy to check and test)
> it only store changed UUID, so next activation of luks device has new UUID.
Well depends on what you look at... in DM it doesn't change it... but
all the cryptsetup commands (luksDump and friends) already show the new
one).
But you already said the most important thing yourself... it stores the
new one that will be used on the next activation... which already solve
all the issues cause any possible collision from a clone will be gone by
then.
And this is just the idea that Alasdair proposed.


> Unfortunately this is currently not easily doable for lvm - since clustered
> support of such feature is somewhat harder - but as  Alasdair pointed out,
> there is option to queue such change - but IMHO we still miss reason why it
> should be necessary.
Sorry... I wouldn't know how to explain it better... ^^ I thought my
example below with the 4 disks should have done it, but apparently not.


 
> > [0] And this is not impossible... just imagine I have four hard disks
> > 1: PV A for VG I
> > 2: PV B for VG I
> > 3: PV A for VG II
> > 4: PV B for VG II
> > The PV As and PV Bs have each identical UUIDs... VG I and VG II as well
> > Now I guess LVM won't activate me any of these VGs, as long as it sees all
> > PVs right?
> > But now my friendly coworker comes along, and removes two disks and takes by
> > accident 2 and 3 instead of 3 and 4:
> > I will have now:
> > 1: PV A for VG I
> > 4: PV B for VG II
> > All PV and VG UUIDs and names fit... so I guess lvm can do nothing but
> > activate the VGs... voilà... data corruption... 
> 
> And how do you think the tool should know  1.) and 4.) is from different set,
> if the user hasn't changed UUID ??
Not at all, of course... and this is just the point why I say... that
changing the UUIDs while the stuff is active would be nice...
Cause then it can be easily done pro-actively, (taking the VM example)
right after cloning the VM (from within the VM!!),... without the need
to stop it, without the need to mount the images from the stopped VM
either on the host or on some rescue USB stick in order to change it
there.

> If you plan to use drives independently you need to fix UUIDs after clone.
Yes... and this is what I want... pro-actively - i.e. before I use the
images on another computer, or in another guest or in another host =>
before anything could happen.
But... and that's the whole point... right now this doesn't work from
the (e.g.) running VM... or the (e.g.) running physical computer, which
I've equipped/booted with dd-cloned harddisks ... since the VGs are in
use then (at least the "system-VGs").
I would need to stop the VM or the phsyical computer... boot from it via
CD... or something like that... and change fix the UUIDs then.

answer on comment 13:


On Tue, 2013-07-02 at 12:46 +0000, bugzilla wrote: 
> Hmm what do you mean here by  'you cannot easily make them inactive'
possible reasons:
- I might not want to stop the VM (since it's perhaps already used
again).
- even if it stops... I may not have the rights to directly access the
virtual disk images (e.g. in our case access to they hypervisor is only
allowed for very few people... cloning though can be done by many... but
they cannot access (and fix the UUID) the physical from the host, since
tey have no access right...
So they'd need to boot the VM from e.g. some live CD (if they' have
rights to attach such CD).

I as the hypervisor admin in turn, may not be allowed (by policy - e.g.
privacy reasons) to access their VM images and do the change for them...
or I simply wouldn't want to


For cloned system images on physical machine it's even more
complicated...
We have big clusters which have exactly the same hardware config... it's
much faster to just dd a base image to their harddisks, than to a normal
installation.
network settings come from DHCP, udev net-rules are empty and created
the first time the node boots... same for ssh keys ... ext4 UUID can be
changed from the running system... LVM UUIDs not.

> They have to be inactive before you want to make them active.
No.. it's possible to clone running VMs...


> As I said in comment 12 hopefully you are not trying to make them active at
> them
> same time in guest and host.
Of course not! :)


> Of course you could change the name of active LV and active VG,
> just in case you have your LV already mounted with the old name,
> it will appear to be mounted with this old name.
He :) that works... yeah!
Haven't tried before because I always got stuck at changing the UUIDs at
first ;-)


> You just cannot change their UUID since this is currently kernel limitation.
Yeah I understood that now... so as Alasdair pointed out.. the only possible way was by queueing it.
Would the current kernel and LVM design allow that you simply do as
cryptsetup and overwrite the UUIDs on the blockdevice but not in the
kernel (and on the next start things automatically are changed)? I guess
not and you re-read these data from the on-disk metadata will in use?! 



> > As I've said,... that e.g. ext* supports this for both (UUID and LABEL(
> > while the fs is mounted, shows that it's not so uncommon and quite useful.
> 
> OK - here is the big misunderstanding of lvm I guess.
> 
> UUID is not comparable to  i.e.  filesystem name.
> Lvm  provides 'tags' - which are designed for the users - you could put there
> anything you need (limited in size and ascii7) - i.e. you could put there
> identifier for your database access.   LVM UUID on the other hand is fully
> under the control of LVM, and the way lvm uses this UUID in the system is
> 'private' to lvm - which means - we may introduce i.e. namespaces into device
> UUID - so the format may change in the future and users are not supposed to use
> these identifiers for anything persistent.
Sure it's clear that the UUIDs serve fully different purposes...
It's clear that you use them internally e.g. to select which PV belongs
to a VG... but as I tried to point out in the example with the 4
disks... it would be also good if one could pro-actively avoid UUID
collisions there.


Cheers,
Chris.

So now we're making progress:

  You've identified a "first boot" requirement, to allow new uuids to be set the first time a system is booted.

Sorry,.. that with the first boot was just an example on when one could do it... and obviously it makes sense to do it immediately after having the cloned machine (VM or not) started the first time... but I don't think (at least in my mind) that this changes a much.

Also a requirement to cope with clones of "running" VMs.

Hm? What do you mean?


Are the PV/VG headers mighty enough to support extensions?

Couldn't a simple working (queuing) solution be the following:
- Define a new field in the PV/VG headers with the semantics of "next UUID".
- When someone changes the PV/VG UUID now (while active) then internally the UUID stays the old one in all places), but the new one is written to the "next UUID" field.

- Whenever VGs (and therefore all its PVs) are deactivated (either locally or via clvm)
OR
- Whenever LVM looks for new (inactive) PVs/VGs
it looks at that field and:
- if all 0x0 does nothing and the current UUID stays
- if not all 0x0, the value is written as new UUID.


That way clvm shouldn't bother,... since nothing ever changes till a VG respectively PV is no longer used by anyone.
The only question missing would be, whether/how one outputs the fact that there's a new queued UUID in the displaying tools.

(In reply to Christoph Anton Mitterer from comment #14)


> On Tue, 2013-07-02 at 12:17 +0000, bugzilla wrote: 

> > And as you've already said - lvm will not allow to activate 2 devices with
> > same UUID - so I miss to see the problem you are still talking about,
> > that you have BOTH devices active.
> > 
> > If you have roorfs active - what's the problem to change UUID of the other
> > for this moment STILL inactive device ??
> Well either we're misunderstanding each other ... or well I don't know
> what else I should write ^^
> Don't think it makes sense to continue that nitpicking and discussing
> back and forth any possible detail (about which use cases/tools should
> be allowed for users and which not), without really getting anything
> new.

Please avoid these sentences in BZ in the first place.
It's not really helping with progress here...

> It's really as simple as that:
> - People clone/dd images with LVM leading to UUID/name collisions... LVM
> should handle that gracefully which it does already (well perhaps unless
> that's what's being discussed in #979121).
> - In order to prevent troubles, it should be easily possible to change
> the UUIDs/names, before anything bad happesn
> - "Easy" means I can also change it when the cloned LVM is running and
> can't be deactivated (as the OS runs from it).

So from what you write here I get impression you have few problems:

First problem:

You activate  cloned PVs in the virtual guest.
And you want to activate these devices at you host at the SAME time while they are still being in use in your guest ?

So you want have 'two' machines - which are able to write to the same metadata area at the same time - and your only 'synchronization' mechanism here is - that you as admin will guarantee the this collision will not happen ?

Am I right here ?


> > I just hope you are not talking about manipulation of metadata when this
> > 'inactive' device is being active on another system?
> > (Which would be seriously wrong for many other reasons)
> Not sure what you mean here exactly.
> 
> 
> > So why do you want to change UUID on your active rootfs - and not of inactive
> > one?
> I wrote that already in my last post about at "With respect to your
> question about why it can't be made in advance:"...
> plus the argument that in order to make it inactive, I have too boot it
> from some other system/rescue-CD/etc... which is always quite and effort
> (especially when automating things).

In comment 6 I've already shown you an easy way you could very easily
change UUID from host without any need of running rescue mode.

(Assuming we are still talking here about multiple guests using same UUID)

BTW for the automation the best starting point is to use vgimportclone after using 'dd' for image cloning - that's way much simpler.

 > > And no - cryptsetup is not modifying live UUID (easy to check and test)
> > it only store changed UUID, so next activation of luks device has new UUID.
> Well depends on what you look at... in DM it doesn't change it... but
> all the cryptsetup commands (luksDump and friends) already show the new
> one).
> But you already said the most important thing yourself... it stores the
> new one that will be used on the next activation... which already solve
> all the issues cause any possible collision from a clone will be gone by
> then.
> And this is just the idea that Alasdair proposed.

Yes - queuing of new UUID is one of options - but it's effectively equal to
running command after deactivation (except it's more automated), but it is  still in line with my explanation, that we can't replace running UUID with something else.

> > > [0] And this is not impossible... just imagine I have four hard disks
> > > 1: PV A for VG I
> > > 2: PV B for VG I
> > > 3: PV A for VG II
> > > 4: PV B for VG II
> > > The PV As and PV Bs have each identical UUIDs... VG I and VG II as well
> > > Now I guess LVM won't activate me any of these VGs, as long as it sees all
> > > PVs right?
> > > But now my friendly coworker comes along, and removes two disks and takes by
> > > accident 2 and 3 instead of 3 and 4:
> > > I will have now:
> > > 1: PV A for VG I
> > > 4: PV B for VG II
> > > All PV and VG UUIDs and names fit... so I guess lvm can do nothing but
> > > activate the VGs... voilà... data corruption... 
> > 
> > And how do you think the tool should know  1.) and 4.) is from different set,
> > if the user hasn't changed UUID ??
> Not at all, of course... and this is just the point why I say... that
> changing the UUIDs while the stuff is active would be nice...

So to recap here:

a.) You assume user has forget to  change UUIDs after cloning.
b.) He runs cloned disk on different system.
c.) He suddenly realizes he made a mistake and he want to make sure he will not run into problem when the VG is activated next time.
d.) While I'm pretty sure that this case rarely happens and >50% will realize
that made a mistake just after they mixed 1.) and 4.)

I'm getting impression - what you want here is with some vgchange command
that VG should internally store some flag/cookie/uuid or whatever - so it would recognize related set of PVs next time when all LVs are inactive would
fix everything.

Now - is that looking like solution for your problem ?

Technically we are now solving slightly 'similar' issue with snapshot merge.


> > If you plan to use drives independently you need to fix UUIDs after clone.
> Yes... and this is what I want... pro-actively - i.e. before I use the
> images on another computer, or in another guest or in another host =>
> before anything could happen.
> But... and that's the whole point... right now this doesn't work from
> the (e.g.) running VM... or the (e.g.) running physical computer, which
> I've equipped/booted with dd-cloned harddisks ... since the VGs are in
> use then (at least the "system-VGs").
> I would need to stop the VM or the phsyical computer... boot from it via
> CD... or something like that... and change fix the UUIDs then.

Well for now your limitation is - you must not forget after stopping
the VM to run proper command to fix UUID on host.

Also if you want to make the change in the  VM machine itself it's a bit more trickier - you could make this (PV/VG UUID) change through 'dracut' boot options (so it doesn't need extra rescue boot image) - but since you mention you are Debian user, it could be that its initramfs is not capable to make this operation ?

(I'm not saying it's user comfortable option here - but since you've mentioned automation - at dracut level it could be handled for this particular case)

(In reply to Christoph Anton Mitterer from comment #15)
> answer on comment 13:
> 
> 
> On Tue, 2013-07-02 at 12:46 +0000, bugzilla wrote: 
> > Hmm what do you mean here by  'you cannot easily make them inactive'
> possible reasons:
> - I might not want to stop the VM (since it's perhaps already used
> again).
> - even if it stops... I may not have the rights to directly access the
> virtual disk images (e.g. in our case access to they hypervisor is only
> allowed for very few people... cloning though can be done by many... but
> they cannot access (and fix the UUID) the physical from the host, since
> tey have no access right...
> So they'd need to boot the VM from e.g. some live CD (if they' have
> rights to attach such CD).
> 
> I as the hypervisor admin in turn, may not be allowed (by policy - e.g.
> privacy reasons) to access their VM images and do the change for them...
> or I simply wouldn't want to
> 

Ok - now it's answering some of my question from my comment 20.
Entries are getting somewhat long here ;)

> network settings come from DHCP, udev net-rules are empty and created
> the first time the node boots... same for ssh keys ... ext4 UUID can be
> changed from the running system... LVM UUIDs not.

Ever tried to change/resize partition table your system is running from ?
While I'm not a kernel developer here - some things are simply very deeply wired in, so there are some major differences in block layer and vfs layer.
  
> 
> > As I said in comment 12 hopefully you are not trying to make them active at
> > them
> > same time in guest and host.
> Of course not! :)

Ok good.


> > Of course you could change the name of active LV and active VG,
> > just in case you have your LV already mounted with the old name,
> > it will appear to be mounted with this old name.
> He :) that works... yeah!
> Haven't tried before because I always got stuck at changing the UUIDs at
> first ;-)

Ok at least one minor plus point we have ;)



> > You just cannot change their UUID since this is currently kernel limitation.
> Yeah I understood that now... so as Alasdair pointed out.. the only possible
> way was by queueing it.
> Would the current kernel and LVM design allow that you simply do as
> cryptsetup and overwrite the UUIDs on the blockdevice but not in the
> kernel (and on the next start things automatically are changed)? I guess
> not and you re-read these data from the on-disk metadata will in use?! 

Well you are forgetting lvm recovery which will scale complexity
and will need major rework - that's complex...

But I guess good starting point is - that on 'next' activation of marked
VG you want to regenerate new UUIDs.  The only issue is - how to ensure
UUIDs uniqueness - while randomness is good start - it's not for 100%

Hey.

Sorry for the delay... was/am a bit busy these days:


On Tue, 2013-07-02 at 19:24 +0000, bugzilla wrote:
Please avoid these sentences in BZ in the first place.
> It's not really helping with progress here...
> Well it's just that I really don't know how to better explain what I mean... and that I had the feeling we're turning in circles...
That doesn't mean that it was intended offensive or so :)


You activate  cloned PVs in the virtual guest.
> And you want to activate these devices at you host at the SAME time while they
> are still being in use in your guest ?
> No... I do NOT want to activate the same PVs at the same time in different "computers" (neither physical nor virtual ones)... I mean does this even work (without clvm)!? And I'd expect it would cause data corruption as soon as one writes.

So you want have 'two' machines - which are able to write to the same metadata
> area at the same time - and your only 'synchronization' mechanism here is -
> that you as admin will guarantee the this collision will not happen ?
> Am I right here ?
> No... absolutely nothing like that.


In comment 6 I've already shown you an easy way you could very easily
> change UUID from host without any need of running rescue mode.
> Just running vgchange --uuid someActiveVG doesn't work...
What's that --config 'devices{filter=["a|dev/vg/host12345|","r|.*|"]}'  ?
The whole --config option seems to be no where documented... and sounds a bit scary ;)

(Assuming we are still talking here about multiple guests using same UUID)
> Not when you mean that they access the same (identical) PVs (that also happen to have the same UUID - due do being clones).

BTW for the automation the best starting point is to use vgimportclone after
> using 'dd' for image cloning - that's way much simpler.
> Well but as told previously, this doesn't really help with the whole idea:
The idea is that you can easily change the UUIDs before you ever come into a situation where one system sees PVs/VGs with the same UUIDs, and people perhaps forget to use vgimportclone... or in some other way mess things up.
When you change the UUIDs from within the clone, that you can be sure that everything's fine (assuming you didn't clone a system that already contained collisions).


Yes - queuing of new UUID is one of options - but it's effectively equal to
> running command after deactivation (except it's more automated)
> Sure... but now you say it... you have to deactivate it first...

but it is 
> still in line with my explanation, that we can't replace running UUID with
> something else.
> Well it's enough if it will be certainly be updated then, the next time it get's used/activated (e.g. after stopping the guest and when running pvscan/vgscan on the host with the virtual disk images being visible to it).

a.) You assume user has forget to  change UUIDs after cloning.
> Either that, yes,... or he cannot change it right now, because the clone needs to be kept running... or some other reasons (like he has no rescue CD which he could boot from and change the UUIDs.

b.) He runs cloned disk on different system.
> Yes... that could be via moving a VM virtual disk into another guest (which already contains colliding ones)... or it could be by making sever colliding ones available in the host...or in case of physical machines it could be by simply moving the HDDs/SSDs to another machine, where there are already PVs/VGs with colliding UUIDs.

c.) He suddenly realizes he made a mistake and he want to make sure he will not
> run into problem when the VG is activated next time.
> Well that point should happen somewhere before (b) ideally... e.g. I clone the system... and remember... "a dammit... it could cause troubles when I move the disks... let me just change the UUIDs to new random ones in order to be safe".


d.) While I'm pretty sure that this case rarely happens and >50% will realize
> that made a mistake just after they mixed 1.) and 4.)
> To be honest... it's the case for all our VMs... (and there are really a lot)... we have to base images (one Debian, one RHEL) where all others build upon... when we deploy a new VM... that disk is cloned, booted and then things like ssh keys/etc. are adapted... I still found no way to change the UUIDs from within the running system... so either I'd need to boot the machine first from some ISO image and change it there... or I leave it as it is; and since the former is time-consuming and I simply hope that nobody that everyone remembers the thing about collisions ... I take the later option.


I'm getting impression - what you want here is with some vgchange command
> that VG should internally store some flag/cookie/uuid or whatever - so it would
> recognize related set of PVs next time when all LVs are inactive would
> fix everything.
> I'm not sure what you mean...

I guess it's best if you perhaps have a chat with Alasdair... I had the impression he exactly understood what I meant...


Also if you want to make the change in the  VM machine itself it's a bit more
> trickier - you could make this (PV/VG UUID) change through 'dracut' boot
> options (so it doesn't need extra rescue boot image) - but since you mention
> you are Debian user, it could be that its initramfs is not capable to make this
> operation ?
> No... should be easily possible there as well... (and actually we use both Deb and RHEL, at work actually mostly RHEL ;) )...
In principle that should be possible,... but it also requires a restart (okay sooner or later one needs to do this anyway)... and some ugly patching in the initramfs (I don't think the initramfs should be responsible for such things, to be honest)...


(I'm not saying it's user comfortable option here - but since you've mentioned
> automation - at dracut level it could be handled for this particular case)
> Yeah... you're right about that... it's not really a clean solution IMHO... but it should work...


Well... as I've said before... if even a queueing solution (in a way I've proposed before) would be very difficult to implement,... than I guess let's just close the bug and forget about it...
Of course there are ways to change the UUIDs... at least when the PVs/VGs are not yet used, which can be done either via some rescue-CD or from within the initramfs as you've mentioned.
Both seems a bit ugly to me... but again... if it's difficult to implement in another way, then it's perhaps the better choice.

On Tue, 2013-07-02 at 19:45 +0000, bugzilla wrote:
Ok - now it's answering some of my question from my comment 20.
> Entries are getting somewhat long here ;)
> Hehe ;) Yeah... sorry about all the confusion...


> network settings come from DHCP, udev net-rules are empty and created
> > the first time the node boots... same for ssh keys ... ext4 UUID can be
> > changed from the running system... LVM UUIDs not.
> Ever tried to change/resize partition table your system is running from ?
> While I'm not a kernel developer here - some things are simply very deeply
> wired in, so there are some major differences in block layer and vfs layer.
> Well things get better... but resizing is really a big change (for partitions)...
It changing e.g. partition types or the disk ID (MBR) and GPT ID seems to work (I think the kernel doesn't care that much about any of these,... the only case I know where it does is the 0xFD for auto raid assembly).



But I guess good starting point is - that on 'next' activation of marked
> VG you want to regenerate new UUIDs.
> I think you got my idea now :-D

  The only issue is - how to ensure
> UUIDs uniqueness - while randomness is good start - it's not for 100%
> You mean because of missing entropy after boot? (One would have the same, probably even worse, problems when doing the UUID change from within the initramfs or from a rescueCD)
But:
a) shouldn't UUID be mostly safe against this as it (hopefully) uses things like current time, PID, MAC addresses?
b) Why not already storing the value of the next UUID when the users says pv/vgchange --uuid someActiveVG...
The system should have some good entropy already and if still not... well than the same would happen when the user "now" creates a new PV/VG


Cheers,
Chris.