+++ This bug is a downstream clone. The original bug is: +++ +++ bug 1449968 +++ ====================================================================== Description of problem: Guest LVs created on raw volumes are auto activated on the hypervisor with FC storage The host still autoactivates everything at boot, LVM freaks out seeing all the guest VGs/LVs many of which have duplicated names/guids (because of cloned VMs). This breaks: 1. RHVH installation - anaconda fails on LVM errors if storage is connected to server 2. Upgrade - nodectl/imgbase used to fail on LVM errors, not sure if still fails or ignores the errors 3. VDSM - cannot correctly deactivate powered off VM disks, because guest LVs are still active on them 4. boot/shutdown issues as lvm services fail to start/stop correctly Version-Release number of selected component (if applicable): rhvh-4.1-0.20170421 How reproducible: Steps to Reproduce: 1.install host and add it to manager 2.add fc storage to datacenter 3.create vm on raw device with lvm on top of it 4.shutdown the vm, put host into maintenance 6, reboot the host Actual results: guest vg and lv are autoactivated on the boot Expected results: guest vg and lv are not autoactivated on the boot Additional info: (Originally by Marian Jankular)
There are several bugs here: 1) We use anaconda from platform, so I'd expect this bug to also be reproducible on RHEL. Can you please open a bug against Anacodna to track this? 2) nodectl/imgbase now ignores LVM errors. 3/4) These look like duplicates of rhbz#1374545, but that shipped with 4.1.1-1. Nir, can you please look at this? RHVH doesn't have any special/different handling for LVs, so I'd expect this to be vdsm/storage. (Originally by Ryan Barry)
(In reply to Ryan Barry from comment #1) > There are several bugs here: > > 1) We use anaconda from platform, so I'd expect this bug to also be > reproducible on RHEL. Can you please open a bug against Anacodna to track > this? I agree, issues with installing stuff on a host with connected FC storage is platform issue, we need a RHEL bug for this. > 2) nodectl/imgbase now ignores LVM errors. I don't know what are these errors but failing on lvm errors sounds reasonable, ignoring lvm errors is very risky. > 3/4) These look like duplicates of rhbz#1374545, but that shipped with > 4.1.1-1. Nir, can you please look at this? RHVH doesn't have any > special/different handling for LVs, so I'd expect this to be vdsm/storage. We handled only iSCSI issues so far, we have couple of bugs open for FC issues, see bug 1450114. (Originally by Nir Soffer)
Marian, I would like to focus on single issue in this bug. If you look in the tracker bug 1450114 we already have bugs about: - duplicate vg/lvs - bug 1130527 - slow boot - bug 1400446 - dirty luns not usable - bug 1253640 We lack a bug about thin_check breaking thin guest lvs with FC storage (see bug 1371939). Do you have a relevant case for this issue? If not, maybe the case you added is relevant to one of the other bugs? (Originally by Nir Soffer)
(In reply to Nir Soffer from comment #6) > (In reply to Ryan Barry from comment #1) > I agree, issues with installing stuff on a host with connected FC storage > is platform issue, we need a RHEL bug for this. I'll open a platform bug > I don't know what are these errors but failing on lvm errors sounds > reasonable, > ignoring lvm errors is very risky. Well, we're not strictly ignoring LVM errors. Simply that with the lvmetad changes, stderr throws out a lot of messages. We're dropping that to get a clean parse, not non-zero return code. > We handled only iSCSI issues so far, we have couple of bugs open for FC > issues, > see bug 1450114. Since this bug is probably not RHVH specific, do you want to grab this for vdsm/storage? (Originally by Ryan Barry)
(In reply to Ryan Barry from comment #8) > Since this bug is probably not RHVH specific, do you want to grab this for > vdsm/storage? Lets understand first what is the unique issue in this bug that is not tracked already in one of the other bugs. Waiting for Marian reply on comment 7. (Originally by Nir Soffer)
Hello Nir, i think none of the bugs in comment #7 is relevant for this one, guest lvs that are autoactivated are not direct luns assigned to the vms, they are from data storage domain and they are raw devices. (Originally by Marian Jankular)
Based on comment 10, moving this to vdsm. (Originally by Nir Soffer)
4.1.4 is planned as a minimal, fast, z-stream version to fix any open issues we may have in supporting the upcoming EL 7.4. Pushing out anything unrelated, although if there's a minimal/trival, SAFE fix that's ready on time, we can consider introducing it in 4.1.4. (Originally by Allon Mureinik)
Hello Nir, thank you for responding, appreciate that even more as you are on PTO. Would it be possible that we would add this as default setting to our rhvh hosts? I think it should not break anything as we are activating rhv storage domains with vdsm. Or at we should highlight this in the documentation. What do you think? (Originally by Marian Jankular)
I'm also on PTO, but from a RHVH perspective, we'd rather see this fixed from vdsm or platform, since this problem is not at all unique to RHVH. Highlighting this in the documentation would be ideal, though (Originally by Ryan Barry)
(In reply to Marian Jankular from comment #16) > Hello Nir, > > thank you for responding, appreciate that even more as you are on PTO. > Would it be possible that we would add this as default setting to our rhvh > hosts? I think it should not break anything as we are activating rhv storage > domains with vdsm. Or at we should highlight this in the documentation. > > What do you think? There's no magic solution here. The current solution is manual configuration. A better, longer term solution would be a script (Ansible?) that creates this LVM filter and keeps it up-to-date. (Originally by Yaniv Kaul)
*** Bug 1507691 has been marked as a duplicate of this bug. *** (Originally by Daniel Erez)
Current state: Vdsm includes now a new vdsm-tool configure-lvm-filter command. This command analyze the current host and recommends the lvm filter for this host. Here is an example run on this command: # vdsm-tool config-lvm-filter Found these mounted logical volumes on this host: logical volume: /dev/mapper/vg0-lv_home mountpoint: /home devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_root mountpoint: / devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_swap mountpoint: [SWAP] devices: /dev/vda2 This is the recommended LVM filter for this host: filter = [ "a|^/dev/vda2$|", "r|.*|" ] To use this LVM filter please edit /etc/lvm/lvm.conf and set the 'filter' option in the 'devices' section. It is recommended to reboot the hypervisor to verify the filter. This filter will allow LVM to access the local devices used by the hypervisor, but not shared storage owned by Vdsm. If you want to add another local device you will have to add the device manually to the LVM filter. The next step is analyzing the currently configured lvm filter, and configuring the lvm filter when a user confirm. We hope to perform this configuation automatically as part of host deployment. (Originally by Nir Soffer)
I posted 2 additional patches, improving this configuration tool. `vdsm-tool config-lvm-filter` analyzes the current LVM configuation to decide if a filter should to be configured, and to configure the filter automatically if possible. If the host is already configured, the tool does nothing: # vdsm-tool config-lvm-filter Analyzing host... LVM filter is already configured for Vdsm On a host not configured yet, the tool configure LVM automatically after the user confirm the operation. # vdsm-tool config-lvm-filter Analyzing host... Found these mounted logical volumes on this host: logical volume: /dev/mapper/vg0-lv_home mountpoint: /home devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_root mountpoint: / devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_swap mountpoint: [SWAP] devices: /dev/vda2 This is the recommended LVM filter for this host: filter = [ "a|^/dev/vda2$|", "r|.*|" ] This filter will allow LVM to access the local devices used by the hypervisor, but not shared storage owned by Vdsm. If you add a new device to the volume group, you will need to edit the filter manually. Configure LVM filter? [yes,NO] ? [NO/yes] yes Configuration completed successfully! Please reboot to verify the LVM configuration. If the host configuration does not match Vdsm required configuration, the user need to configure LVM filter manually. # vdsm-tool config-lvm-filter Analyzing host... Found these mounted logical volumes on this host: logical volume: /dev/mapper/vg0-lv_home mountpoint: /home devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_root mountpoint: / devices: /dev/vda2 logical volume: /dev/mapper/vg0-lv_swap mountpoint: [SWAP] devices: /dev/vda2 This is the recommended LVM filter for this host: filter = [ "a|^/dev/vda2$|", "r|.*|" ] This filter will allow LVM to access the local devices used by the hypervisor, but not shared storage owned by Vdsm. If you add a new device to the volume group, you will need to edit the filter manually. This is the current LVM filter: filter = [ "a|^/dev/vda2$|", "a|^/dev/vdb1$|", "r|.*|" ] WARNING: The current LVM filter does not match the recommended filter, Vdsm cannot configure the filter automatically. Please edit /etc/lvm/lvm.conf and set the 'filter' option in the 'devices' section to the recommended value. It is recommend to reboot after changing LVM filter. This completes the work planned for 4.2.0. After additional testing, we can integrate this into host deployment, so in the common cases, LVM filter will be configured automatically. In some cases, when a user configured in compatible LVM filter the user will have to modify the configuration manually. (Originally by Nir Soffer)
2 Additional patches merged fixing issues we found in the last week. (Originally by Nir Soffer)
Tal/Nir, can you please explain why we need this clone?
And is "NEW" the correct status?
(In reply to Allon Mureinik from comment #24) > Tal/Nir, can you please explain why we need this clone? Need for backporting lvm filter patches for 4.1.9, as we discussed.
According to git log this bug may be included in 4.2.0. Can you please cross check?
(In reply to Sandro Bonazzola from comment #27) > According to git log this bug may be included in 4.2.0. Can you please cross > check? I already replied about this multiple times in the other related bugs. All the patches are included in 4.2.0, except 2 fixes for the new code introduce after 4.2.0 was built. So this will be available in 4.2.1.
WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops
WARN: Bug status (ON_QA) wasn't changed but the folowing should be fixed: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops: Bug status (ON_QA) wasn't changed but the folowing should be fixed: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops
BZ<2>Jira re-sync