Description of problem: Vdsm overrides lvm configuration for vdsm lvm calls. We are using obtain_device_list_from_udev=0 which was a workaround for old udev bugs in EL 6. The udev issues should be fixed in Fedora 21 and EL 7. Test vdsm without obtain_device_list_from_udev=0 and remove this option if it works now. The advantage should be faster operation as lvm does not have to search /dev for block devices but get the devices from udev. See https://bugzilla.redhat.com/show_bug.cgi?id=1259587#c11
We started using obtain_device_list_from_udev=0 here: commit 23ce1d87fa98f35d83005e2958ff4065b78ff9d8 Author: Nir Soffer <nsoffer> Date: Mon Nov 4 22:08:45 2013 +0200 lvm: Do not use udev cache for obtaining device list lvm is obtaining the device list from udev. When using concurrently, udev sometimes returns incomplete list, which cause lvm to think that a vg is missing, and the command may fail with "Volume group not found" error, or "Cannot change VG test while PVs are missing". lvm fixed this issue in version 2.02.100-7.el6 by disabling udev cache, setting obtain_device_list_from_udev to 0. Unfortunatlly, lvm fix is not enough for vdsm, as the fix is applied only if no lvm.conf file exists. When upgrading existing lvm installation, lvm creates a lvm.conf.rpmnew file, and the system administrator is responsible for updating lvm configuration. This patch disable udev cache using the --config option, used to override many other lvm options, ensuring proper configuration on both new and upgraded systems. Disabling udev cache may have minimal performance effect according to lvm developers. Change-Id: Ib55c8d444f3be9f63bfd23d8def60607b5b3dff0 Bug-Url: https://bugzilla.redhat.com/1014942 Signed-off-by: Nir Soffer <nsoffer> Reviewed-on: http://gerrit.ovirt.org/20890 Reviewed-by: Dan Kenigsberg <danken> Reviewed-by: Ayal Baron <abaron> Reviewed-by: Allon Mureinik <amureini>
The title says "Use obtain_device_list_from_udev=1", but comment 0 says: > Test vdsm without obtain_device_list_from_udev=0 and remove this > option if it works now. So, just to be clear - the plan is to remove this parameter and let lvm use its own default, right?
(In reply to Allon Mureinik from comment #2) > The title says "Use obtain_device_list_from_udev=1", but comment 0 says: > > > Test vdsm without obtain_device_list_from_udev=0 and remove this > > option if it works now. > > So, just to be clear - the plan is to remove this parameter and let lvm use > its own default, right? Yes.
The plan is to remove the parameter "obtain_device_list_from_udev"
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Peter, I need to add to my patch information about the udev bugs that have been solved and on which version the fixes are available. Can you provide this info ? Thanks, Fred
(In reply to Fred Rolland from comment #6) > Peter, > > I need to add to my patch information about the udev bugs that have been > solved and on which version the fixes are available. > Well, I don't remember the exact details of the bug, but there was a bugzilla report filed for that with the note that the changes won't be backported to RHEL6. I'll try to look that bug up...
*** Bug 1272026 has been marked as a duplicate of this bug. ***
Fred, would you explain how to verify this?
Created attachment 1086520 [details] Test script Python code for test concurrent creation of LVs
In order to test I used the attached code that creates LVs in different threads. In order to use this, you need to create a ISCSI Storage Domain in engine with at least 2 LUNs. Edit the test.py by updating the VG and LUNs IDs from your setup. i - number of iterations c - number of threads python test.py -i 100 -c 30
(In reply to Peter Rajnoha from comment #7) > (In reply to Fred Rolland from comment #6) > > Peter, > > > > I need to add to my patch information about the udev bugs that have been > > solved and on which version the fixes are available. > > > > Well, I don't remember the exact details of the bug, but there was a > bugzilla report filed for that with the note that the changes won't be > backported to RHEL6. I'll try to look that bug up... Sorry, I haven't found the exact bug in the end - it's been a while since then...
In oVirt testing is done on single release by default. Therefore I'm removing the 4.0 flag. If you think this bug must be tested in 4.0 as well, please re-add the flag. Please note we might not have testing resources to handle the 4.0 clone.
Please set target release or I can't move the bug to ON_QA automatically.
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.
Tested concurrent creation of LVs using the code provided in comment #10. All operation succeeded, no error raised. 2016-01-18 12:23:21,733 INFO worker 'test-17': starting 2016-01-18 12:23:21,733 INFO creating '852aeaa3-27d9-4501-9e0e-582f46aa4c44/test-17-01' 2016-01-18 12:23:21,753 INFO worker 'test-18': starting 2016-01-18 12:23:21,753 INFO creating '852aeaa3-27d9-4501-9e0e-582f46aa4c44/test-18-01' 2016-01-18 12:23:21,758 INFO worker 'test-19': starting 2016-01-18 12:23:21,758 INFO creating '852aeaa3-27d9-4501-9e0e-582f46aa4c44/test-19-01' 2016-01-18 12:23:21,772 INFO worker 'test-20': starting Tested using: vdsm-4.17.17-0.el7ev.noarch lvm2-2.02.130-5.el7.x86_64