Bug 895982 - LVMError: lvcreate failed for vg_system1/LogVol00: 07:19:23,331 ERROR : Volume group "vg_system1" has insufficient free space (3774 extents): 3775 required.
Summary: LVMError: lvcreate failed for vg_system1/LogVol00: 07:19:23,331 ERROR : V...
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: anaconda   
(Show other bugs)
Version: 6.4
Hardware: i686 Unspecified
low
unspecified
Target Milestone: rc
: ---
Assignee: David Lehman
QA Contact: Ľuboš Kardoš
URL:
Whiteboard: abrt_hash:acd9f4a70acffca58c9ab4ea354...
Keywords:
Depends On:
Blocks: 960065 841211
TreeView+ depends on / blocked
 
Reported: 2013-01-16 12:24 UTC by Ľuboš Kardoš
Modified: 2016-08-01 01:26 UTC (History)
5 users (show)

Fixed In Version: anaconda-13.21.202-1
Doc Type: Known Issue
Doc Text:
Physical-extents size less than 32MB on top of an MD physical volume leads to problems with calculating the capacity of a volume group. To work around this problem, use a physical-extent size of 32MB or leave space double the physical-extent size free when allocating logical volumes. Another option is to change the default 4MB size of a physical extent to 32MB.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-21 09:58:29 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
File: anaconda-tb-o6SI8e (432.84 KB, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: release (24 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: version (3 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: hashmarkername (8 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: type (9 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: exnFileName (23 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: product (24 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
File: environ (1019 bytes, text/plain)
2013-01-16 12:24 UTC, Ľuboš Kardoš
no flags Details
partitioning (79.82 KB, image/png)
2013-01-16 12:29 UTC, Ľuboš Kardoš
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:1588 normal SHIPPED_LIVE anaconda bug fix and enhancement update 2013-11-21 00:40:22 UTC

Description Ľuboš Kardoš 2013-01-16 12:24:28 UTC
Version-Release number of selected component:
anaconda-13.21.193

Additional info:
libreport version: 2.0.9
cmdline:        /usr/bin/python  /usr/bin/anaconda
kernel:         2.6.32-353.el6.i686

description:
:The following was filed automatically by anaconda:
:anaconda 13.21.193 exception report
:Traceback (most recent call first):
:  File "/usr/lib/anaconda/storage/devicelibs/lvm.py", line 382, in lvcreate
:    raise LVMError("lvcreate failed for %s/%s: %s" % (vg_name, lv_name, msg))
:  File "/usr/lib/anaconda/storage/devices.py", line 2562, in create
:    lvm.lvcreate(self.vg.name, self._name, self.size, progress=w)
:  File "/usr/lib/anaconda/storage/deviceaction.py", line 204, in execute
:    self.device.create(intf=intf)
:  File "/usr/lib/anaconda/storage/devicetree.py", line 713, in processActions
:    action.execute(intf=self.intf)
:  File "/usr/lib/anaconda/storage/__init__.py", line 355, in doIt
:    self.devicetree.processActions()
:  File "/usr/lib/anaconda/packages.py", line 110, in turnOnFilesystems
:    anaconda.id.storage.doIt()
:  File "/usr/lib/anaconda/dispatch.py", line 210, in moveStep
:    rc = stepFunc(self.anaconda)
:  File "/usr/lib/anaconda/dispatch.py", line 126, in gotoNext
:    self.moveStep()
:  File "/usr/lib/anaconda/gui.py", line 1388, in nextClicked
:    self.anaconda.dispatch.gotoNext()
:LVMError: lvcreate failed for vg_system1/LogVol00: 07:19:23,331 ERROR   :   Volume group "vg_system1" has insufficient free space (3774 extents): 3775 required.

Comment 1 Ľuboš Kardoš 2013-01-16 12:24:34 UTC
Created attachment 679507 [details]
File: anaconda-tb-o6SI8e

Comment 2 Ľuboš Kardoš 2013-01-16 12:24:36 UTC
Created attachment 679508 [details]
File: release

Comment 3 Ľuboš Kardoš 2013-01-16 12:24:39 UTC
Created attachment 679509 [details]
File: version

Comment 4 Ľuboš Kardoš 2013-01-16 12:24:40 UTC
Created attachment 679510 [details]
File: hashmarkername

Comment 5 Ľuboš Kardoš 2013-01-16 12:24:44 UTC
Created attachment 679511 [details]
File: type

Comment 6 Ľuboš Kardoš 2013-01-16 12:24:47 UTC
Created attachment 679512 [details]
File: exnFileName

Comment 7 Ľuboš Kardoš 2013-01-16 12:24:48 UTC
Created attachment 679513 [details]
File: product

Comment 8 Ľuboš Kardoš 2013-01-16 12:24:50 UTC
Created attachment 679514 [details]
File: environ

Comment 9 Ľuboš Kardoš 2013-01-16 12:29:43 UTC
Created attachment 679516 [details]
partitioning

Comment 11 Ľuboš Kardoš 2013-01-16 12:49:15 UTC
Steps to reproduce
1. Strat graphical installation.
2. Proceed to the partitioning screen.
3. Create partitioning like in attached "partitioning.png" image.
4. Click next.

Actual results:
Traceback is showed.

Expected results:
Anaconda should successfully creates partitions and then it should show package selection screen. 

Additional info:
The same problem is in bug 876450 but in my case I am not able to reproduce it by kickstart. I am able to reproduce this problem only in graphical installation.

Because bug 876450 was blocker I propose this bug as blocker too.

Comment 12 David Lehman 2013-01-19 21:21:34 UTC
(In reply to comment #11)
> Additional info:
> The same problem is in bug 876450 but in my case I am not able to reproduce
> it by kickstart. I am able to reproduce this problem only in graphical
> installation.

When you tried to reproduce with kickstart how did you define the volgroup? Did you specify a pesize?

Comment 13 David Lehman 2013-01-19 21:35:18 UTC
Perhaps more interesting: Do you still hit the error if you set the volume group's physical extent size to 32MB in the graphical install?

Comment 14 Ľuboš Kardoš 2013-01-21 09:02:50 UTC
When I tried to reproduce with kickstart I didn't specify a pesize.
I don't hit the error if I set the volume group's physical extent size to 32MB in the graphical install.

Comment 16 Brian Lane 2013-01-23 01:10:40 UTC
I'm trying to track down exactly what's going on here, but I don't think this is a blocker. Using a larger PE works around it. I think this is an interaction between partition size and PE size and isn't new.

Comment 17 David Lehman 2013-01-23 15:00:12 UTC
I also think this has been a bug since earlier releases. I verified that the bug exists in a RHEL6 tree containing anaconda-13.21.181-1.

We have the option of just documenting the issue: PE size of <32MB on top of MD PVs leads to problems calculating VG capacity. Workarounds include using 32MB PE size or making sure to leave 2*pesize free when allocating LVs.

Another option is to change the current default of 4MB pesize to 32MB.

Comment 21 Samantha N. Bueno 2013-05-21 15:18:48 UTC
Setting this to DevCond NAK capacity. This is definitely an issue that should be addressed, but there is a documented work-around to bypass it. If there is time at the end of the devel window, we will certainly make the effort to fix this.

Comment 22 Charlie Brady 2013-07-04 15:44:54 UTC
> I also think this has been a bug since earlier releases. I verified that the 
> bug exists in a RHEL6 tree containing anaconda-13.21.181-1.

Did you try 13.21.176? Something has changed since RHEL6.3.

We are using a custom setDefaultPartitioning, and have lvm over raid1, which worked fine for RHEL < 6.4, but now hits this bug, which looks like it might be a rounding issue. We have '/' and swap sharing a LV, with '/' of 2G but growable, and swap sized by:

        (minswap, maxswap) = iutil.swapSuggestion()
        autorequests.append(PartSpec(fstype="swap", size=minswap, maxSize=maxswap, grow=True, asVol=True))

Comment 23 Charlie Brady 2013-07-04 15:54:32 UTC
> I also think this has been a bug since earlier releases. I verified that the 
> bug exists in a RHEL6 tree containing anaconda-13.21.181-1.

Did you try 13.21.176? Something has changed since RHEL6.3.

We are using a custom setDefaultPartitioning, and have lvm over raid1, which worked fine for RHEL < 6.4, but now hits this bug, which looks like it might be a rounding issue. We have '/' and swap sharing a LV, with '/' of 2G but growable, and swap sized by:

        (minswap, maxswap) = iutil.swapSuggestion()
        autorequests.append(PartSpec(fstype="swap", size=minswap, maxSize=maxswap, grow=True, asVol=True))


I notice that anaconda 13.21.195 introduces variable superBlockSize. Is the algorithm off-by-one, or has a rounding error, or otherwise wrong?

Comment 24 Charlie Brady 2013-07-04 18:15:30 UTC
With my failed install using a RHEL6.4 based system, anaconda creates lv_root using '-L 269736m"  (67434 * 4) and then failes to create lv_swap using "-L 16104m" (4026 * 4). pvdisplay shows:

PV size 279.14 GiB/not usable 4.81MiB
PE Size 4.00 MiB
Total PE   71459
Free PE  4025
Allocated PE 67434

With a successful RHEL6.3 based system, anaconda creates lv_root using "-L 27836m" (69591 * 4) and lv_swap using "-L 7600m" (1900 * 4). pvdisplay shows:

...
PV Size 279.27 GiB/not usable 3.87MiB
Total BE 71491
...

So it looks to me as though anaconda is assuming there are 71460 PEs available. Could "not usable" being > 4MiB be the factor here? Where does anaconda estimate/evaluage this "not usable" quantity?

Comment 25 Charlie Brady 2013-07-04 20:10:32 UTC
s/evaluage/evaluate/

Comment 26 Charlie Brady 2013-07-08 19:08:33 UTC
>  Workarounds include using 32MB PE size or making sure to leave
> 2*pesize free when allocating LVs.

In my case changing PE size from 4MB to 32MB seems sufficient on my hardware. Presumably the underlying bug is still there, however, so it would be good to have a proper diagnosis and proper fix.

Comment 27 David Lehman 2013-07-17 16:15:21 UTC
I have a patch that could use some testing. If people tell me what version of anaconda want to test with I can provide updates image containing the patch to enable you to try it.

Comment 28 David Lehman 2013-07-17 16:19:57 UTC
Undoing unintentional changes to flags, &c from previous update.

Comment 29 Charlie Brady 2013-07-17 17:01:57 UTC
I am using 13.21.195-1. A patch would be easier for me to use than an updates img (since I already make my own updates img).

Comment 30 David Lehman 2013-07-17 21:07:00 UTC
(In reply to Charlie Brady from comment #29)
> I am using 13.21.195-1. A patch would be easier for me to use than an
> updates img (since I already make my own updates img).

http://dlehman.fedorapeople.org/20130717-lvm-padding-895982.1.patch

Let me know how it goes.

Comment 31 Charlie Brady 2013-07-17 21:13:43 UTC
(In reply to David Lehman from comment #30)

> Let me know how it goes.

Wouldn't be for a week, I'm about to head off on vacation.

Comment 32 Charlie Brady 2013-07-17 21:20:53 UTC
I'd appreciate it if you could explain what you think is going wrong. 

--- a/storage/devices.py
+++ b/storage/devices.py
@@ -2249,6 +2250,11 @@ class LVMVolumeGroupDevice(DMDevice):
         used = sum(lv.vgSpaceUsed for lv in self.lvs) + self.snapshotSpace
         used += self.reservedSpace
         free = self.size - used
+        
+        pad = self.peSize * 2 * len(self.pvs)
+        if free >= pad:
+            free -= pad
+
         log.debug("vg %s has %dMB free" % (self.name, free))
         return free

Isn't the problem here that 'used' is being underestimated? In that case, shouldn't something just be added to used? or reservedSpace be corrected?

Comment 33 David Lehman 2013-07-18 13:54:24 UTC
It looks like md is using more space for metadata than we expect. Rather than try to match up the code in anaconda to match md's behavior exactly, only to have md (or lvm) change again in a few more weeks, I am just going to pad out the lvm calculations so there is some buffer.

Comment 34 Ľuboš Kardoš 2013-07-18 14:26:20 UTC
I tested patch from comment 30 and I can confirm that it fixes problem described in comment 11.

Comment 36 Ľuboš Kardoš 2013-10-18 07:53:28 UTC
Verified on  anaconda-13.21.211-1 (RHEL6.5-20131009.0)

Comment 38 errata-xmlrpc 2013-11-21 09:58:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1588.html

Comment 39 Charlie Brady 2014-01-03 21:49:10 UTC
The patch from comment #30 is not included in anaconda-13.21.215, which was used in RHEL6.5. Neither is the change in default peSize from 4.0 to 32.0 included.

How do we find out what code was changed in the errata?

* Mon Aug 12 2013 Samantha N. Bueno <sbueno+anaconda@redhat.com> - 13.21.202-1
- Add support for preexisting lvm using raid segment types. (dlehman)
  Resolves: rhbz#873281
- Add a bit of padding for metadata in new md size estimates. (dlehman)
  Resolves: rhbz#895982
- Expand warning text for package install issues. (sbueno+anaconda)
  Resolves: rhbz#895098
- Don't filter partitions on mpath devices in lvm (sbueno+anaconda)
  Resolves: rhbz#885755
- Move "nousbstorage" and "nousb" handling into init.c. (clumens)
  Resolves: rhbz#947704

Comment 40 Charlie Brady 2014-01-03 21:52:54 UTC
I can answer my own question:

https://lists.fedorahosted.org/pipermail/anaconda-patches/2013-August/005157.html

diff --git a/storage/devices.py b/storage/devices.py
index 42e3b6f..02d73ee 100644
--- a/storage/devices.py
+++ b/storage/devices.py
@@ -2789,7 +2789,9 @@ class MDRaidArrayDevice(StorageDevice):
             elif self.level == mdraid.RAID10:
                 size = (self.memberDevices / 2.0) * smallestMemberSize
                 size -= size % self.chunkSize
-            log.debug("non-existant RAID %s size == %s" % (self.level, size))
+
+            size -= 1   # account for unexpected metadata
+            log.debug("non-existent RAID %s size == %s" % (self.level, size))
         else:
             size = self.partedDevice.getSize()
             log.debug("existing RAID %s size == %s" % (self.level, size))

Comment 41 joshua.kugler@intel.com 2014-11-14 00:40:48 UTC
I am hitting this exact same bug in Redhat 6.6, but in a really odd failure mode.

I have this line in the kickstart:

logvol /export/backups --fstype=ext4 --name=lv_backup --vgname=vg_01 --size=4096 --grow.

That works great, works fine.  but, if I change --fstype to xfs, I get this error in the install:

lvcreate failed for vg_01/lv_backup: 15:31:17,673 ERROR Volume group "vg_01" has insufficient free space (2563336 extents): 2563337 required.

Is this related to the above bug? Or should I open a new bug?

Comment 42 Charlie Brady 2014-11-19 23:42:26 UTC
(In reply to joshua.kugler@intel.com from comment #41)

> I am hitting this exact same bug in Redhat 6.6, ...
...
> Is this related to the above bug? Or should I open a new bug?

It might be related to this bug, but you should open a new bug, since you can't re-open this one, and it might be a different root cause.


Note You need to log in before you can comment on or make changes to this bug.