Bug 1200812 - Blivet depends on library that can't handle sizes >= 16 EiB
Summary: Blivet depends on library that can't handle sizes >= 16 EiB
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-blivet
Version: rawhide
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: mulhern
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker abrt_hash:5d84c0811b7...
: 1202444 1203324 (view as bug list)
Depends On:
Blocks: F22BetaBlocker
TreeView+ depends on / blocked
 
Reported: 2015-03-11 12:25 UTC by Jiri Konecny
Modified: 2015-03-23 08:15 UTC (History)
16 users (show)

Fixed In Version: anaconda-22.20.6-1.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-23 07:11:03 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
File: anaconda-tb (260.96 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: anaconda.log (9.47 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: dnf.log (6.70 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: dnf.rpm.log (147 bytes, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: environ (492 bytes, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: ks.cfg (934 bytes, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: lsblk_output (1.15 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: nmcli_dev_list (1.51 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: os_info (447 bytes, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: program.log (20.31 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: storage.log (54.73 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: syslog (78.72 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: ifcfg.log (1.59 KB, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details
File: packaging.log (866 bytes, text/plain)
2015-03-11 12:25 UTC, Jiri Konecny
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1201947 0 unspecified CLOSED fix the way libblockdev handles sizes 2021-02-22 00:41:40 UTC

Internal Links: 1201947

Description Jiri Konecny 2015-03-11 12:25:03 UTC
Description of problem:
I waited few seconds at "Installation Summary" screen (nothing was set yet).

Version-Release number of selected component:
anaconda-23.1-1

The following was filed automatically by anaconda:
anaconda 23.1-1 exception report
Traceback (most recent call first):
  File "/usr/lib64/python2.7/site-packages/gi/overrides/BlockDev.py", line 109, in lvm_round_size_to_pe
    return _lvm_round_size_to_pe(size, pe_size, roundup)
  File "/tmp/updates/blivet/devices/lvm.py", line 377, in align
    return Size(blockdev.lvm_round_size_to_pe(size, self.peSize, roundup))
  File "/tmp/updates/blivet/partitioning.py", line 941, in __init__
    lv.vg.align(lv.format.maxSize)) if l > Size(0)]
  File "/tmp/updates/blivet/partitioning.py", line 1848, in growLVM
    chunk = VGChunk(vg, requests=[LVRequest(l) for l in fatlvs])
  File "/tmp/updates/pyanaconda/kickstart.py", line 808, in execute
    growLVM(storage)
  File "/tmp/updates/pyanaconda/kickstart.py", line 2066, in doKickstartStorage
    ksdata.logvol.execute(storage, ksdata, instClass)
  File "/tmp/updates/pyanaconda/ui/gui/spokes/storage.py", line 326, in _doExecute
    doKickstartStorage(self.storage, self.data, self.instclass)
  File "/usr/lib64/python2.7/threading.py", line 766, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/tmp/updates/pyanaconda/threads.py", line 238, in run
    threading.Thread.run(self, *args, **kwargs)
OverflowError: long too big to convert

Additional info:
addons:         com_redhat_kdump
cmdline:        /usr/bin/python2  /sbin/anaconda
cmdline_file:   initrd=/test/jk/fedora/initrd.img root=live:http://cobra02/trees/jk/my_comps_squashfs.img nokill updates=http://cobra02/ks/jk/master_updates.img resolution=1024x768 ks=http://cobra02/ks/jk/testing.ks BOOT_IMAGE=/test/jk/fedora/vmlinuz 
executable:     /sbin/anaconda
hashmarkername: anaconda
kernel:         4.0.0-0.rc1.git2.1.fc23.x86_64
product:        Fedora"
release:        Cannot get release name.
type:           anaconda
version:        Fedora

Comment 1 Jiri Konecny 2015-03-11 12:25:07 UTC
Created attachment 1000371 [details]
File: anaconda-tb

Comment 2 Jiri Konecny 2015-03-11 12:25:08 UTC
Created attachment 1000372 [details]
File: anaconda.log

Comment 3 Jiri Konecny 2015-03-11 12:25:09 UTC
Created attachment 1000373 [details]
File: dnf.log

Comment 4 Jiri Konecny 2015-03-11 12:25:11 UTC
Created attachment 1000374 [details]
File: dnf.rpm.log

Comment 5 Jiri Konecny 2015-03-11 12:25:12 UTC
Created attachment 1000375 [details]
File: environ

Comment 6 Jiri Konecny 2015-03-11 12:25:13 UTC
Created attachment 1000376 [details]
File: ks.cfg

Comment 7 Jiri Konecny 2015-03-11 12:25:14 UTC
Created attachment 1000377 [details]
File: lsblk_output

Comment 8 Jiri Konecny 2015-03-11 12:25:15 UTC
Created attachment 1000378 [details]
File: nmcli_dev_list

Comment 9 Jiri Konecny 2015-03-11 12:25:16 UTC
Created attachment 1000379 [details]
File: os_info

Comment 10 Jiri Konecny 2015-03-11 12:25:18 UTC
Created attachment 1000380 [details]
File: program.log

Comment 11 Jiri Konecny 2015-03-11 12:25:19 UTC
Created attachment 1000381 [details]
File: storage.log

Comment 12 Jiri Konecny 2015-03-11 12:25:21 UTC
Created attachment 1000382 [details]
File: syslog

Comment 13 Jiri Konecny 2015-03-11 12:25:23 UTC
Created attachment 1000383 [details]
File: ifcfg.log

Comment 14 Jiri Konecny 2015-03-11 12:25:24 UTC
Created attachment 1000384 [details]
File: packaging.log

Comment 15 mulhern 2015-03-11 13:38:37 UTC
size is 16EiB which is too big for a 64 bit unsigned integer by 1 byte.

It looks like libblockdev will need to store sizes in two integers (or blivet will have to restrict its sizes to 16 EiB - 1 bytes).

Comment 16 mulhern 2015-03-11 13:58:29 UTC
or maybe gmp is an option.

Comment 17 Jiri Konecny 2015-03-11 15:52:45 UTC
To add more information that was on virtual machine with only one Disk: 

Size: 14GiB 
Disk bus: VirtIO
Storage format: qcow2

Comment 18 Vratislav Podzimek 2015-03-12 16:18:29 UTC
(In reply to mulhern from comment #16)
> or maybe gmp is an option.
Or maybe rewriting the new bytesize project into C which would make it usable for libblockdev too.

Comment 19 Vratislav Podzimek 2015-03-12 16:21:46 UTC
Other than that I think this is a Blivet bug, because the real limits for these filesystems are "what fits into an unsigned 64bit integer", i.e. '16 EiB - 1'. I checked the xfsprogs sources and they use either 'long long' or '_uint64_t' as types for their size variables. I strongly believe the same applies to btrfs.

Comment 20 Jiri Konecny 2015-03-13 07:25:41 UTC
It's doing only with the kickstart file, I didn't know which specific configuration line doing this but I could try to find it if you want.

If you need more info it's no problem to reproduce this bug. Feel free to ask.

Comment 21 mulhern 2015-03-13 12:41:32 UTC
In bytesize, it will not be possible, or at least it will be discouraged, to add numbers to sizes (or subtract numbers from sizes). That's not the real problem with this patch, though.

*) You haven't confirmed that xfs or btrfs are actually constrained to this number...maybe they store all values as multiples of sectors. We should make sure that we know the _actual_ limits of these two, probably by asking, before we make a patch based on their actual limits.

*) This doesn't fix the problem of the way that libblockdev represents integers, even in this particular instance. It is true that this particular test case will no longer raise that particular exception. But, it will do the wrong thing, because the integer will overflow in bd_lvm_round_size_to_pe, since it is rounding up. A quick glance seems to indicate that the result of rounding 2^64 - 1 up to the nearest pe_size boundary will return pe_size, which is wrong.

Reassigning to libblockdev.

Comment 22 Vratislav Podzimek 2015-03-16 13:15:51 UTC
Let's discuss the changes in patch review, not in the bug report.

> Reassigning to libblockdev.
Reassigning back as this clearly is a bug in blivet. The libblockdev clearly states that what it takes is a number of bytes of the 64bit unsigned int type. Blivet is giving it something that is not a correct value of such type.

Comment 23 mulhern 2015-03-16 14:14:16 UTC
In that case, blivet should not be using a library which can't accept the sizes that blivet works with. That library needs to be fixed soon or the dependency on it removed. See related bug: bz#1201947.

Comment 24 Vratislav Podzimek 2015-03-16 16:29:36 UTC
*** Bug 1202444 has been marked as a duplicate of this bug. ***

Comment 25 Vratislav Podzimek 2015-03-16 17:52:39 UTC
Here is the link to the PR targetting this issue:
https://github.com/rhinstaller/blivet/pull/46

Comment 26 Dan Mossor [danofsatx] 2015-03-16 17:55:17 UTC
Discussed at Fedora Blocker Review Meeting 2015-03-16[0]:

AcceptedBlocker Beta - As proposed on RHBZ#1202444, this bug is a clear violation of the Beta criterion[1]: "Cleanly install to a disk with a valid ms-dos or gpt disk label and partition table which contains existing data and sufficient unpartitioned space for a Fedora installation."

[0]: http://meetbot.fedoraproject.org/fedora-blocker-review/2015-03-16/f22-blocker-review.2015-03-16-16.01.log.txt
[1]: https://fedoraproject.org/wiki/Fedora_22_Beta_Release_Criteria#Guided_partitioning

Comment 27 David Lehman 2015-03-16 19:06:25 UTC
As a short-term solution we have the following options:

1) export a value from libblockdev indicating the maximum size it can work with
   and make sure in blivet that we do not exceed it
2) protect this call by limiting the size to the maximum size for an LV
3) lower the maximum size for xfs/btrfs to 15 EiB (it is a theoretical maximum
   anyway, so there is no need to panic about not supporting 16 EiB for now)

We agreed to go with option 3 this morning during the team meeting. I think we should explore at least option 2 in addition.

Comment 28 Vratislav Podzimek 2015-03-17 08:41:02 UTC
Both 2) and 3) are now implemented in the PR:
https://github.com/rhinstaller/blivet/pull/46

Comment 29 David Lehman 2015-03-17 18:13:19 UTC
While I still think the patch that implements #2 from comment 27 should be pushed to master, we have another way to address this particular issue in F22 without having to change the max sizes for any filesystems.

Comment 30 Adam Williamson 2015-03-18 03:03:55 UTC
Another user experienced a similar problem:

Test string.

addons:         com_redhat_kdump
cmdline:        /usr/bin/python2  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-22_B_T2-x86_64 quiet
dnf.rpm.log:    Mar 18 01:53:35 INFO --- logging initialized ---
hashmarkername: anaconda
kernel:         4.0.0-0.rc1.git0.1.fc22.x86_64
package:        anaconda-22.20.4-1
product:        Fedora"
reason:         OverflowError: long too big to convert
release:        Cannot get release name.
version:        Fedora

Comment 31 Adam Williamson 2015-03-18 05:15:18 UTC
Another user experienced a similar problem:

test test test

addons:         com_redhat_kdump
cmdline:        /usr/bin/python2  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-22_B_T2-x86_64 quiet
dnf.rpm.log:    Mar 18 01:53:35 INFO --- logging initialized ---
hashmarkername: anaconda
kernel:         4.0.0-0.rc1.git0.1.fc22.x86_64
package:        anaconda-22.20.4-1
product:        Fedora"
reason:         OverflowError: long too big to convert
release:        Cannot get release name.
version:        Fedora

Comment 32 Lukas Brabec 2015-03-18 11:56:55 UTC
Another user experienced a similar problem:

Tried to install F22 Beta TC2

addons:         com_redhat_kdump
cmdline:        /usr/bin/python2  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-22_B_T2-x86_64 quiet
dnf.rpm.log:    Mar 18 11:55:07 INFO --- logging initialized ---
hashmarkername: anaconda
kernel:         4.0.0-0.rc1.git0.1.fc22.x86_64
package:        anaconda-22.20.4-1
product:        Fedora"
reason:         OverflowError: long too big to convert
release:        Cannot get release name.
version:        Fedora

Comment 33 Fedora Update System 2015-03-18 12:13:37 UTC
anaconda-22.20.5-1.fc22, python-blivet-1.0.4-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/FEDORA-2015-4126/python-blivet-1.0.4-1.fc22,anaconda-22.20.5-1.fc22

Comment 34 Jan Sedlák 2015-03-18 14:54:50 UTC
I hadn't seen anyone mentioning it, doing custom partitioning in Anaconda doesn't trigger this error.

Comment 35 David Shea 2015-03-18 15:17:03 UTC
*** Bug 1203324 has been marked as a duplicate of this bug. ***

Comment 36 Adam Williamson 2015-03-18 16:18:31 UTC
jan: I'd kind of expect it to if you use xfs and create a partition with no specific size (i.e. ask anaconda to auto-size it).

Comment 37 Brandon Perkins 2015-03-18 18:14:18 UTC
If it's at all helpful, here is the partitioning/filesystem block of the kickstart I am using (each has a specific size specifified, but I do use 'grow' and 'recommended') that results in this issue:

======================================================
ignoredisk --only-use=vda

# System bootloader configuration
bootloader --location=mbr --boot-drive=vda

# Clear the Master Boot Record
zerombr

# Partition clearing information
clearpart --all --initlabel --drives=vda

#I am creating partitions here
part /boot --fstype=xfs  --size=256  --ondisk=vda --asprimary
#I will create the lvm stuff farther down
part pv.01 --fstype=lvmpv --size=8664 --ondisk=vda --asprimary --grow 

#Very important to have the two part lines before the lvm stuff
volgroup vg --pesize=4096 pv.01
logvol /     --fstype=xfs  --size=256  --name=lv_root --vgname=vg --grow
logvol /usr  --fstype=xfs  --size=2048 --name=lv_usr  --vgname=vg
logvol /tmp  --fstype=xfs  --size=64   --name=lv_tmp  --vgname=vg
logvol /var  --fstype=xfs  --size=1024 --name=lv_var  --vgname=vg
logvol /home --fstype=xfs  --size=128  --name=lv_home --vgname=vg
logvol swap  --fstype=swap --size=1048 --name=lv_swap --vgname=vg --recommended

Comment 38 Adam Williamson 2015-03-18 18:20:36 UTC
Brandon: it's not really needed, the devs are perfectly clear on what the bug is.

Comment 39 Lukas Brabec 2015-03-19 08:43:25 UTC
(In reply to Fedora Update System from comment #33)
> anaconda-22.20.5-1.fc22, python-blivet-1.0.4-1.fc22 has been submitted as an
> update for Fedora 22.
> https://admin.fedoraproject.org/updates/FEDORA-2015-4126/python-blivet-1.0.4-
> 1.fc22,anaconda-22.20.5-1.fc22

this fixed the problem for me

Comment 40 Fedora Update System 2015-03-19 18:21:37 UTC
anaconda-22.20.6-1.fc22, python-blivet-1.0.5-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/python-blivet-1.0.5-1.fc22,anaconda-22.20.6-1.fc22

Comment 41 Fedora Update System 2015-03-19 18:40:30 UTC
Package anaconda-22.20.5-1.fc22, python-blivet-1.0.4-1.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-22.20.5-1.fc22 python-blivet-1.0.4-1.fc22'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-4126/python-blivet-1.0.4-1.fc22,anaconda-22.20.5-1.fc22
then log in and leave karma (feedback).

Comment 42 colin 2015-03-20 21:10:15 UTC
Another user experienced a similar problem:

Boot install DVD to upgrade F22 Alpha
python exception occurs when try to remove F22Alpha partitions


addons:         com_redhat_kdump
cmdline:        /usr/bin/python2  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-22_B_T2-x86_64 quiet
dnf.rpm.log:    Mar 20 21:06:04 INFO --- logging initialized ---
hashmarkername: anaconda
kernel:         4.0.0-0.rc1.git0.1.fc22.x86_64
package:        anaconda-22.20.4-1
product:        Fedora"
reason:         OverflowError: long too big to convert
release:        Cannot get release name.
version:        Fedora

Comment 43 DO NOT USE account not monitored (old adamwill) 2015-03-20 21:18:49 UTC
colin, You don't need to run the installer again to update F22. Just update as usual, with GNOME Software or KDE Apper or dnf or yum or whatever. You will get the latest F22 bits.

Comment 44 colin 2015-03-21 12:47:37 UTC
Thanks Adam.
 
 I can report that I no longer encountered this bug when installing the updated Test Candidate 3  Fedora-Server-DVD-x86_64-22_Beta_TC3.iso

Comment 45 Fedora Update System 2015-03-23 07:11:03 UTC
anaconda-22.20.6-1.fc22, python-blivet-1.0.5-1.fc22, libblockdev-0.7-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.