Bug 1979179
| Summary: | Conversion of ext4 filesystem to btrfs fails with error. | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | piio <bugzilla> | ||||||
| Component: | btrfs-progs | Assignee: | Josef Bacik <josef> | ||||||
| Status: | CLOSED EOL | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 35 | CC: | adam900710, bugzilla, davide, esandeen, giorgos, josef, michel, ngompa13 | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2022-12-13 15:26:02 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Upstream thread: https://lore.kernel.org/linux-btrfs/CAJCQCtQjx2xmYvdoGj5+Sx9_R+5SZkXMkUg0kzXnEM5ZvEx9=Q@mail.gmail.com/T/#u piio would you attach the file from the following command for the ext4 file system you want to convert? substitute /dev/loop0 for your device # debugfs -R show_super_stats /dev/loop0 > ext4-stats.txt For the available space mismatch thing, it's more or less expected. Btrfs needs large chunks of unallocated space, thus fragmented space is not going to be counted as available. With around 500 MiB space left, I highly doubt if we have enough space for metadata (especially csum, which can be pretty big for this particular case) Thus I thing the final error is related to the ENOSPC. For better debugging and to enhance btrfs-convert for its error message, would you mind to dump the source ext4 fs by: # e2image -Q <device> And upload it? Created attachment 1798520 [details]
show_super_stats result
debugfs show_super_stats result
gzipped e2image dump - https://we.tl/t-xNUVnY0ReA >Block count: 13107200
>Free blocks: 5395324
50G file system with 20G free? The super shows quite a lot of block groups unused, so it shouldn't be badly fragmented.
Can't reproduce the problem here.
The same btrfs-progs v5.12.1.
But convert seems to detect everything correctly:
free space report:
total: 991297536000
free: 152739930112 (15.41%)
The total size matches the raw image, and the free space also more or less matches the report from e2fsprogs.
The convert is taking a quite long time to run, but so far no crash.
I'm wondering why the initial report has a completely different (maybe something underflowed?) for the free space:
total: 53687091200
free: 578277376 (1.08%)
Or is the e2image dump a different filesystem?
My bad, confused with another image... And yes, I can reproduce the reported free space problem now, but I can't reproduce the crash. The error I got is a graceful error: creating btrfs metadata ERROR: failed to commit transaction: -28 ERROR: error during copy_inodes -28 WARNING: error during conversion, the original filesystem is not modified Will look into the problem locally, and update when found something. I'm afraid btrfs-convert is correct. I dumped all used space reported from e2progs libs and calculated the free space. It looks like, although the ext4 system has quite some free space left, the free space is very fragmented. The average size of all free space is only around 350KiB. Thus only a very small set of those free space can be used. To be more accurate, there are 64848 free space fragments, but only 141 of them are larger than 16MiB, the minimal chunk size of btrfs. Thus it's really super framgmented free space causing the problem. Not sure about ext4, but maybe defrag could help? This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle. Changing version to 35. Hi, I'm sorry but till now I didn't have time to verify this issue. I don't found better way to defragment free space of existing ext4, so I created gzipped archive with most of data from this partition (/usr and /opt where i got git repositories). After that I converted successfully ext4 to btrfs, after that I restored previously removed data. Maybe is it possible to detect such highly-framented free space filesystem and suggest some actions to user? Or maybe there is any tool to automatically defragment ext4 free space? (In reply to piio from comment #11) > Hi, > > I'm sorry but till now I didn't have time to verify this issue. > I don't found better way to defragment free space of existing ext4, so I > created gzipped archive with most of data from this partition (/usr and /opt > where i got git repositories). After that I converted successfully ext4 to > btrfs, after that I restored previously removed data. > > Maybe is it possible to detect such highly-framented free space filesystem > and suggest some actions to user? Or maybe there is any tool to > automatically defragment ext4 free space? I ran into this issue today as well. One way to defragment free space is to use resize2fs to shrink the file system by the amount of space one expects btrfs to require for conversion (a few GiB?), then expand it back to its original size. All data will have been moved towards the beginning of the file system and btrfs-convert will have contiguous space at the end of it to work with. This message is a reminder that Fedora Linux 35 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '35'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 35 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed. Fedora Linux 35 entered end-of-life (EOL) status on 2022-12-13. Fedora Linux 35 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora Linux please feel free to reopen this bug against that version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see the version field. If you are unable to reopen this bug, please file a new report against an active release. Thank you for reporting this bug and we are sorry it could not be fixed. |
Created attachment 1798025 [details] coredump of btrfs-convert process Description of problem: Conversion of ext4 filesystem to btrfs fails with error Version-Release number of selected component (if applicable): btrfs-progs-5.12.1-2.fc35.x86_64 How reproducible: In my filesystem always Steps to Reproduce: 1. Run Fedora Rawhide livecd 2. fsck volume which i want to convert, no errors found [root@localhost-live ~]# fsck.ext4 -fyv /dev/mapper/fedora--volgroup e2fsck 1.46.2 (28-Feb-2021) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information 707293 inodes used (21.58%, out of 3276800) 7406 non-contiguous files (1.0%) 1851 non-contiguous directories (0.3%) # of inodes with ind/dind/tind blocks: 0/0/0 Extent depth histogram: 647435/2158 7705257 blocks used (58.79%, out of 13107200) 0 bad blocks 1 large file 567653 regular files 63898 directories 12 character device files 0 block device files 0 fifos 48357 links 75535 symbolic links (57494 fast symbolic links) 186 sockets ------------ 755641 files 3. run conversion: [root@localhost-live ~]# btrfs-convert -p /dev/mapper/fedora--volgroup create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) checksum: crc32c free space report: total: 53687091200 free: 578277376 (1.08%) creating ext2 image file creating btrfs metadata btrfs unable to find ref byte nr 54126084096 parent 0 root 2 owner 0 offset 0 path->slots[0]: 0 path->nodes[0]: Segmentation fault (core dumped) Actual results: btrfs-convert segfaults. Coredump in attachment Expected results: Filesystem converted from ext4 to btrfs without any errors Additional info: I checked few times, with the same arguments it's always falling. When I disabled datasum, ignored xattrs and ACL and disable inlining small files (btrfs-convert -d -n -i ), conversion succeded.