Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
I've a backup script that stores data on a btrfs volume, take btrfs snapshots daily, update the data and keep a maximum number of snapshots by deleting the old snapshots.
When you run the script multiple times - more than 15 times - , then try to delete multiple old snapshots by running
btrfs subvolume delete /data/_backup/backup.8 , for example, and running it again for deletion of few other subvolume/snapshots the system crashes with kernel panic , crash dump, then automatic reboot.
I've tried this on another machine, freshly created the fs, copied the data, ran the script multiple times, tried to multiple/all the snapshots and the same problem happened.
I've finally found a temporary solution, but pulling the latest kernel from git repos, linux-HEAD-37cf951 , compiled it and reboot the system .. Now the problem is solved, and it doesn't occur any more. I've aggressively tested it and I'm sure now that this bug is solved at this bleeding edge kernel.
The used data size is 2.6TB, the number of files is ~ 2,500,000 .. The total btrfs volume size is 4TB .
The crash dumps with 6.1 kernel and 6.2 beta kernel and the script is attached .. "The rsync lines are commented to reproduce the bug faster"
Version-Release number of selected component (if applicable):
The problem occurred on kernels;
kernel-2.6.32-131.12.1.el6.x86_64
kernel-2.6.32-131.17.1.el6.x86_64
kernel-2.6.32-202.el6.x86_64
It's solved on;
kernel-3.1.0_rc9-1.x86_64 compiled from linux-HEAD-37cf951
How reproducible:
Create a btrfs volume > 3.5TB . Mount it, under /data for example, without any special options. Populate it with random data of 2.6TB size and ~ 2,500,00 , avg file size is ~ 1MB .
Steps to Reproduce:
1. Take 25 snapshot of the whole voulme, btrfs subvolume snapshot /data/ /data/_backup/snap.1 , again 25 times until btrfs subvolume snapshot /data/ /data/_backup/snap.25 .. Try to make any little data modification between each snapshot ..
2. Run this script to delete then randomly ,
for i in `ls -1 | sort -R`; do btrfs subvolume delete /data/_backup/$i ; sleep 5; done
Actual results:
The system should crash within few seconds.
Expected results:
It should work fine, and those subvolume should be deleted without a problem.
Additional info:
Comment 11Rodrigo A B Freire
2012-05-07 14:03:48 UTC
Hi Amr,
Unfortunately, the fix didn't yet hit RHEL 6.3.
However, we have a work around for this:
1) Umount your btrfs volume
2) Check it with btrfsck
3) Remount it
4) Then, delete your snapshot.
Hope that helps,
- RF