Red Hat Bugzilla – Bug 1239154
appliance fails to start with "supermin: ext2fs_file_write: /var/log/tallylog: Could not allocate block in ext2 filesystem"
Last modified: 2016-11-03 13:54:18 EDT
Description of problem: 'virt-df <instance>' command fails with "supermin: ext2fs_file_write: /var/log/tallylog: Could not allocate block in ext2 filesystem" error. # virt-df instance-000155e2 libguestfs: error: /usr/bin/supermin5 exited with error status 1. To see full error messages you may need to enable debugging. See http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs Errors reported in debug logs : ------ supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.rxy85058/root' supermin: ext2: populating from base image supermin: ext2: copying files from host filesystem **supermin: ext2fs_file_write: /var/log/tallylog: Could not allocate block in ext2 filesystem** **supermin: failure: ext2fs_file_write** **libguestfs: error: /usr/bin/supermin5 exited with error status 1, see debug messages above*** libguestfs: closing guestfs handle 0x7f8b23aaf400 (state 0) ---- Version-Release number of selected component (if applicable): Red Hat Enterprise Linux 7.1 Name : libguestfs-tools Arch : noarch Epoch : 1 Version : 1.28.1 Release : 1.18.el7 Size : 30 k Repo : installed Name : supermin Arch : x86_64 Version : 5.1.10 Release : 1.2.el7 Size : 217 k How reproducible: always on particular system at a customer end. Additional info: The issue is reporducible on some particular systems only. It is not on all the identical system.
If you look into /var/log on the customer system, you should find that one file is particularly large. It may be /var/log/tallylog, or it might be another file from /var/log. Having a listing of ls -l /var/log would be useful. Once you've identified the large file, you can work around the problem by editing /usr/lib64/guestfs/supermin.d/excludefiles and add the following line: -/var/log/tallylog (Or if another file is the large file, put that filename instead). Note the filename is preceeded by a '-' character, and there must be no extra spaces, blank lines, etc in the file.
Created attachment 1050107 [details] ls -lR /var/log output
Patch posted upstream: https://www.redhat.com/archives/libguestfs/2015-July/msg00092.html Prasad, could you try (or get the customer to try) this: Edit /usr/lib64/guestfs/supermin.d/excludefiles Remove all lines from the file that begin with -/var/log/ Add a single line at the end of the file: -/var/log/* (note it must have a leading minus sign, and no extra spaces or blank lines). Run 'libguestfs-test-tool' and see if the failure is fixed.
Upstream commit is 95bb1916eb1b2f6f9e9575cf4bf92501292bf545. Unfortunately we didn't manage to collect the debug information required to ensure this fix is the correct one. Therefore I am moving this to RHEL 7.3. This will be automatically picked up by the planned rebase for RHEL 7.3 (bug 1218766).
*** Bug 1264241 has been marked as a duplicate of this bug. ***
Hi Rich, I can't reproduce this bug with low version libguestfs: libguestfs-1.28.1-1.55.el7_2.4.x86_64 libguestfs-tools-1.28.1-1.55.el7_2.4.noarch supermin5-5.1.10-1.2.el7.x86_64 So I can't confirm if it has been modified now. From the above comments, I could get limited info about customer env, this my steps : 1. Prepare a RHEL7 host. 2. Create a big log file: # dd if=/dev/zero of=/var/log/bigfile count=10240 bs=1M 2. Install a RHEL7 guest image with ext2, and start it. # virsh list --all Id Name State ---------------------------------------------------- 20 rhel7.1-ext2 running 3. # virt-df -d rhel7.1-ext2 Filesystem 1K-blocks Used Available Use% rhel7.1-ext2:/dev/sda1 495844 104564 365680 22% rhel7.1-ext2:/dev/rhel/root 18578172 1706384 15928072 10% Can you help to tell me what else should I do to prepare for the env? Or just tell me how to verify this bug in other way? Thank you very much:)
(In reply to Xianghua Chen from comment #17) > Hi Rich, > > I can't reproduce this bug with low version libguestfs: > libguestfs-1.28.1-1.55.el7_2.4.x86_64 > libguestfs-tools-1.28.1-1.55.el7_2.4.noarch > supermin5-5.1.10-1.2.el7.x86_64 > > So I can't confirm if it has been modified now. From the above comments, I > could get limited info about customer env, this my steps : > 1. Prepare a RHEL7 host. > 2. Create a big log file: > # dd if=/dev/zero of=/var/log/bigfile count=10240 bs=1M I think if you create a massive /var/log/tallylog (instead of bigfile) on the host then you should be able to reproduce this. And then just run libguestfs-test-tool. There is no need to create any guests.
Verified with packages: libguestfs-1.32.6-1.el7.x86_64 supermin5-5.1.16-4.el7.x86_64 Steps: 1. Prepare a RHEL7 host. 2. Create a big log file: # dd if=/dev/zero of=/var/log/tallylog count=10240 bs=1M 3. # libguestfs-test-tool Command finished successfully, there is no error occurred like: supermin: ext2: copying files from host filesystem supermin: error: statvfs: No space left on device: /var/tmp/.guestfs-0/appliance.d.kjkoazrb/root libguestfs: error: /usr/bin/supermin5 exited with error status 1, see debug messages above
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2016-2576.html