RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1418283 - virt-v2v: appliance runs out of memory running setfiles command
Summary: virt-v2v: appliance runs out of memory running setfiles command
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.3
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard: V2V
Depends On: 1359086
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-02-01 12:56 UTC by Pawan kumar Vilayatkar
Modified: 2020-03-11 15:42 UTC (History)
9 users (show)

Fixed In Version: libguestfs-1.36.1-1.el7
Doc Type: Bug Fix
Doc Text:
Cause: when relabelling SELinux guests, the setfiles utility could require more memory (RAM) than needed Consequence: setfiles using all the memory of the libguestfs appliance (helper VM), and being killed because of that, causing the conversion to fail Fix: virt-v2v reserves more memory when launching its appliance Result: the relabelling operation can complete successfully, and so the conversion of SELinux guests
Clone Of:
Environment:
Last Closed: 2017-08-01 22:11:26 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:2023 0 normal SHIPPED_LIVE libguestfs bug fix and enhancement update 2017-08-01 19:32:01 UTC

Description Pawan kumar Vilayatkar 2017-02-01 12:56:54 UTC 
Description of problem:

While converting a VM using virt-v2v fails with below :
"[ 1147.056078] Out of memory: Kill process 8209 (setfiles) score 876 or sacrifice child
[ 1147.056994] Killed process 8209 (setfiles) total-vm:795640kB, anon-rss:711884kB, file-rss:0kB,
guestfsd: error: Can't stat exclude path "/sysroot/selinux", No such file or directory - ignoring.: Success
libguestfs: trace: v2v: selinux_relabel = -1 (error)
virt-v2v: error: libguestfs error: selinux_relabel: Can't stat exclude path 
"/sysroot/selinux", No such file or directory - ignoring.: Success


Version-Release number of selected component (if applicable):

virt-v2v-1.32.7-3.el7_3.2.x86_64                            Thu Jan 26 09:56:01 2017
libvirt-2.0.0-10.el7_3.4.x86_64                             Thu Jan 26 10:21:47 2017
libvirt-client-2.0.0-10.el7_3.4.x86_64                      Thu Jan 26 09:55:50 2017
libguestfs-1.32.7-3.el7_3.2.x86_64                          Thu Jan 26 09:56:00 2017
libguestfs-tools-c-1.32.7-3.el7_3.2.x86_64                  Thu Jan 26 09:56:01 2017




Actual results:

"LIBGUESTFS_DEBUG=1 virt-v2v -v -x -o rhev -n netecodmz -os kvm191m103:/opt/rhev/export2 satellite.ecofis.de 2>&1 | tee /root/v2v_satellite_002.log"

[ 1147.048860] 183 total pagecache pages
[ 1147.049302] 0 pages in swap cache
[ 1147.049693] Swap cache stats: add 0, delete 0, find 0/0
[ 1147.050322] Free swap  = 0kB
[ 1147.050667] Total swap = 0kB
[ 1147.051023] 204698 pages RAM
[ 1147.051373] 0 pages HighMem/MovableOnly
[ 1147.051821] 7398 pages reserved
[ 1147.052212] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
[ 1147.053132] [  100]     0   100    11399      143      22        0         -1000 systemd-udevd
[ 1147.054146] [  182]     0   182    31263      176      32        0             0 guestfsd
[ 1147.055110] [ 8209]     0  8209   198910   177971     362        0             0 setfiles
[ 1147.056078] Out of memory: Kill process 8209 (setfiles) score 876 or sacrifice child
[ 1147.056994] Killed process 8209 (setfiles) total-vm:795640kB, anon-rss:711884kB, file-rss:0kB, shmem-rss:0kB
guestfsd: error: Can't stat exclude path "/sysroot/selinux", No such file or directory - ignoring.: Success
libguestfs: trace: v2v: selinux_relabel = -1 (error)
virt-v2v: error: libguestfs error: selinux_relabel: Can't stat exclude path 
"/sysroot/selinux", No such file or directory - ignoring.: Success
umount '/tmp/v2v.xIBg9W'
rm -rf '/var/tmp/null.u7Q4NG'
libguestfs: trace: v2v: close




Additional info:

Attaching virt-v2v logs.
Comment 1 Pawan kumar Vilayatkar 2017-02-01 12:59:18 UTC 
Created attachment 1246644 [details]
virt-v2v.log
Comment 3 Richard W.M. Jones 2017-02-01 13:33:48 UTC 
Before running virt-v2v, can you set this environment variable and
see if it fixes the problem:

export LIBGUESTFS_MEMSIZE=2048
Comment 4 Pawan kumar Vilayatkar 2017-02-02 09:33:44 UTC 
Hello Richard,

Setting the environment variable resolved the problem. 
I am closing the bugzilla

Thanks
Comment 6 Richard W.M. Jones 2017-02-02 09:37:59 UTC 
Thanks for testing this.  We have an upstream fix which
increases the memory available to the appliance:

https://github.com/libguestfs/libguestfs/commit/2b5173712ba07e3e89926c82a236390a4ea6626a

Please leave this bug open so it can be included in RHEL 7.4.
Comment 8 Richard W.M. Jones 2017-02-21 20:05:17 UTC 
Reproducer:

$ virt-builder fedora-25 --size 10G --output test.img
$ guestfish -a test.img -i fill-dir /etc 4000000
$ virt-v2v -i disk test.img -o null

Both the guestfish and virt-v2v commands may take a very long
time to run.  If you can't reproduce the bug, try increasing
the constant (4000000) to something even larger.

Notice that you may see various different errors from
selinux_relabel, because it can run out of memory in slightly
different places.  However as long as it's selinux_relabel
which is breaking, then you are seeing the error.

This bug (which is a bug in SELinux) appears to be fixed
in Fedora 25+
Comment 11 kuwei@redhat.com 2017-03-14 05:03:12 UTC 
Verify with below bulids:
virt-v2v-1.36.2-1.el7.x86_64
libvirt-3.1.0-2.el7.x86_64
qemu-kvm-rhev-2.8.0-6.el7.x86_64

steps:(same as comment 8):
$ virt-builder fedora-25 --size 10G --output test.img
$ guestfish -a test.img -i fill-dir /etc 4000000
# virt-v2v -i disk test.img -o null
[   0.0] Opening the source -i disk test.img
[   0.1] Creating an overlay to protect the source from being modified
[   0.3] Initializing the target -o null
[   0.3] Opening the overlay
[   1.5] Inspecting the overlay
[   6.1] Checking for sufficient free disk space in the guest
[   6.1] Estimating space required on target for each disk
[   6.1] Converting Fedora 25 (Twenty Five) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[ 708.8] Mapping filesystem data to avoid copying unused and blank areas
[ 708.9] Closing the overlay
[ 709.0] Checking if the guest needs BIOS or UEFI to boot
[ 709.0] Assigning disks to buses
[ 709.0] Copying disk 1/1 to /var/tmp/null.K51C7B/sda (raw)
    (100.00/100%)
[ 725.2] Creating output metadata
[ 725.2] Finishing off

Hi,rjones,from your comment 8 reproduce steps ,the  conversion is successfully . But i am not sure if it can work better in customer Environment.
Comment 12 Richard W.M. Jones 2017-03-14 08:50:28 UTC 
The real bug is a memory leak in the setfiles program.  By increasing
the memory available to the program, we can make it fail after
a longer time (hopefully much longer), but there will always be a
case where a disk is so complex that it triggers the memory leak.

Apparently we cannot fix the actual setfiles program because that's
too complex or something.

Anyway, I think we should move this to VERIFIED.
Comment 13 kuwei@redhat.com 2017-03-14 08:59:12 UTC 
From comment 11 and comment 12,move the bug from ON_QA to VERIFIED
Comment 14 kuwei@redhat.com 2017-03-14 11:10:46 UTC 
(In reply to kuwei from comment #11)
> Verify with below bulids:
> virt-v2v-1.36.2-1.el7.x86_64
> libvirt-3.1.0-2.el7.x86_64
> qemu-kvm-rhev-2.8.0-6.el7.x86_64
> 
> steps:(same as comment 8):
> $ virt-builder fedora-25 --size 10G --output test.img
> $ guestfish -a test.img -i fill-dir /etc 4000000
> # virt-v2v -i disk test.img -o null


Try to builder a rhel7.3 geust to convert as the steps:
steps::
$ virt-builder rhel-7.3 --size 10G --output test.img
$ guestfish -a test.img -i fill-dir /etc 4000000
# virt-v2v -i disk test.img -o null
[   0.0] Opening the source -i disk test.img
[   0.1] Creating an overlay to protect the source from being modified
[   0.3] Initializing the target -o null
[   0.3] Opening the overlay
[   1.5] Inspecting the overlay
[  10.2] Checking for sufficient free disk space in the guest
[  10.2] Estimating space required on target for each disk
[  10.2] Converting Red Hat Enterprise Linux Server 7.3 (Maipo) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[ 722.9] Mapping filesystem data to avoid copying unused and blank areas
[ 723.7] Closing the overlay
[ 723.7] Checking if the guest needs BIOS or UEFI to boot
[ 723.7] Assigning disks to buses
[ 723.7] Copying disk 1/1 to /var/tmp/null.nUCcAd/sda (raw)
    (100.00/100%)
[ 741.7] Creating output metadata
[ 741.7] Finishing off

Result: The conversion successfully too.
Comment 15 errata-xmlrpc 2017-08-01 22:11:26 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2023
Note You need to log in before you can comment on or make changes to this bug.