RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1159804 - virt-v2v will hang when convert guest with lvm partition
Summary: virt-v2v will hang when convert guest with lvm partition
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libguestfs
Version: 7.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Virtualization Bugs
URL:
Whiteboard: V2V
Depends On: 1154778
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-11-03 11:38 UTC by zhoujunqin
Modified: 2014-11-07 07:23 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1160488 (view as bug list)
Environment:
Last Closed: 2014-11-07 07:23:42 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
detail debug info for virt-v2v (25.79 KB, text/plain)
2014-11-03 11:38 UTC, zhoujunqin
no flags Details

Description zhoujunqin 2014-11-03 11:38:28 UTC
Created attachment 953094 [details]
detail debug info for virt-v2v

Description of problem:
virt-v2v will hang when convert guest with lvm partition

Version-Release number of selected component (if applicable):
virt-v2v-1.28.1-1.6.el7.x86_64
libguestfs-1.28.1-1.6.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Prepare a guest with lvm partition on esx server. Login in guest and check
# fdisk -l

Disk /dev/sda: 6442 MB, 6442450944 bytes
199 heads, 62 sectors/track, 1019 cylinders
Units = cylinders of 12338 * 512 = 6317056 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000a2972

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          84      512000   83  Linux
Partition 1 has different physical/logical beginnings (non-Linux?):
     phys=(0, 32, 33) logical=(0, 33, 3)
Partition 1 has different physical/logical endings:
     phys=(63, 221, 30) logical=(83, 32, 10)
Partition 1 does not end on cylinder boundary.
/dev/sda2              84        1020     5778432   8e  Linux LVM
Partition 2 has different physical/logical beginnings (non-Linux?):
     phys=(63, 221, 31) logical=(83, 32, 11)
Partition 2 has different physical/logical endings:
     phys=(783, 63, 48) logical=(1019, 169, 12)
Partition 2 does not end on cylinder boundary.

Disk /dev/mapper/vg_dhcp66106221-lv_root: 5272 MB, 5272240128 bytes
255 heads, 63 sectors/track, 640 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/vg_dhcp66106221-lv_swap: 641 MB, 641728512 bytes
255 heads, 63 sectors/track, 78 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


2. Use virt-v2v convert guest "rhel6-tzheng" to local

# virt-v2v -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1  rhel6-tzheng


Actual results:
virt-v2v command hangs:
[   0.0] Opening the source -i libvirt -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1 rhel6-tzheng
Enter root's password for 10.66.111.25: 
Enter host password for user 'root':
[  14.0] Creating an overlay to protect the source from being modified
[  15.0] Opening the overlay

Expected results:
virt-v2v should convert guest with lvm partition.

Additional info:
I will attach debug info.
# virt-v2v -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1  rhel6-tzheng -v -x |& tee /tmp/rhel6-tzheng_debug.log
...
/init: line 73: /sys/block/ubd*/queue/scheduler: No such file or directory
/init: line 73: /sys/block/vd*/queue/scheduler: No such file or directory
mdadm: No arrays found in config file or automatically
[    5.615402] device-mapper: uevent: version 1.0.3
[    5.616128] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel
  lvmetad is not active yet, using direct activation during sysinit
  2 logical volume(s) in volume group "vg_dhcp66106221" now active

Comment 2 Richard W.M. Jones 2014-11-03 12:58:15 UTC
This is essentially the same as:
https://bugzilla.redhat.com/show_bug.cgi?id=1153589
which was caused by the terrible performance of VMware vCenter.

I added some patches to mitigate this, which are included in
the version you are testing.

I think it's important to find out: is it hanging completely,
or just going very slowly?  When it hangs, can you check the
file /var/tmp/v2vovl*.qcow2 and see if this file grows in size
(perhaps slowly)?

Comment 3 tingting zheng 2014-11-04 02:25:55 UTC
(In reply to Richard W.M. Jones from comment #2)
> This is essentially the same as:
> https://bugzilla.redhat.com/show_bug.cgi?id=1153589
> which was caused by the terrible performance of VMware vCenter.

I file bug 1153589 as it hangs everytime caused by the terrible performance of VMware vCenter.In new version of vCenter 5.5,only guests with lvm partition will hang,other guests like linux guests with standard partitions and windows can be converted quickly and successfully.


> I added some patches to mitigate this, which are included in
> the version you are testing.
> 
> I think it's important to find out: is it hanging completely,
> or just going very slowly?  When it hangs, can you check the
> file /var/tmp/v2vovl*.qcow2 and see if this file grows in size
> (perhaps slowly)?

It indeed hang,juzhou will provide more info for you.

Comment 4 zhoujunqin 2014-11-04 02:55:02 UTC
> I added some patches to mitigate this, which are included in
> the version you are testing.
> 
> I think it's important to find out: is it hanging completely,
> or just going very slowly?  When it hangs, can you check the
> file /var/tmp/v2vovl*.qcow2 and see if this file grows in size
> (perhaps slowly)?

I think it's hanging completely, my reason:
# virt-v2v -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1  rhel6-tzheng-lvm -on rhel6-tzheng-lvm-second -v -x |& tee rhel6-tzheng-lvm-second_debug.log
...
[    5.596448] device-mapper: uevent: version 1.0.3
[    5.597177] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel
  lvmetad is not active yet, using direct activation during sysinit
  2 logical volume(s) in volume group "vg_dhcp66106221" now active

then check the size of file /var/tmp/v2vovl*.qcow2 several times, after it's hanging it will keep in one size:

# ll /var/tmp/v2vovl7d1915.qcow2 
-rw-------. 1 root root 1179648 Nov  4 10:34 /var/tmp/v2vovl7d1915.qcow2

Comment 5 Richard W.M. Jones 2014-11-04 23:15:37 UTC
I am able to reproduce this now.  It seems like a udev bug, but I
am still investigating.

Comment 6 Richard W.M. Jones 2014-11-04 23:56:26 UTC
Can you try with systemd = 208-16.1.rwmj.el7 available as
a scratch build here:

https://brewweb.devel.redhat.com/taskinfo?taskID=8196934

Comment 7 zhoujunqin 2014-11-05 03:18:15 UTC
Hi rjones,
i tested with your scratch build:

Steps:
libgudev1-208-16.1.rwmj.el7.x86_64
systemd-python-208-16.1.rwmj.el7.x86_64
systemd-libs-208-16.1.rwmj.el7.x86_64
systemd-sysv-208-16.1.rwmj.el7.x86_64
systemd-208-16.1.rwmj.el7.x86_64
libguestfs-1.28.1-1.7.el7.x86_64
virt-v2v-1.28.1-1.7.el7.x86_64

# virt-v2v -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1  rhel6-tzheng-lvm 
[   0.0] Opening the source -i libvirt -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1 rhel6-tzheng-lvm
Enter root's password for 10.66.111.25: 
Enter host password for user 'root':
[  16.0] Creating an overlay to protect the source from being modified
[  17.0] Opening the overlay
[  30.0] Initializing the target -o libvirt -os default
[  30.0] Inspecting the overlay
[  63.0] Checking for sufficient free disk space in the guest
[  63.0] Estimating space required on target for each disk
[  63.0] Converting Red Hat Enterprise Linux Server release 6.6 (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 407.0] Mapping filesystem data to avoid copying unused and blank areas
[ 409.0] Closing the overlay
[ 409.0] Copying disk 1/1 to /var/lib/libvirt/images/rhel6-tzheng-lvm-sda (raw)
    (100.00/100%)
[ 514.0] Creating output metadata
Pool default refreshed

Domain rhel6-tzheng-lvm defined from /tmp/v2vlibvirt416088.xml

[ 515.0] Finishing off

Result: Conversion finished successfully without hanging and guest can boot up successfully.

Comment 8 Richard W.M. Jones 2014-11-06 13:46:46 UTC
This was a systemd bug which has been fixed:

https://bugzilla.redhat.com/show_bug.cgi?id=1154778#c5

Should we close this bug?  Leave it open for testing?  Attach
it to the erratum?

Comment 9 tingting zheng 2014-11-07 02:35:52 UTC
(In reply to Richard W.M. Jones from comment #8)
> This was a systemd bug which has been fixed:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1154778#c5
> 
> Should we close this bug?  Leave it open for testing?  Attach
> it to the erratum?

We will test some guests with lvm partition,if there are no such problems,then the bug can be closed.

Comment 10 zhoujunqin 2014-11-07 07:23:42 UTC
As tzheng said on Comment 9, i tried with other guest with lvm partition:
packages:
libguestfs-1.28.1-1.8.el7.x86_64
virt-v2v-1.28.1-1.8.el7.x86_64

steps:
1. Prepare a guest named "test-rhel7.0-released-gui" on esx server with lvm partitions:
# fdisk -l

Disk /dev/vda: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00059586

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1   *        2048     1026047      512000   83  Linux
/dev/vda2         1026048    16777215     7875584   8e  Linux LVM

Disk /dev/mapper/rhel-root: 7201 MB, 7201619968 bytes, 14065664 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/rhel-swap: 859 MB, 859832320 bytes, 1679360 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

2. 
# virt-v2v -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1 test-rhel7.0-released-gui 
[   0.0] Opening the source -i libvirt -ic vpx://root.111.25/tzheng-demo/10.66.106.63/?no_verify=1 test-rhel7.0-released-gui
Enter root's password for 10.66.111.25: 
Enter host password for user 'root':
[  15.0] Creating an overlay to protect the source from being modified
[  15.0] Opening the overlay
[ 125.0] Initializing the target -o libvirt -os default
[ 125.0] Inspecting the overlay
[ 186.0] Checking for sufficient free disk space in the guest
[ 186.0] Estimating space required on target for each disk
[ 186.0] Converting Red Hat Enterprise Linux Server release 7.0 (Maipo) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 573.0] Mapping filesystem data to avoid copying unused and blank areas
[ 577.0] Closing the overlay
[ 577.0] Copying disk 1/1 to /var/lib/libvirt/images/test-rhel7.0-released-gui-sda (raw)
    (100.00/100%)
[ 819.0] Creating output metadata
Pool default refreshed

Domain test-rhel7.0-released-gui defined from /tmp/v2vlibvirtd5a28b.xml

[ 820.0] Finishing off


Result:  Conversion finished successfully without hanging and guest can boot up successfully.

Seen above comments, so closed this bug as CURRENTRELEASED.


Note You need to log in before you can comment on or make changes to this bug.