Bug 1170073
| Summary: | virt-v2v picks debug kernels over non-debug kernels when versions are equal | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | tingting zheng <tzheng> | ||||||||
| Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> | ||||||||
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | medium | ||||||||||
| Version: | 7.1 | CC: | codong, dyuan, juzhou, mbooth, mzhan, ptoscano, rjones, tzheng | ||||||||
| Target Milestone: | rc | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | Unspecified | ||||||||||
| OS: | Unspecified | ||||||||||
| Whiteboard: | V2V | ||||||||||
| Fixed In Version: | libguestfs-1.28.1-1.17.el7 | Doc Type: | Bug Fix | ||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2015-03-05 13:47:41 UTC | Type: | Bug | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
Hi Tingting, as always I need the full output (virt-v2v -v -x). The code does actually read the 'default' setting from the grub config, or it's supposed to be doing that. (In reply to Richard W.M. Jones from comment #2) > Hi Tingting, as always I need the full output (virt-v2v -v -x). > > The code does actually read the 'default' setting from the grub > config, or it's supposed to be doing that. I found that if there are several versions of kernel,virt-v2v always use the latest one whatever the default you set. If there are only 1 normal kernel and 1 debug kernel,it use the debug one(as the description). Created attachment 964419 [details]
Log file of virt-v2v
I'm moving this to 7.2, because the guest doesn't fail to boot. What's actually happening here is that virt-v2v chooses the best kernel, and deliberately changes the default setting in grub's configuration to point to the best kernel. If you have two kernels with equal version, but one is non-debug and one is debug, then the sort ordering will favour the debug one just because kernel-2.6.32-496.el6 < kernel-debug-2.6.32-496.el6 The way to fix this is to modify the sort ordering slightly so that when the versions are equal, we favour the non-debug kernel. (In reply to Richard W.M. Jones from comment #5) > I'm moving this to 7.2, because the guest doesn't fail to boot. Actually the guest failed to boot with the debug kernel,refer to the screenshot. > What's actually happening here is that virt-v2v chooses the best > kernel, and deliberately changes the default setting in grub's > configuration to point to the best kernel. Yes,virt-v2v always pickup the latest kernel. Created attachment 964522 [details]
sceenshot of debug kernel fail to boot.
To reproduce this bug, I did: $ virt-builder rhel-6.6 --size=20G --root-password password:123456 I then booted this guest in KVM directly: $ qemu-kvm -hda rhel-6.6.img -m 4096 & Logged in as root, and did: # yum install kernel-debug This results in two kernels in /boot/grub/menu.lst, with default=1, essentially matching the description in comment 0. Doing a regular conversion switches the default=0, as described in this bug report. However the guest does still boot OK. I did not reproduce the boot failure (comment 6). When I applied the patch and re-ran virt-v2v, it now chooses the non-debug kernel (so default=1 after conversion). Boot is successful. v2 of upstream patch: https://www.redhat.com/archives/libguestfs/2014-December/msg00041.html Upstream fix: https://github.com/libguestfs/libguestfs/commit/c712f880db640163a0ce913e42115ecf4f6aa0c2 https://github.com/libguestfs/libguestfs/commit/0c2e676f6c92efce890006903585dabd1f72f3b1 I can reproduce this issue with package:
virt-v2v-1.28.1-1.14.el7.x86_64
libguestfs-1.28.1-1.14.el7.x86_64
Then try to verify this bug with new build:
virt-v2v-1.28.1-1.15.el7.x86_64
libguestfs-1.28.1-1.15.el7.x86_64
Steps:
1. Prepare a guest with 2 kernel installed(debug kernels & non-debug kernels), set the second kernel as default kernel,default=1.
kernel-2.6.32-496.el6.i686
kernel-debug-2.6.32-496.el6.i686
2.Grub file before conversion:
# cat /boot/grub/grub.conf
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.debug.img
title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.img
3.Use virt-v2v to convert the guest:
# virt-v2v -ic xen+ssh://10.66.106.64 -os default rhel6.6-i386-hvm-2kernel -of raw
[ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel6.6-i386-hvm-2kernel
[ 0.0] Creating an overlay to protect the source from being modified
[ 1.0] Opening the overlay
[ 5.0] Initializing the target -o libvirt -os default
[ 5.0] Inspecting the overlay
[ 10.0] Checking for sufficient free disk space in the guest
[ 10.0] Estimating space required on target for each disk
[ 10.0] Converting Red Hat Enterprise Linux Server release 6.6 Beta (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 44.0] Mapping filesystem data to avoid copying unused and blank areas
[ 45.0] Closing the overlay
[ 45.0] Copying disk 1/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-sda (raw)
(100.00/100%)
[ 108.0] Copying disk 2/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-sdb (raw)
(100.00/100%)
[ 230.0] Creating output metadata
Pool default refreshed
Domain rhel6.6-i386-hvm-2kernel defined from /tmp/v2vlibvirt5c11a0.xml
[ 230.0] Finishing off
4. Boot up guest and check.
Result1: Guest can boot up with non-debug kernel as default kernel automatically with no error(Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)), login in guest and check:
# uname -r
2.6.32-496.el6.i686
# cat /boot/grub/grub.conf
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.debug.img
title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.img
Result2: Guest will hang when choose debug-kernel by manual as attachment as Comment 7.
So rjones, please help me have a look about above results，now after conversion, kernel boot sequence is ok, but the debug-kernel also cannot work. Do you think it's ok to verify this bug, thanks.
No idea, but as long as it chooses the non-debug kernel and boots without user intervention, I think this bug can be moved to VERIFIED. (In reply to Richard W.M. Jones from comment #14) > No idea, but as long as it chooses the non-debug kernel and > boots without user intervention, I think this bug can be moved > to VERIFIED. Hi rjones, thanks for your reply first, and i also test another scenario: 1. Use same guest on xen server, but start guest and login guest change default boot kernel to debug-kernel: # uname -r 2.6.32-496.el6.i686.debug # cat /boot/grub/grub.conf default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug) root (hd0,0) kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-496.el6.i686.debug.img title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686) root (hd0,0) kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-496.el6.i686.img 2. Shut down guest and convert this guest to local kvm. # virt-v2v -ic xen+ssh://10.66.106.64 -os default rhel6.6-i386-hvm-2kernel -of raw -on rhel6.6-i386-hvm-2kernel-debug-kernel [ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel6.6-i386-hvm-2kernel [ 0.0] Creating an overlay to protect the source from being modified [ 1.0] Opening the overlay [ 5.0] Initializing the target -o libvirt -os default [ 5.0] Inspecting the overlay [ 11.0] Checking for sufficient free disk space in the guest [ 11.0] Estimating space required on target for each disk [ 11.0] Converting Red Hat Enterprise Linux Server release 6.6 Beta (Santiago) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 45.0] Mapping filesystem data to avoid copying unused and blank areas [ 45.0] Closing the overlay [ 45.0] Copying disk 1/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-debug-kernel-sda (raw) (100.00/100%) [ 109.0] Copying disk 2/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-debug-kernel-sdb (raw) (100.00/100%) [ 232.0] Creating output metadata Pool default refreshed Domain rhel6.6-i386-hvm-2kernel-debug-kernel defined from /tmp/v2vlibvirtbb6ed7.xml [ 232.0] Finishing off 3. After conversion, boot up guest: Result: Guest choose the first kernel (debug-kernel) as default boot kernel, and this is expected, but i will hang with debug-kernel as attached in Comment 7. And if you choose non-debug kernel, it will boot up. so rjones, i think there is also something wrong with xen guest with 2 kernels，especially the debug-kernel, please help check. what more, i also test the two scenario with a local rhel6 guest, after convert it to rhevm, both debug-kernel and non-debug kernel works well, thanks. (In reply to zhoujunqin from comment #15) > # cat /boot/grub/grub.conf > title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug) > title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686) > Result: Guest choose the first kernel (debug-kernel) as default boot kernel, > and this is expected, but i will hang with debug-kernel as attached in > Comment 7. I would have expected it to choose the non-debug kernel in this case. What's the virt-v2v -v -x output? Also I wonder why these debug kernels don't boot. It sounds like they are broken or there is another kernel bug there. Created attachment 966738 [details]
rhel6.6-i386-hvm-2kernel-debug-kernel_debug.log
Indeed there was a further bug which prevented it from picking any kernel other than the first one in the grub file. Fix: https://github.com/libguestfs/libguestfs/commit/e1fd9615cc4a9ceb68f4a47a289712fc31a0af5e Try to verify it with new build:
libguestfs-1.28.1-1.17.el7.x86_64
virt-v2v-1.28.1-1.17.el7.x86_64
Scenario 1:
1. Use same guest on xen server, but start guest and login guest change default boot kernel to debug-kernel:
# uname -r
2.6.32-496.el6.i686.debug
# cat /boot/grub/grub.conf
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.debug.img
title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.img
2. Shut down guest and convert this guest to local kvm.
# virt-v2v -ic xen+ssh://10.66.106.64 -os default rhel6.6-i386-hvm-2kernel -of raw -on rhel6.6-i386-hvm-2kernel-debug-kernel
[ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel6.6-i386-hvm-2kernel
[ 1.0] Creating an overlay to protect the source from being modified
[ 2.0] Opening the overlay
[ 9.0] Initializing the target -o libvirt -os default
[ 9.0] Inspecting the overlay
[ 15.0] Checking for sufficient free disk space in the guest
[ 15.0] Estimating space required on target for each disk
[ 15.0] Converting Red Hat Enterprise Linux Server release 6.6 Beta (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 49.0] Mapping filesystem data to avoid copying unused and blank areas
[ 50.0] Closing the overlay
[ 50.0] Copying disk 1/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-debug-kernel-sda (raw)
(100.00/100%)
[ 112.0] Copying disk 2/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-debug-kernel-sdb (raw)
(100.00/100%)
[ 232.0] Creating output metadata
Pool default refreshed
Domain rhel6.6-i386-hvm-2kernel-debug-kernel defined from /tmp/v2vlibvirt01c60c.xml
[ 232.0] Finishing off
3. Boot up guest.
Result: when boot, guest choose non-debug kernel as default boot kernel and can boot up with no error.
Scenario 2
1. Prepare a guest with 2 kernel installed(debug kernels & non-debug kernels), set the second kernel as default kernel,default=1.
kernel-2.6.32-496.el6.i686
kernel-debug-2.6.32-496.el6.i686
2.Grub file before conversion:
# cat /boot/grub/grub.conf
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.debug.img
title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)
root (hd0,0)
kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-496.el6.i686.img
3.Use virt-v2v to convert the guest:
# virt-v2v -ic xen+ssh://10.66.106.64 -os default rhel6.6-i386-hvm-2kernel -of raw -on rhel6.6-i386-hvm-2kernel-nondebug-kernel
[ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel6.6-i386-hvm-2kernel
[ 1.0] Creating an overlay to protect the source from being modified
[ 1.0] Opening the overlay
[ 6.0] Initializing the target -o libvirt -os default
[ 6.0] Inspecting the overlay
[ 11.0] Checking for sufficient free disk space in the guest
[ 11.0] Estimating space required on target for each disk
[ 11.0] Converting Red Hat Enterprise Linux Server release 6.6 Beta (Santiago) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 47.0] Mapping filesystem data to avoid copying unused and blank areas
[ 48.0] Closing the overlay
[ 48.0] Copying disk 1/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-nondebug-kernel-sda (raw)
(100.00/100%)
[ 116.0] Copying disk 2/2 to /var/lib/libvirt/images/rhel6.6-i386-hvm-2kernel-nondebug-kernel-sdb (raw)
(100.00/100%)
[ 248.0] Creating output metadata
Pool default refreshed
Domain rhel6.6-i386-hvm-2kernel-nondebug-kernel defined from /tmp/v2vlibvirt0d43b4.xml
[ 248.0] Finishing off
Result1: Guest can boot up with non-debug kernel as default kernel automatically with no error(Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686)).
But debug-kernel can not use in above two Scenarios.
Since this bug issue has been fixed, move this bug to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0303.html |
Description virt-v2v always use the first kernel (0th kernel) instead of the default kernel (ex: default=1) refer to old version of virt-v2v bug:829859 Version: virt-v2v-1.28.1-1.14.el7.x86_64 libguestfs-1.28.1-1.14.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Prepare a guest with 2 kernel installed,set the second kernel as default kernel,default=1. 2.Grub file before conversion: # cat /etc/grub.conf default=1 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.32-496.el6.i686.debug) root (hd0,0) kernel /vmlinuz-2.6.32-496.el6.i686.debug ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-496.el6.i686.debug.img title Red Hat Enterprise Linux 6 (2.6.32-496.el6.i686) root (hd0,0) kernel /vmlinuz-2.6.32-496.el6.i686 ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-496.el6.i686.img # uname -r 2.6.32-496.el6.x86_64 3.Use virt-v2v to convert the guest: # virt-v2v -ic xen+ssh://10.66.106.64 -os test rhel6.6-i386-hvm -of raw [ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel6.6-i386-hvm [ 17.0] Creating an overlay to protect the source from being modified [ 47.0] Opening the overlay [ 83.0] Initializing the target -o libvirt -os test [ 83.0] Inspecting the overlay [ 91.0] Checking for sufficient free disk space in the guest [ 91.0] Estimating space required on target for each disk [ 91.0] Converting Red Hat Enterprise Linux Server release 6.6 Beta (Santiago) to run on KVM virt-v2v: This guest has virtio drivers installed. [ 149.0] Mapping filesystem data to avoid copying unused and blank areas [ 149.0] Closing the overlay [ 149.0] Copying disk 1/2 to /var/lib/libvirt/images/test/rhel6.6-i386-hvm-sda (raw) (100.00/100%) [ 231.0] Copying disk 2/2 to /var/lib/libvirt/images/test/rhel6.6-i386-hvm-sdb (raw) (100.00/100%) [ 370.0] Creating output metadata Pool test refreshed Domain rhel6.6-i386-hvm defined from /tmp/v2vlibvirt4d721a.xml [ 370.0] Finishing off 4.After conversion,check the boot kernel is the first kernel:2.6.32-496.el6.i686.debug instead of the default kernel. Actual results： virt-v2v use the first kernel instead of default kernel to create new converted guest. Expected results: virt-v2v should use the default kernel to create new converted guest. Additional info: