Created attachment 938770 [details] Debug info about rpmdb error Description virt-v2v fails with error: cannot open Packages index using db5 Version: libguestfs-1.27.48-1.1.el7.x86_64 virt-v2v-1.27.48-1.1.el7.x86_64 How reproducible: 50% Steps to Reproduce: 1.Use virt-v2v to convert some guests,met the below error info. #virt-v2v -o rhev -os 10.66.90.115:/vol/v2v_auto/auto_export --network rhevm rhel7-rc-graphics -v -x |& tee /tmp/v2v-rpmdb.log virt-v2v: error: libguestfs error: command_lines: error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages database in /var/lib/rpm If reporting bugs, run virt-v2v with debugging enabled and include the complete output: virt-v2v -v -x [...] Actual results： virt-v2v fails with error: cannot open Packages index using db5 Expected results: virt-v2v should convert guests successfully without such error info. Additional info: The above issue can not be reproduced every time,sometimes I met similar error: # virt-v2v -ic xen+ssh://10.66.106.64 -os default xen-pv-rhel5.8-i386 [ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 xen-pv-rhel5.8-i386 [ 16.0] Creating an overlay to protect the source from being modified [ 31.0] Opening the overlay [ 103.0] Initializing the target -o libvirt -os default [ 103.0] Inspecting the overlay [ 116.0] Checking for sufficient free disk space in the guest [ 116.0] Estimating space required on target for each disk [ 116.0] Converting Red Hat Enterprise Linux Server release 5.8 (Tikanga) to run on KVM virt-v2v: error: libguestfs error: command_lines: rpmdb: Program version 4.3 doesn't match environment version error: db4 error(-30974) from dbenv->open: DB_VERSION_MISMATCH: Database environment version mismatch error: cannot open Packages index using db3 - (-30974) error: cannot open Packages database in /var/lib/rpm If reporting bugs, run virt-v2v with debugging enabled and include the complete output: virt-v2v -v -x [...]
The error is that we run a very simple rpm command in the guest and it fails rather badly: # rpm -ql kernel error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages database in /var/lib/rpm Is the rhel7-rc-graphics guest OK? Obviously it cannot be running because we check for that. You could try booting up the guest and running various simple rpm commands like 'rpm -ql kernel' and 'rpm -qa' and just check that the RPM database really isn't corrupt.
(In reply to Richard W.M. Jones from comment #2) > The error is that we run a very simple rpm command in the guest > and it fails rather badly: > > # rpm -ql kernel > error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 > Thread died in Berkeley DB library > error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal > error, run database recovery > error: cannot open Packages index using db5 - (-30973) > error: cannot open Packages database in /var/lib/rpm > error: rpmdb: BDB0113 Thread/process 12680/139878611654464 failed: BDB1507 > Thread died in Berkeley DB library > error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal > error, run database recovery > error: cannot open Packages database in /var/lib/rpm > > Is the rhel7-rc-graphics guest OK? Obviously it cannot be > running because we check for that. Yes,it is ok,after conversion,I can boot it successfully. > You could try booting up the guest and running various simple > rpm commands like 'rpm -ql kernel' and 'rpm -qa' and just check > that the RPM database really isn't corrupt. I tried these commands,it worked fine. btw,I can not reproduce this error everytime for the same guest,there is a rhel6 bug 806714 in old v2v for reference.
I think I see what's going on. I have pushed these patches which I hope will fix things (virt-v2v >= 1.27.50): https://github.com/libguestfs/libguestfs/commit/6db2ff52f9557bc3644700386320dc7a3fa54830 https://github.com/libguestfs/libguestfs/commit/4aa38bba2ca77cf0ff7271a8f419182c57f59f7a
Reproduce this bug with: libguestfs-1.27.49-1.1.el7.x86_64 virt-v2v-1.27.49-1.1.el7.x86_64 1.Prepare a guest,do some operations to make the rpmdb corrupted. # yum install libvirt error: rpmdb: BDB0113 Thread/process 3350/140483719948096 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm CRITICAL:yum.main: Error: rpmdb open failed 2.Use virt-v2v to convert the guest,error shows: # virt-v2v -ic xen+ssh://10.66.106.64 -o rhev -os 10.66.90.115:/vol/v2v_auto/auto_export --network rhevm rhel7-rc-graphics -of raw [ 0.0] Opening the source -i libvirt -ic xen+ssh://10.66.106.64 rhel7-rc-graphics [ 16.0] Creating an overlay to protect the source from being modified [ 30.0] Opening the overlay [ 230.0] Initializing the target -o rhev -os 10.66.90.115:/vol/v2v_auto/auto_export [ 230.0] Inspecting the overlay [ 245.0] Checking for sufficient free disk space in the guest [ 245.0] Estimating space required on target for each disk [ 245.0] Converting Red Hat Enterprise Linux Server release 7.0 (Maipo) to run on KVM virt-v2v: error: libguestfs error: command_lines: error: rpmdb: BDB0113 Thread/process 3350/140483719948096 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages index using db5 - (-30973) error: cannot open Packages database in /var/lib/rpm error: rpmdb: BDB0113 Thread/process 3350/140483719948096 failed: BDB1507 Thread died in Berkeley DB library error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery error: cannot open Packages database in /var/lib/rpm If reporting bugs, run virt-v2v with debugging enabled and include the complete output: virt-v2v -v -x [...] Tested with latest virt-v2v version: libguestfs-1.27.50-1.1.el7.x86_64 virt-v2v-1.27.50-1.1.el7.x86_64 Repeat the above steps,no such error shows. Refer to the above comments,move the bug to VERIFIED.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-0303.html