Bug 2032043 - Upgrading from RHV-H 4.4.8 to 4.4.9 fails with nerror: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch\nerror: cannot open Pac
Summary: Upgrading from RHV-H 4.4.8 to 4.4.9 fails with nerror: db5 error(-30969) from...
Keywords:
Status: CLOSED DUPLICATE of bug 1930550
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: redhat-virtualization-host
Version: 4.4.9
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Yedidyah Bar David
QA Contact: peyu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-13 22:25 UTC by amashah
Modified: 2023-09-15 01:50 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-04-08 16:15:01 UTC
oVirt Team: Node
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-44361 0 None None None 2021-12-13 22:27:24 UTC
Red Hat Knowledge Base (Solution) 6586801 0 None None None 2021-12-15 19:12:27 UTC

Description amashah 2021-12-13 22:25:31 UTC
Description of problem:
When upgrading RHV-H from 4.4.8 to 4.4.9, the rpm gets installed but one of the scripts in the installation fails with:

~~~
# yum reinstall redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64
Updating Subscription Management repositories.
Red Hat Virtualization Host for RHEL 8 x86_64 (RPMs)                                     8.7 MB/s | 1.5 MB     00:00
Dependencies resolved.
=========================================================================================================================
 Package                                    Arch      Version                     Repository                        Size
=========================================================================================================================
Reinstalling:
 redhat-virtualization-host-image-update    x86_64    4.4.9-202111172338_8.5      rhvh-4-for-rhel-8-x86_64-rpms    995 M

Transaction Summary
=========================================================================================================================

Total download size: 995 M
Installed size: 994 M
Is this ok [y/N]: y
Downloading Packages:
redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm                122 MB/s | 995 MB     00:08
-------------------------------------------------------------------------------------------------------------------------
Total                                                                                    122 MB/s | 995 MB     00:08
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                 1/1
  Running scriptlet: redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           1/2
  Reinstalling     : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           1/2
  Running scriptlet: redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           1/2
warning: %post(redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64) scriptlet failed, exit status 1

Error in POSTIN scriptlet in rpm package redhat-virtualization-host-image-update
  Cleanup          : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           2/2
  Verifying        : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           1/2
  Verifying        : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                           2/2
Installed products updated.
Unpersisting: redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm

Reinstalled:
  redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64
~~~


In reviewing the imgbased.log we can see:

~~~
2021-12-13 09:39:57,511 [DEBUG] (MainThread) Calling: (['rpm', '-U', '--quiet', '--justdb', '--root', '/tmp/mnt.esQlr//', '/var/cache/dnf/rhvh-4-for-rhel-8-x86_64-rpms-3b2457f6cf900c31/packages/redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm'],) {'close_fds': True, 'stderr': -2}
2021-12-13 09:39:57,535 [DEBUG] (MainThread) Exception! b'warning: /var/cache/dnf/rhvh-4-for-rhel-8-x86_64-rpms-3b2457f6cf900c31/packages/redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY\nerror: db5 error(-30969) from dbenv->open: BDB0091 DB_VERSION_MISMATCH: Database environment version mismatch\nerror: cannot open Packages index using db5 -  (-30969)\nerror: cannot open Packages database in /tmp/mnt.esQlr/var/lib/rpm\n'
~~~

So it seems that we mount the on this /tmp/mnt.esQlr mount point the 4.4.9 image:

~~~
2021-12-13 09:35:27,155 [DEBUG] (MainThread) Calling: (['lvs', '--noheadings', '--ignoreskippedcluster', '-olv_path', 'rhvh/rhvh-4.4.9.2-0.20211117.0+1'],) {'stderr': <_io.TextIOWrapper name='/dev/null' mode
='w' encoding='UTF-8'>, 'close_fds': True}
2021-12-13 09:35:27,231 [DEBUG] (MainThread) Returned: b'/dev/rhvh/rhvh-4.4.9.2-0.20211117.0+1'
2021-12-13 09:35:27,231 [DEBUG] (MainThread) Calling: (['mktemp', '-d', '--tmpdir', 'mnt.XXXXX'],) {'close_fds': True, 'stderr': -2}
2021-12-13 09:35:27,234 [DEBUG] (MainThread) Returned: b'/tmp/mnt.esQlr'
2021-12-13 09:35:27,234 [DEBUG] (MainThread) Calling: (['mount', '/dev/rhvh/rhvh-4.4.9.2-0.20211117.0+1', '/tmp/mnt.esQlr'],) {'close_fds': True, 'stderr': -2}
~~~


Then inside the /tmp/mnt.esQlr/var/lib/rpm we get the error above about cannot open the Packages database.




Version-Release number of selected component (if applicable):
4.4.9

How reproducible:
So far seen in two support cases

Steps to Reproduce:
1. Update RHV-H 4.4.8 to RHV-H 4.4.9 from UI or CLI

Actual results:
The post scriptlet fails when checking the /mnt/tmp.XXXX/var/lib/rpm rpm database, with db5 error.

Expected results:
The upgrade should complete as expected

Additional info:
Logs/sosreport will be uploaded shortly.

Comment 2 peyu 2021-12-14 06:47:20 UTC
QE did not reproduce this bug.

Test Version:
RHVH: upgrade RHVH from rhvh-4.4.8.1-0.20210903.0+1 to rhvh-4.4.9.2-0.20211117.0+1
RHVM: 4.4.10-0.17.el8ev

Test Steps:
1. Install redhat-virtualization-host-4.4.8-20210903.0.el8_4
2. Download "redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm"
3. Add RHVH to RHVM
4. Set RHVH to "Maintenance" in RHVM
5. Upgrade RHVH via CLI
   # yum install redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm 

Test Result:
The upgrade was successful, there are no db5 related errors in /var/log/imgbased.log
 
~~~~~~
# yum install redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64.rpm 
Updating Subscription Management repositories.
Unable to read consumer identity

This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.

Dependencies resolved.
==============================================================================================================================
 Package                                          Architecture    Version                         Repository             Size
==============================================================================================================================
Installing:
 redhat-virtualization-host-image-update          x86_64          4.4.9-202111172338_8.5          @commandline          995 M
     replacing  redhat-virtualization-host-image-update-placeholder.noarch 4.4.8-1.el8ev

Transaction Summary
==============================================================================================================================
Install  1 Package

Total size: 995 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                      1/1 
  Running scriptlet: redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                                1/2 
  Installing       : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                                1/2 
  Running scriptlet: redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                                1/2 
  Obsoleting       : redhat-virtualization-host-image-update-placeholder-4.4.8-1.el8ev.noarch                             2/2 
  Verifying        : redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                                1/2 
  Verifying        : redhat-virtualization-host-image-update-placeholder-4.4.8-1.el8ev.noarch                             2/2 
Unpersisting: redhat-virtualization-host-image-update-placeholder-4.4.8-1.el8ev.noarch.rpm
Installed products updated.

Installed:
  redhat-virtualization-host-image-update-4.4.9-202111172338_8.5.x86_64                                                       

Complete!
~~~~~~


But when RHVH was upgraded from 4.4.8 to 4.4.9, we had a known db5 related bug: Bug 1930550 
I don’t know if the root causes of this bug is the same as that of https://bugzilla.redhat.com/show_bug.cgi?id=1930550#c14, but they are all db5 related errors.

Comment 5 Sandro Bonazzola 2021-12-16 08:15:11 UTC
Reducing severity as rebuilding the rpmdb as workaround fixes the issues.

Comment 6 Marina Kalinin 2021-12-16 16:21:05 UTC
(In reply to Sandro Bonazzola from comment #5)
> Reducing severity as rebuilding the rpmdb as workaround fixes the issues.

Well, it still fails the upgrade. So I think it should be high. Sorry about that.

Comment 10 amashah 2022-04-08 16:15:01 UTC

*** This bug has been marked as a duplicate of bug 1930550 ***

Comment 11 Red Hat Bugzilla 2023-09-15 01:50:33 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 365 days


Note You need to log in before you can comment on or make changes to this bug.