Bug 1739106 - [RFE] Compare storage with database for discrepancies
Summary: [RFE] Compare storage with database for discrepancies
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhv-log-collector-analyzer
Version: 4.3.5
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.3.8
: ---
Assignee: Benny Zlotnik
QA Contact: Petr Kubica
URL:
Whiteboard:
: 1489145 (view as bug list)
Depends On:
Blocks: 902971 1489145 1547336
TreeView+ depends on / blocked
 
Reported: 2019-08-08 13:44 UTC by Marina Kalinin
Modified: 2024-12-20 18:52 UTC (History)
12 users (show)

Fixed In Version: rhv-log-collector-analyzer-0.2.15-0.el7ev
Doc Type: Enhancement
Doc Text:
This release adds a new feature, rhv-image-discrepancies, which reports discrepancies between images in the engine database and storage. The report lists images that are present in one location but missing from the other. Also, for images that are present in both locations, the report lists discrepancies in the values of attributes such as status, parent_id, and type.
Clone Of:
Environment:
Last Closed: 2020-02-13 15:24:47 UTC
oVirt Team: Integration
Target Upstream Version:
Embargoed:
lsvaty: testing_plan_complete-


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Article) 3740351 0 None None None 2020-02-19 21:58:24 UTC
Red Hat Product Errata RHSA-2020:0498 0 None None None 2020-02-13 15:25:11 UTC
oVirt gerrit 96154 0 'None' ABANDONED WIP: contrib: add cross_check_sd script 2020-09-25 12:58:12 UTC

Description Marina Kalinin 2019-08-08 13:44:34 UTC
We need a tool to detect discrepancy between the database and the actual storage. A mismatch between the two can happen due to various reasons, and even if a bug that caused it is fixed, the discrepancy is still there and we need a way to find those discrepancies to avoid future problems, like broken snapshots (most common scenario, but not only).

How this can be implemented:
- Today, vdsm-tool already provides storage contents from SPM in an sqlite format[1]. 
- We have the database on the other hand available. 
- Now we just need to compare the two databases.

Outcome: 
In log collector analyzer we will have this output and next step would be to have this also be an insight rule.

Existing RFEs:
bz#1580130 - [RFE] Need an FSCK-like tool to find and clean up inconsistencies between the database and storage domains
This RFE here comes with a more narrow specific request to implement the missing step.


[1] vdsm-tool dump-volume-chains <sd_uuid> -o sqlite -f out.db
https://gerrit.ovirt.org/#/c/89150/

Comment 5 Michal Skrivanek 2019-11-19 14:59:50 UTC
rhv-log-collector-analyzer-0.2.14-0.el7ev

Comment 6 Petr Kubica 2019-11-28 12:50:46 UTC
Hi,
I would like to ask how to use this new feature?

I used "rhv-log-collector-analyzer --live" which generates a html report but there was nothing about comparing storage with database for discrepancies.

It seems like the patch is not included in build (according to comment #5)
tested with rhv-log-collector-analyzer-0.2.14-0.el7ev.noarch

# rpm -ql rhv-log-collector-analyzer | grep rhv-image-discrepancies.py
or
# find / -name "rhv-image-discrepancies.py"
didn't find anything

Comment 7 Benny Zlotnik 2019-11-28 13:29:09 UTC
If I am not mistaken, it should be in under /bin so ./rhv-image-discrepancies should work

Comment 8 Petr Kubica 2019-11-28 13:41:15 UTC
Ah, yes, you are right.

And I tried to run the utility, got:
# rhv-image-discrepancies 
Traceback (most recent call last):
  File "/usr/bin/rhv-image-discrepancies", line 22, in <module>
    import engine_db_query as db
ModuleNotFoundError: No module named 'engine_db_query'

Comment 9 Benny Zlotnik 2019-12-01 10:14:02 UTC
Douglas, shouldn't engine_db_query be available? It's a dependency of rhv-log-collector-analyzer

Comment 10 Sandro Bonazzola 2019-12-02 08:29:49 UTC
Not blocking 4.3.7 on this, re-targeting to 4.3.8.

Comment 11 Douglas Schilling Landgraf 2019-12-03 14:39:49 UTC
(In reply to Petr Kubica from comment #8)
> Ah, yes, you are right.
> 
> And I tried to run the utility, got:
> # rhv-image-discrepancies 
> Traceback (most recent call last):
>   File "/usr/bin/rhv-image-discrepancies", line 22, in <module>
>     import engine_db_query as db
> ModuleNotFoundError: No module named 'engine_db_query'

Hi Petr, do you have the machine available so I can take a look?

Thanks!

Comment 12 Douglas Schilling Landgraf 2019-12-05 18:36:01 UTC
(In reply to Douglas Schilling Landgraf from comment #11)
> (In reply to Petr Kubica from comment #8)
> > Ah, yes, you are right.
> > 
> > And I tried to run the utility, got:
> > # rhv-image-discrepancies 
> > Traceback (most recent call last):
> >   File "/usr/bin/rhv-image-discrepancies", line 22, in <module>
> >     import engine_db_query as db
> > ModuleNotFoundError: No module named 'engine_db_query'
> 
> Hi Petr, do you have the machine available so I can take a look?
> 
> Thanks!

Just reproduced the report. The rhv-image-discrepancies script is python3 hard-coded.

# cat /usr/bin/rhv-image-discrepancies | grep python3
#!/usr/bin/python3

However, engine-db-query is python2 only at this moment.

Solution:

1. For now, I would replace in rhv-image-discrepancies #!/usr/bin/python3 to #!/usr/bin/env python
2. Open a bug to rhv-log-collector-analyzer and engine-db-query to port missing bits to python3 and build a new rpm.

Just a note, the default python in RHEL 7.7 is still python2. I would target such builds in python3 to RHEL8 based distro.

# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.7 (Maipo)

# ls -la /usr/bin/python
lrwxrwxrwx. 1 root root 7 Dec  5 10:56 /usr/bin/python -> python2

Comment 20 Petr Kubica 2020-01-23 15:40:31 UTC
Verified in rhv-log-collector-analyzer-0.2.15-0.el7ev.noarch

Comment 24 errata-xmlrpc 2020-02-13 15:24:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:0498

Comment 28 Marina Kalinin 2020-02-20 20:23:04 UTC
*** Bug 1489145 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.