Bug 979825

Summary:	Log collector engine sos plugin should collect DB schema
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Eli Mesika <emesika>
Component:	ovirt-engine-log-collector	Assignee:	Sandro Bonazzola <sbonazzo>
Status:	CLOSED WONTFIX	QA Contact:	Pavel Stehlik <pstehlik>
Severity:	low	Docs Contact:
Priority:	medium
Version:	3.2.0	CC:	acathrow, emesika, iheim, jkt, knesenko, kroberts, oschreib, Rhev-m-bugs, sbonazzo
Target Milestone:	---	Keywords:	Improvement, Triaged
Target Release:	3.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	integration
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-10-28 12:13:42 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1019461

Description Eli Mesika 2013-06-30 21:57:11 UTC

Description of problem:

We have lot of cases when an upgraded database ahs some problems.
Collecting the database schema will enable us to take the generated file, create a local database with the same last version as we will find in the given schema and compare both to find the problems.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:
Log collector does not collect db schema

Expected results:

Log collector should collect db schema

Additional info:

Log Collector should use the following utility (located @ dbscripts)

Usage: exportDbSchema.sh [-h] [-s SERVERNAME] [-d DATABASE] [-u USERNAME] [-v]

 -s SERVERNAME - The database servername for the database (def. localhost)
 -d DATABASE   - The database name                        (def. engine)
 -u USERNAME   - The username for the database            (def. engine)
 -l LOGFILE    - The logfile for capturing output         (def. exportDbSchema.sh.log
 -v            - Turn on verbosity (WARNING: lots of output)
 -h            - This help text.

Comment 1 Sandro Bonazzola 2013-09-19 13:24:26 UTC

(In reply to Eli Mesika from comment #0)
> Description of problem:
> 
> We have lot of cases when an upgraded database ahs some problems.
> Collecting the database schema will enable us to take the generated file,
> create a local database with the same last version as we will find in the
> given schema and compare both to find the problems.

Just to be sure I've understood correctly the request here:
you need the version of the schema stored in a file right?

Because the dump of the database is already done by postgresql sos plugin,
it's in log-collector-data/postgresql-sosreport-*.tar.xz stored in tar format.

Comment 2 Eli Mesika 2013-09-21 20:32:01 UTC

(In reply to Sandro Bonazzola from comment #1)
> (In reply to Eli Mesika from comment #0)
> > Description of problem:
> > 
> > We have lot of cases when an upgraded database ahs some problems.
> > Collecting the database schema will enable us to take the generated file,
> > create a local database with the same last version as we will find in the
> > given schema and compare both to find the problems.
> 
> Just to be sure I've understood correctly the request here:
> you need the version of the schema stored in a file right?
> 

No, I need the schema itself to be able to compare it with a database having the same version and compare the differences easily 

> Because the dump of the database is already done by postgresql sos plugin,
> it's in log-collector-data/postgresql-sosreport-*.tar.xz stored in tar
> format.

Comment 3 Sandro Bonazzola 2013-10-25 13:20:25 UTC

(In reply to Eli Mesika from comment #0)

> Actual results:
> Log collector does not collect db schema
> 
> Expected results:
> Log collector should collect db schema

Eli, the bug description asks to add db schema collection to engine sos plugin.
But the expected results just require the schema to be collected by log-collector, not also by 'sos -o engine'.

Is it desired to have the schema collected by sos? If not, I can just change log-collector without touching the sos plugin.

Keith, maybe this can be useful also for GSS plugin.

Comment 4 Keith Robertson 2013-10-25 15:08:09 UTC

(In reply to Sandro Bonazzola from comment #3)
> (In reply to Eli Mesika from comment #0)
> 
> > Actual results:
> > Log collector does not collect db schema
> > 
> > Expected results:
> > Log collector should collect db schema
> 
> Eli, the bug description asks to add db schema collection to engine sos
> plugin.
> But the expected results just require the schema to be collected by
> log-collector, not also by 'sos -o engine'.
> 
> Is it desired to have the schema collected by sos? If not, I can just change
> log-collector without touching the sos plugin.
> 
> Keith, maybe this can be useful also for GSS plugin.

AFAIK, when you dump a DB via psql it will include the schema (it must).  My question here is why can't you...

1) Load the dumped DB supplied via the LC via pg_restore
2) Inspect the newly loaded DB's schema via pgadmin3, or export it and diff the text with a previous version

Comment 5 Eli Mesika 2013-10-26 20:48:20 UTC

(In reply to Keith Robertson from comment #4)
> (In reply to Sandro Bonazzola from comment #3)
> > (In reply to Eli Mesika from comment #0)
> > 
> > > Actual results:
> > > Log collector does not collect db schema
> > > 
> > > Expected results:
> > > Log collector should collect db schema
> > 
> > Eli, the bug description asks to add db schema collection to engine sos
> > plugin.
> > But the expected results just require the schema to be collected by
> > log-collector, not also by 'sos -o engine'.
> > 
> > Is it desired to have the schema collected by sos? If not, I can just change
> > log-collector without touching the sos plugin.
> > 
> > Keith, maybe this can be useful also for GSS plugin.
> 
> AFAIK, when you dump a DB via psql it will include the schema (it must).  My
> question here is why can't you...
> 
> 1) Load the dumped DB supplied via the LC via pg_restore
> 2) Inspect the newly loaded DB's schema via pgadmin3, or export it and diff
> the text with a previous version

Because restore can be a heavy process on large installtions that will require storage allocations and time to complete while this simple scheam generation will cost us nothing and will always be there for our usage

Comment 6 Keith Robertson 2013-10-28 01:45:08 UTC

(In reply to Eli Mesika from comment #5)
> (In reply to Keith Robertson from comment #4)
> > (In reply to Sandro Bonazzola from comment #3)

snip

> > AFAIK, when you dump a DB via psql it will include the schema (it must).  My
> > question here is why can't you...
> > 
> > 1) Load the dumped DB supplied via the LC via pg_restore
> > 2) Inspect the newly loaded DB's schema via pgadmin3, or export it and diff
> > the text with a previous version
> 
> Because restore can be a heavy process on large installtions that will
> require storage allocations and time to complete while this simple scheam
> generation will cost us nothing and will always be there for our usage

I have no problem adding this but you should know that pg_restore has a "--schema-only" option[1] which allows reconstitution of the db without data.  Would this work?

[1] http://www.postgresql.org/docs/9.2/static/app-pgrestore.html

Comment 7 Eli Mesika 2013-10-28 09:17:36 UTC

(In reply to Keith Robertson from comment #6)
> (In reply to Eli Mesika from comment #5)
> > (In reply to Keith Robertson from comment #4)
> > > (In reply to Sandro Bonazzola from comment #3)
> 
> snip
> 
> > > AFAIK, when you dump a DB via psql it will include the schema (it must).  My
> > > question here is why can't you...
> > > 
> > > 1) Load the dumped DB supplied via the LC via pg_restore
> > > 2) Inspect the newly loaded DB's schema via pgadmin3, or export it and diff
> > > the text with a previous version
> > 
> > Because restore can be a heavy process on large installtions that will
> > require storage allocations and time to complete while this simple scheam
> > generation will cost us nothing and will always be there for our usage
> 
> I have no problem adding this but you should know that pg_restore has a
> "--schema-only" option[1] which allows reconstitution of the db without
> data.  Would this work?

Yes , it will work , I still think that having that ready in plain-text will save us time and I will be happy to skip the step of creating an empty DB and use pg_restore jsut to know what scripts were installed on the database 

> 
> [1] http://www.postgresql.org/docs/9.2/static/app-pgrestore.html

Comment 8 Sandro Bonazzola 2013-10-28 10:54:23 UTC

So, is the execution of exportDbSchema inside the sos plugin still needed?
May it be done by log-collector instead of the sos plugin?
Should we just close this as wontfix since pgrestore is able to do what needed with the db dump already provided by postgres sos plugin?

Comment 9 Eli Mesika 2013-10-28 12:00:37 UTC

(In reply to Sandro Bonazzola from comment #8)
> So, is the execution of exportDbSchema inside the sos plugin still needed?
> May it be done by log-collector instead of the sos plugin?
> Should we just close this as wontfix since pgrestore is able to do what
> needed with the db dump already provided by postgres sos plugin?

You may close as WONTFIX

Comment 10 Sandro Bonazzola 2013-10-28 12:13:42 UTC

(In reply to Eli Mesika from comment #9)
> (In reply to Sandro Bonazzola from comment #8)
> > So, is the execution of exportDbSchema inside the sos plugin still needed?
> > May it be done by log-collector instead of the sos plugin?
> > Should we just close this as wontfix since pgrestore is able to do what
> > needed with the db dump already provided by postgres sos plugin?
> 
> You may close as WONTFIX

Done