Bug 1658509

Summary:	ovirt-log-collector-analyzer doesn't work with PG10
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Lucie Leistnerova <lleistne>
Component:	rhv-log-collector-analyzer	Assignee:	Douglas Schilling Landgraf <dougsland>
Status:	CLOSED ERRATA	QA Contact:	Ivana Saranova <isaranov>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.3.0	CC:	dougsland, isaranov, mtessun, sbonazzo
Target Milestone:	ovirt-4.3.6	Keywords:	AutomationBlocker, FieldEngineering, ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-10-10 15:37:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	External	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1673784
Bug Blocks:	1642318, 1673299

Description Lucie Leistnerova 2018-12-12 10:24:00 UTC

Description of problem:
Script ovirt-log-collector-analyzer fails on tar error. But it is related to scl PG10.

Version-Release number of selected component (if applicable):
ovirt-log-collector-analyzer-4.3.0-0.0.master.20180828120238.git316ba14.el7.noarch
rhv-log-collector-analyzer-0.1.7-0.el7ev.noarch

How reproducible: always


Steps to Reproduce:
1. run ovirt-log-collector
2. add read permissions on created sosreport for postgres user
3. run analyzer with postgres user
e.g.:
$ su - postgres -c "cd /tmp; ovirt-log-collector-analyzer /tmp/sosreport-LogCollector-20181212110314.tar.xz"


Actual results:
Unpacking postgres data. This can take up to several minutes.
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors


Expected results:
no error, analyzer creates report


Additional info:
The main problem is in this command:

/usr/share/ovirt-log-collector/analyzer/unpackAndPrepareDump.sh line 97

tar -Oxf "/tmp/tmp.PgmOSjb4Vj/unpacked_sosreport/sosreport-LogCollector-20181212100955/log-collector-data/postgresql-sosreport-system-ge-4-2018-12-12-qmkputf.tar.xz" "sosreport-system-ge-4-2018-12-12-qmkputf/sos_commands/postgresql/pgdump-scl-rh-postgresql95.tar"

it fails on pg_dump version mismatch and so no tar file goes to piped tar command.

I tested it on upgraded 4.2 -> 4.3 engine and also clean 4.3 d/s build.

Comment 4 Ivana Saranova 2019-01-30 15:37:23 UTC

Steps to Reproduce:
1. run ovirt-log-collector
2. add read permissions on created sosreport for postgres user
3. run analyzer with postgres user

    su - postgres -c "cd /tmp; rhv-log-collector-analyzer /tmp/sosreport-LogCollector-20190130135549.tar.xz"


Results:

Preparing environment:
======================
Temporary working directory is /tmp/tmp.04WrPttIEC
Unpacking postgres data. This can take up to several minutes.
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors
rm: cannot remove ‘/tmp/ovirt-log-collector-analyzer-hosts’: Operation not permitted


Expected results:
No errors, report created

Tested in: 
rhv-log-collector-analyzer-0.2.3-0.el7ev.noarch
ovirt-log-collector-4.3.0-1.el7ev.noarch

Comment 5 Douglas Schilling Landgraf 2019-01-30 16:02:38 UTC

(In reply to Ivana Saranova from comment #4)
> Steps to Reproduce:
> 1. run ovirt-log-collector
> 2. add read permissions on created sosreport for postgres user
> 3. run analyzer with postgres user
> 
>     su - postgres -c "cd /tmp; rhv-log-collector-analyzer
> /tmp/sosreport-LogCollector-20190130135549.tar.xz"
> 
> 
> Results:
> 
> Preparing environment:
> ======================
> Temporary working directory is /tmp/tmp.04WrPttIEC
> Unpacking postgres data. This can take up to several minutes.
> tar: This does not look like a tar archive
> tar: Exiting with failure status due to previous errors
> rm: cannot remove ‘/tmp/ovirt-log-collector-analyzer-hosts’: Operation not
> permitted
> 

Could you please check the permissions for /tmp/ovirt-log-collector-analyzer-hosts? 
postgres user doesn't have the permission to remove it.

I would suggest removing the dir and trying again run the tool as postgres.
Could you please also share the sosreport?

Thanks!

Comment 8 Sandro Bonazzola 2019-02-01 14:40:08 UTC

Not blocking ovirt-4.3.0 on this. Moving to 4.3.1.

Comment 10 Ivana Saranova 2019-02-05 14:16:54 UTC

No, even with ovirt-log-collector --log-size=0 analyzer still doesn't recognize the sos report as tar archive.

Comment 11 Douglas Schilling Landgraf 2019-02-05 15:50:27 UTC

(In reply to Ivana Saranova from comment #10)
> No, even with ovirt-log-collector --log-size=0 analyzer still doesn't
> recognize the sos report as tar archive.

Could you please share the test machine?

Comment 14 Sandro Bonazzola 2019-02-18 07:54:59 UTC

Moving to 4.3.2 not being identified as blocker for 4.3.1.

Comment 15 Ivana Saranova 2019-03-28 16:23:51 UTC

Please retarget if fix is not in compose 4.3.3 GA or move to ON_QA.

Comment 16 Sandro Bonazzola 2019-04-09 11:51:40 UTC

Moving needinfo to Douglas.

Comment 19 Sandro Bonazzola 2019-07-08 11:52:39 UTC

To be tested with sos shipped within RHEL 7.7

Comment 20 Lucie Leistnerova 2019-07-08 12:46:25 UTC

(In reply to Sandro Bonazzola from comment #19)
> To be tested with sos shipped within RHEL 7.7

It seems it is not only sos problem. We have already tested it and sos now generates PG10 dump but the analyzer can't consume it. It tries to get only PG95 dump.
So we still get error 'tar: This does not look like a tar archive'

See
/usr/share/rhv-log-collector-analyzer/analyzer/unpackAndPrepareDump.sh

84: PG_DUMP_TAR=$(tar tf "$TAR_WITH_POSTGRES_SOSREPORT" | grep "pgdump-scl-rh-postgresql95.tar") || :

Tested with rhv-log-collector-analyzer-0.2.4-0.el7ev.noarch and sos-3.7-5.el7.noarch

Comment 21 Douglas Schilling Landgraf 2019-07-11 20:38:45 UTC

Hi there,

(In reply to Lucie Leistnerova from comment #20)
> (In reply to Sandro Bonazzola from comment #19)
> > To be tested with sos shipped within RHEL 7.7
> 
> It seems it is not only sos problem. We have already tested it and sos now
> generates PG10 dump but the analyzer can't consume it. It tries to get only
> PG95 dump.
> So we still get error 'tar: This does not look like a tar archive'
> 
> See
> /usr/share/rhv-log-collector-analyzer/analyzer/unpackAndPrepareDump.sh
> 
> 84: PG_DUMP_TAR=$(tar tf "$TAR_WITH_POSTGRES_SOSREPORT" | grep
> "pgdump-scl-rh-postgresql95.tar") || :
> 
> Tested with rhv-log-collector-analyzer-0.2.4-0.el7ev.noarch and
> sos-3.7-5.el7.noarch

What's the content for pgdump-scl-rh-postgresql95.tar ? What it generated in a environment with recent sosreport?
Can I have access to env?

Comment 22 Douglas Schilling Landgraf 2019-07-11 20:56:29 UTC

(In reply to Douglas Schilling Landgraf from comment #21)
> Hi there,
> 
> (In reply to Lucie Leistnerova from comment #20)
> > (In reply to Sandro Bonazzola from comment #19)
> > > To be tested with sos shipped within RHEL 7.7
> > 
> > It seems it is not only sos problem. We have already tested it and sos now
> > generates PG10 dump but the analyzer can't consume it. It tries to get only
> > PG95 dump.
> > So we still get error 'tar: This does not look like a tar archive'
> > 
> > See
> > /usr/share/rhv-log-collector-analyzer/analyzer/unpackAndPrepareDump.sh
> > 
> > 84: PG_DUMP_TAR=$(tar tf "$TAR_WITH_POSTGRES_SOSREPORT" | grep
> > "pgdump-scl-rh-postgresql95.tar") || :
> > 
> > Tested with rhv-log-collector-analyzer-0.2.4-0.el7ev.noarch and
> > sos-3.7-5.el7.noarch
> 
> What's the content for pgdump-scl-rh-postgresql95.tar ? What it generated in
> a environment with recent sosreport?
> Can I have access to env?

Replying to myself, it seems we should patch analyzer to accept any postgresql version generated.

Comment 25 Ivana Saranova 2019-08-06 14:58:51 UTC

As of now, rhv-log-collector-analyzer is not intended to work with tar archives, therefore the functionality was tested with running only the rhv-log-collector-analyzer and checking the generated html file.

Steps:
1) Run `rhv-log-collector-analyzer`
2) Check that analyzer did not end with error and html report file is correct

Result:
Analyzer was successful and html report file is correct.

Verified in:
rhv-log-collector-analyzer-0.2.8-0.el7ev.noarch
ovirt-engine-4.3.6-0.1.el7.noarch

Comment 27 errata-xmlrpc 2019-10-10 15:37:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3012