Bug 1656278

Summary: sosreport collects only the tail of a postgresql dump
Product: Red Hat Enterprise Linux 7 Reporter: Yedidyah Bar David <didi>
Component: sosAssignee: Pavel Moravec <pmoravec>
Status: CLOSED ERRATA QA Contact: Miroslav HradĂ­lek <mhradile>
Severity: high Docs Contact:
Priority: high    
Version: 7.6CC: agk, bmr, didi, gavin, gchakkar, lrotenbe, mhradile, mkalinin, plambri, pmoravec, rcyriac, sbradley
Target Milestone: rcKeywords: OtherQA, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sos-3.7-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1658571 1658939 (view as bug list) Environment:
Last Closed: 2019-08-06 13:15:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1654068, 1658571, 1658939    

Description Yedidyah Bar David 2018-12-05 06:53:16 UTC
Description of problem:

If the db dump is > 25MiB, sos collects only last 25MiB. This makes the dump unusable and requires asking the provider of the report to manually dump and provide the dump.

PR for a fix is here:

https://github.com/sosreport/sos/pull/1497

Comment 2 Pavel Moravec 2018-12-05 09:20:19 UTC
devel_ack+ for 7.7 and also possibly to 7.6.z.

Yedidyah: do you require also 7.6.z-stream (that was my understanding from other discussion)? If so, please provide business justification and raise 7.6.z? flag.

Comment 3 Yedidyah Bar David 2018-12-05 09:28:44 UTC
(In reply to Pavel Moravec from comment #2)
> devel_ack+ for 7.7 and also possibly to 7.6.z.

Thanks

> 
> Yedidyah: do you require also 7.6.z-stream (that was my understanding from
> other discussion)?

I think so, yes. We hope to get it fixed by RHV 4.2.8 GA, which should be in 1.5 months or so.

> If so, please provide business justification and raise
> 7.6.z? flag.

Set the flag.

Gajanan (requester of original RHV bug 1654068), can you provide business justification? Thanks.

Comment 4 Yedidyah Bar David 2018-12-05 10:04:23 UTC
Already verified my patch locally. Will ask RHV QE to mark this one as verified together with RHV bug 1654068, once they verify it (with an official build).

Comment 7 Marina Kalinin 2018-12-05 20:33:58 UTC
I provided the business justification for z-stream. Removing need info from Gajanan.

Comment 8 Pavel Moravec 2018-12-05 21:22:16 UTC
GSSApproved for 7.6.z

Comment 10 Marina Kalinin 2018-12-11 23:50:12 UTC
Hi Pavel,

Any update on the bug status?
Does the patch from Didi looks good?

Thanks,
Marina.

Comment 11 Pavel Moravec 2018-12-12 08:15:58 UTC
(In reply to Marina from comment #10)
> Hi Pavel,
> 
> Any update on the bug status?
> Does the patch from Didi looks good?
> 
> Thanks,
> Marina.

From my point of view, yes - verified also on my Satellite6. Where I just had to apply patch

https://github.com/sosreport/sos/pull/1416

that is required in case postgres installed from SCL (I think not the case for RHEV behind this BZ, but I will backport both PRs within this BZ).

Just waiting for Bryn to merge into upstream and will build package for 7.6.2.

Comment 12 Pavel Moravec 2018-12-12 12:09:33 UTC
pushed to 7.7 dist-git in commit 3fe7ccc

Comment 15 Pavel Moravec 2019-03-27 20:20:15 UTC
Hello,
could you please verify the BZ against below build? Thanks in advance.


A yum repository for the build of sos-3.7-1.el7 (task 20748546) is available at:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.7/1.el7/

You can install the rpms locally by putting this .repo file in your /etc/yum.repos.d/ directory:

http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.7/1.el7/sos-3.7-1.el7.repo

RPMs and build logs can be found in the following locations:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.7/1.el7/noarch/

The full list of available rpms is:
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.7/1.el7/noarch/sos-3.7-1.el7.src.rpm
http://brew-task-repos.usersys.redhat.com/repos/official/sos/3.7/1.el7/noarch/sos-3.7-1.el7.noarch.rpm

Build output will be available for the next 21 days.

Comment 16 Yedidyah Bar David 2019-03-28 06:34:40 UTC
Liran, can you please verify above build? It's the same process as bug 1658571 comment 5, but with RHEL 7.7. Thanks!

Comment 17 Liran Rotenberg 2019-04-03 12:35:44 UTC
Verified on:
Red Hat Enterprise Linux Server release 7.6 (Maipo)
sos-3.7-1.el7.noarch
ovirt-engine-4.3.3.1-0.1.el7.noarch
ovirt-log-collector-4.3.2-1.el7ev.noarch

Steps taken from: https://bugzilla.redhat.com/show_bug.cgi?id=1654068#c9
1. Make sure your engine db is large enough, so that its dump is > 25MiB. I did this with:

su - postgres
scl enable rh-postgresql95 bash

psql engine
create table test1(a varchar(512))
alter table test1 owner to engine
\q

a=$(printf 'a%.0s' $(seq 511)); for i in $(seq 50000); do echo "insert into test1 values('$a');"; done | psql engine

Obviously, there are other means (such as creating many VMs/Disks/NICs/etc. and/or causing lots of errors so that the audit_log is filled).

2. ovirt-log-collector

This creates an archive /tmp/sosreport-LogCollector-$TIMESTAMP.tar.xz

3. cd /tmp; tar xpf sosreport-LogCollector-$TIMESTAMP.tar.xz

4. cd sosreport-LogCollector-$TIMESTAMP

5. cd log-collector-data

6. tar xpf postgresql-sosreport-*.tar.xz

7. cd to generated directory postgresql-sosreport-*

8. cd sos_commands/postgresql

9. check the files there (e.g. with ls -l, 'file', or 'less < file').

Results:
# ls -sh
total 30M
4.0K du_-sh_.var.lib.pgsql
30M pgdump-scl-rh-postgresql10.tar
0 pgdump.tar

# file pgdump-scl-rh-postgresql10.tar
pgdump-scl-rh-postgresql10.tar: POSIX tar archive

Comment 20 errata-xmlrpc 2019-08-06 13:15:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2295