committed to rhel-7.4 dist-git

Steps to verify:

ensure that sosreport collects the same data for postgreSQL installed from SCL as for "regular" / regularly installed postgreSQL. In particular, sosreport should collect:

1)
From pghome(*) dir, collect:
- all *log files under that dir
- all *conf files under that dir
- data/PG_VERSION
- data/postmaster.opts

(*) /var/lib/pgsql for regular postgres, /var/opt/rh/rh-postgresql95/lib/pgsql for SCL postgres


2)
In case env.variable PGPASSWORD is set(*) or sosreport is run with -k postgresql.password=somePassword, sosreport takes postgres dump via command like:

scl enable rh-postgresql95 -- pg_dump -U engine -h localhost -p 5432 -w -f /tmp/tmpXMhHFL/sos_scl_pgdump.tar -F t engine

and stores it in

sos_commands/postgresql/sos_scl_pgdump.tar


(*) ovirt-log-collector should set the password env.variable


Lukas, could you pls. verify this against the errata in previous comment?

tested in 
sos-3.4-10.el7_4.noarch
ovirt-log-collector-4.2.0-0.0.master.20171107155516.git3b8cca7.el7.centos.noarch

1) 
- logs - ok
|-- sos_logs
|   |-- sos.log
|   `-- ui.log
|-- var
|   |-- lib
|   |   `-- pgsql
|   |       `-- initdb_rh-postgresql95-postgresql.log
|   `-- opt
|       `-- rh
|           `-- rh-postgresql95
|               `-- lib
|                   `-- pgsql
|                       `-- data
|                           |-- pg_log
|                           |   |-- postgresql-Fri.log
|                           |   |-- postgresql-Mon.log
|                           |   |-- postgresql-Sat.log
|                           |   |-- postgresql-Sun.log
|                           |   |-- postgresql-Thu.log
|                           |   |-- postgresql-Tue.log
|                           |   `-- postgresql-Wed.log


- conf - ok
|-- var
|   `-- opt
|       `-- rh
|           `-- rh-postgresql95
|               `-- lib
|                   `-- pgsql
|                       `-- data
|                           |-- pg_hba.conf
|                           |-- pg_ident.conf
|                           |-- postgresql.auto.conf
|                           |-- postgresql.conf

- data/PG_VERSION - ok
  - 9.5

- data/postmaster.opts - ok
  - /opt/rh/rh-postgresql95/root/usr/bin/postgres "-D" "/var/opt/rh/rh-postgresql95/lib/pgsql/data"



2)
In case env.variable PGPASSWORD is set(*) or sosreport is run with -k postgresql.password=somePassword, sosreport takes postgres dump - FAIL

no dump available
sos.log:
2017-11-20 14:55:42,874 INFO: [sos.sosreport:setup] executing 'sosreport --batch -o postgresql --tmp-dir=/tmp/logcollector-LZsTXh/working/log-collector-data -k postgresql.dbname=engine -k postgresql.dbhost=localhost -k postgresql.dbport=5432 -k postgresql.username=engine'
2017-11-20 14:55:42,882 INFO: [plugin:postgresql] command 'pg_dump' not found in / - re-trying in host root
2017-11-20 14:55:42,888 INFO: [plugin:postgresql] Unable to execute pg_dump. Error()
2017-11-20 14:55:42,963 INFO: [plugin:postgresql] command 'scl' not found in / - re-trying in host root
2017-11-20 14:55:42,977 INFO: [plugin:postgresql] Unable to execute pg_dump. Error()


(*) ovirt-log-collector should set the password env.variable
# env | grep PGPASS
# 

Might be cause by this, but I suspect it is setting the var only for scope of ovirt-log-collector so should be ok


Pavel can you please take a look?

That is due to:

[Bug 1513071] Log Analyzer unable to detect sos_pgdump.tar

that is outside sosreport.

When I manually "fix" the path via adding lines:

                print "PATH=%s, pg_dump_command=%s" % (os.environ["PATH"], pg_dump_command)
                import re
                p = re.compile('pg_dump')
                pg_dump_command = p.sub('/opt/rh/rh-postgresql95/root/usr/bin/pg_dump', pg_dump_command)
                print "updated pg_dump_command to: %s" % pg_dump_command

to /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py, then it starts to work well:


# env | grep PGPASSWORD
PGPASSWORD=*******************
# sosreport --batch -o postgresql -k postgresql.dbname=engine -k postgresql.dbhost=localhost -k postgresql.dbport=5432 -k postgresql.username=engine

sosreport (version 3.4)

This command will collect diagnostic and configuration information from
this Red Hat Enterprise Linux system and installed applications.

An archive containing the collected information will be generated in
/var/tmp/sos.tav5tE and may be provided to a Red Hat support
representative.

Any information provided to Red Hat will be treated in accordance with
the published support policies at:

  https://access.redhat.com/support/

The generated archive may contain data considered sensitive and its
content should be reviewed by the originating organization before being
passed to any third party.

No changes will be made to system configuration.


 Setting up archive ...
 Setting up plugins ...
PATH=/usr/sbin:/usr/bin:/root/bin:/usr/local/bin:/usr/local/sbin, pg_dump_command=pg_dump
updated pg_dump_command to: /opt/rh/rh-postgresql95/root/usr/bin/pg_dump
PATH=/usr/sbin:/usr/bin:/root/bin:/usr/local/bin:/usr/local/sbin, pg_dump_command=scl enable rh-postgresql95 -- pg_dump
updated pg_dump_command to: scl enable rh-postgresql95 -- /opt/rh/rh-postgresql95/root/usr/bin/pg_dump
 Running plugins. Please wait ...

  Running 1/1: postgresql...        

Creating compressed archive...

Your sosreport has been generated and saved in:
  /var/tmp/sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030.tar.xz

The checksum is: fbf551e6bcc54ec32d1786ca016dee15

Please send this file to your support representative.

# tar xJvf /var/tmp/sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030.tar.xz | grep dump
sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030/tmp/tmpz1LqXE/sos_scl_pgdump.tar
sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030/tmp/tmpIqB_OA/sos_pgdump.tar
# 


If the PATH would be set properly (or e.g. if I link pg_dump from /usr/bin), it will work without the above manual change. So I suppose this BZ is fixed in sosreport.

Newly committed changes due to bz1513071 & re-spinning build in the z-stream errata.

Perspective workaround patch for the generic "command output is treated as UTF-8 only" problem:


--- /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py.orig	2017-12-19 17:39:00.076871338 +0100
+++ /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py	2017-12-19 17:38:52.031781055 +0100
@@ -54,22 +54,33 @@ class PostgreSQL(Plugin):
                 if self.get_option("password") is not False:
                     os.environ["PGPASSWORD"] = str(self.get_option("password"))
 
+                self.tmp_dir = tempfile.mkdtemp()
+                dest_file = os.path.join(self.tmp_dir, filename)
                 if self.get_option("dbhost"):
-                    cmd = "pg_dump -U %s -h %s -p %s -w -F t %s" % (
+                    cmd = "pg_dump -U %s -h %s -p %s -w -f %s -F t %s" % (
                         self.get_option("username"),
                         self.get_option("dbhost"),
                         self.get_option("dbport"),
+                        dest_file,
                         self.get_option("dbname")
                     )
                 else:
-                    cmd = "pg_dump -C -U %s -w -F t %s " % (
+                    cmd = "pg_dump -C -U %s -w -f %s -F t %s " % (
                         self.get_option("username"),
+                        dest_file,
                         self.get_option("dbname")
                     )
 
                 if scl is not None:
                     cmd = self.convert_cmd_scl(scl, cmd)
-                self.add_cmd_output(cmd, suggest_filename=filename)
+                result = self.call_ext_prog(cmd)
+                if (result['status'] == 0):
+                    self.add_copy_spec(dest_file)
+                else:
+                    self._log_info(
+                        "Unable to execute pg_dump. Error(%s)" %
+                        (result['output'])
+                    )
             else:  # no password in env or options
                 self.soslog.warning(
                     "password must be supplied to dump a database."

*** Bug 1513071 has been marked as a duplicate of this bug. ***

Thanks for the ping,



Tested on sos-3.4-13.el7_4.noarch, within
ovirt-log-collector-4.2.0-1.el7ev.noarch


Tree structure of archive:
.
|-- sos_commands
|   `-- postgresql
|       `-- pgdump-scl-rh-postgresql95.tar
|-- sos_logs
|   |-- sos.log
|   `-- ui.log
|-- sos_reports
|   |-- sos.html
|   `-- sos.txt
|-- var
|   |-- lib
|   |   `-- pgsql
|   |       `-- initdb_rh-postgresql95-postgresql.log
|   `-- opt
|       `-- rh
|           `-- rh-postgresql95
|               `-- lib
|                   `-- pgsql
|                       `-- data
|                           |-- pg_hba.conf
|                           |-- pg_ident.conf
|                           |-- pg_log
|                           |   |-- postgresql-01.log
|                           |   `-- postgresql-Wed.log
|                           |-- PG_VERSION
|                           |-- postgresql.auto.conf
|                           |-- postgresql.conf
|                           `-- postmaster.opts
`-- version.txt


Config files collected ok

sos log seems ok


pg_dump in different path 
`── sos_commands
    └── postgresql
        └── pgdump-scl-rh-postgresql95.tar
 
 
Lets see what's inside:
# tar xvf sos_commands/postgresql/pgdump-scl-rh-postgresql95.tar.gz 
success

directory contains all the sql data

lets restore
# pg_restore sos_commands/postgresql/pgdump-scl-rh-postgresql95.tar
...output omitted...
--
-- PostgreSQL database dump complete
--

restart engine using this db

# service ovirt-engine restart
success!

Moving to verified.

*** Bug 1538365 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0138