Bug 1515113
Summary: | sos doesn't support PostgreSQL95 from Software Collections [rhel-7.4.z] | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Oneata Mircea Teodor <toneata> |
Component: | sos | Assignee: | Pavel Moravec <pmoravec> |
Status: | CLOSED ERRATA | QA Contact: | Miroslav Hradílek <mhradile> |
Severity: | urgent | Docs Contact: | |
Priority: | high | ||
Version: | 7.4 | CC: | agk, bmr, dnecpal, dornelas, gavin, lsvaty, mhradile, mkalinin, plambri, pmoravec, sbonazzo, sbradley, toneata, ylavi |
Target Milestone: | rc | Keywords: | OtherQA, ZStream |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | sos-3.4-12.el7_4 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | 1494420 | Environment: | |
Last Closed: | 2018-01-25 11:34:34 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1494420 | ||
Bug Blocks: | 1494419, 1538360 |
Description
Oneata Mircea Teodor
2017-11-20 07:30:12 UTC
committed to rhel-7.4 dist-git Steps to verify: ensure that sosreport collects the same data for postgreSQL installed from SCL as for "regular" / regularly installed postgreSQL. In particular, sosreport should collect: 1) From pghome(*) dir, collect: - all *log files under that dir - all *conf files under that dir - data/PG_VERSION - data/postmaster.opts (*) /var/lib/pgsql for regular postgres, /var/opt/rh/rh-postgresql95/lib/pgsql for SCL postgres 2) In case env.variable PGPASSWORD is set(*) or sosreport is run with -k postgresql.password=somePassword, sosreport takes postgres dump via command like: scl enable rh-postgresql95 -- pg_dump -U engine -h localhost -p 5432 -w -f /tmp/tmpXMhHFL/sos_scl_pgdump.tar -F t engine and stores it in sos_commands/postgresql/sos_scl_pgdump.tar (*) ovirt-log-collector should set the password env.variable Lukas, could you pls. verify this against the errata in previous comment? tested in sos-3.4-10.el7_4.noarch ovirt-log-collector-4.2.0-0.0.master.20171107155516.git3b8cca7.el7.centos.noarch 1) - logs - ok |-- sos_logs | |-- sos.log | `-- ui.log |-- var | |-- lib | | `-- pgsql | | `-- initdb_rh-postgresql95-postgresql.log | `-- opt | `-- rh | `-- rh-postgresql95 | `-- lib | `-- pgsql | `-- data | |-- pg_log | | |-- postgresql-Fri.log | | |-- postgresql-Mon.log | | |-- postgresql-Sat.log | | |-- postgresql-Sun.log | | |-- postgresql-Thu.log | | |-- postgresql-Tue.log | | `-- postgresql-Wed.log - conf - ok |-- var | `-- opt | `-- rh | `-- rh-postgresql95 | `-- lib | `-- pgsql | `-- data | |-- pg_hba.conf | |-- pg_ident.conf | |-- postgresql.auto.conf | |-- postgresql.conf - data/PG_VERSION - ok - 9.5 - data/postmaster.opts - ok - /opt/rh/rh-postgresql95/root/usr/bin/postgres "-D" "/var/opt/rh/rh-postgresql95/lib/pgsql/data" 2) In case env.variable PGPASSWORD is set(*) or sosreport is run with -k postgresql.password=somePassword, sosreport takes postgres dump - FAIL no dump available sos.log: 2017-11-20 14:55:42,874 INFO: [sos.sosreport:setup] executing 'sosreport --batch -o postgresql --tmp-dir=/tmp/logcollector-LZsTXh/working/log-collector-data -k postgresql.dbname=engine -k postgresql.dbhost=localhost -k postgresql.dbport=5432 -k postgresql.username=engine' 2017-11-20 14:55:42,882 INFO: [plugin:postgresql] command 'pg_dump' not found in / - re-trying in host root 2017-11-20 14:55:42,888 INFO: [plugin:postgresql] Unable to execute pg_dump. Error() 2017-11-20 14:55:42,963 INFO: [plugin:postgresql] command 'scl' not found in / - re-trying in host root 2017-11-20 14:55:42,977 INFO: [plugin:postgresql] Unable to execute pg_dump. Error() (*) ovirt-log-collector should set the password env.variable # env | grep PGPASS # Might be cause by this, but I suspect it is setting the var only for scope of ovirt-log-collector so should be ok Pavel can you please take a look? That is due to: [Bug 1513071] Log Analyzer unable to detect sos_pgdump.tar that is outside sosreport. When I manually "fix" the path via adding lines: print "PATH=%s, pg_dump_command=%s" % (os.environ["PATH"], pg_dump_command) import re p = re.compile('pg_dump') pg_dump_command = p.sub('/opt/rh/rh-postgresql95/root/usr/bin/pg_dump', pg_dump_command) print "updated pg_dump_command to: %s" % pg_dump_command to /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py, then it starts to work well: # env | grep PGPASSWORD PGPASSWORD=******************* # sosreport --batch -o postgresql -k postgresql.dbname=engine -k postgresql.dbhost=localhost -k postgresql.dbport=5432 -k postgresql.username=engine sosreport (version 3.4) This command will collect diagnostic and configuration information from this Red Hat Enterprise Linux system and installed applications. An archive containing the collected information will be generated in /var/tmp/sos.tav5tE and may be provided to a Red Hat support representative. Any information provided to Red Hat will be treated in accordance with the published support policies at: https://access.redhat.com/support/ The generated archive may contain data considered sensitive and its content should be reviewed by the originating organization before being passed to any third party. No changes will be made to system configuration. Setting up archive ... Setting up plugins ... PATH=/usr/sbin:/usr/bin:/root/bin:/usr/local/bin:/usr/local/sbin, pg_dump_command=pg_dump updated pg_dump_command to: /opt/rh/rh-postgresql95/root/usr/bin/pg_dump PATH=/usr/sbin:/usr/bin:/root/bin:/usr/local/bin:/usr/local/sbin, pg_dump_command=scl enable rh-postgresql95 -- pg_dump updated pg_dump_command to: scl enable rh-postgresql95 -- /opt/rh/rh-postgresql95/root/usr/bin/pg_dump Running plugins. Please wait ... Running 1/1: postgresql... Creating compressed archive... Your sosreport has been generated and saved in: /var/tmp/sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030.tar.xz The checksum is: fbf551e6bcc54ec32d1786ca016dee15 Please send this file to your support representative. # tar xJvf /var/tmp/sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030.tar.xz | grep dump sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030/tmp/tmpz1LqXE/sos_scl_pgdump.tar sosreport-system-ge-3.rhev.lab.eng.brq.redhat.com-20171120181030/tmp/tmpIqB_OA/sos_pgdump.tar # If the PATH would be set properly (or e.g. if I link pg_dump from /usr/bin), it will work without the above manual change. So I suppose this BZ is fixed in sosreport. Newly committed changes due to bz1513071 & re-spinning build in the z-stream errata. Perspective workaround patch for the generic "command output is treated as UTF-8 only" problem: --- /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py.orig 2017-12-19 17:39:00.076871338 +0100 +++ /usr/lib/python2.7/site-packages/sos/plugins/postgresql.py 2017-12-19 17:38:52.031781055 +0100 @@ -54,22 +54,33 @@ class PostgreSQL(Plugin): if self.get_option("password") is not False: os.environ["PGPASSWORD"] = str(self.get_option("password")) + self.tmp_dir = tempfile.mkdtemp() + dest_file = os.path.join(self.tmp_dir, filename) if self.get_option("dbhost"): - cmd = "pg_dump -U %s -h %s -p %s -w -F t %s" % ( + cmd = "pg_dump -U %s -h %s -p %s -w -f %s -F t %s" % ( self.get_option("username"), self.get_option("dbhost"), self.get_option("dbport"), + dest_file, self.get_option("dbname") ) else: - cmd = "pg_dump -C -U %s -w -F t %s " % ( + cmd = "pg_dump -C -U %s -w -f %s -F t %s " % ( self.get_option("username"), + dest_file, self.get_option("dbname") ) if scl is not None: cmd = self.convert_cmd_scl(scl, cmd) - self.add_cmd_output(cmd, suggest_filename=filename) + result = self.call_ext_prog(cmd) + if (result['status'] == 0): + self.add_copy_spec(dest_file) + else: + self._log_info( + "Unable to execute pg_dump. Error(%s)" % + (result['output']) + ) else: # no password in env or options self.soslog.warning( "password must be supplied to dump a database." *** Bug 1513071 has been marked as a duplicate of this bug. *** Thanks for the ping, Tested on sos-3.4-13.el7_4.noarch, within ovirt-log-collector-4.2.0-1.el7ev.noarch Tree structure of archive: . |-- sos_commands | `-- postgresql | `-- pgdump-scl-rh-postgresql95.tar |-- sos_logs | |-- sos.log | `-- ui.log |-- sos_reports | |-- sos.html | `-- sos.txt |-- var | |-- lib | | `-- pgsql | | `-- initdb_rh-postgresql95-postgresql.log | `-- opt | `-- rh | `-- rh-postgresql95 | `-- lib | `-- pgsql | `-- data | |-- pg_hba.conf | |-- pg_ident.conf | |-- pg_log | | |-- postgresql-01.log | | `-- postgresql-Wed.log | |-- PG_VERSION | |-- postgresql.auto.conf | |-- postgresql.conf | `-- postmaster.opts `-- version.txt Config files collected ok sos log seems ok pg_dump in different path `── sos_commands └── postgresql └── pgdump-scl-rh-postgresql95.tar Lets see what's inside: # tar xvf sos_commands/postgresql/pgdump-scl-rh-postgresql95.tar.gz success directory contains all the sql data lets restore # pg_restore sos_commands/postgresql/pgdump-scl-rh-postgresql95.tar ...output omitted... -- -- PostgreSQL database dump complete -- restart engine using this db # service ovirt-engine restart success! Moving to verified. *** Bug 1538365 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:0138 |