Bug 1133993 - postgres data is not collected
Summary: postgres data is not collected
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-logcollector
Version: 3.5
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
: 3.5.1
Assignee: Sandro Bonazzola
QA Contact: Pavel Novotny
URL:
Whiteboard: integration
Depends On:
Blocks: 1192945
TreeView+ depends on / blocked
 
Reported: 2014-08-26 15:22 UTC by Petr Beňas
Modified: 2015-02-16 10:02 UTC (History)
11 users (show)

Fixed In Version: ovirt-3.5.1_rc1
Doc Type: Bug Fix
Doc Text:
Cause: sosreport command line parser converted numerical passwords to integer so posgtgresql sos plugin crashed while setting the PG_PASSWORD env variable Consequence: postgres sos plugin did not work if a numerical password was specified Fix: the result of the command line parsing is now correctly converted to string Result: postgres sos plugin now works with numerical passwords
Clone Of:
Environment:
Last Closed: 2015-01-21 16:04:28 UTC
oVirt Team: ---
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 35166 0 master MERGED postgresql: avoid to crash with numerical passwords Never
oVirt gerrit 35168 0 ovirt-log-collector-3.5 MERGED postgresql: avoid to crash with numerical passwords Never

Description Petr Beňas 2014-08-26 15:22:43 UTC
Description of problem:
14:24:14 2014-08-26 14:24:14,750 - MainThread - util - ERROR - number of occurrence is out of limit: 0 not in <1:1>; .*/var/lib/pgsql
14:24:14 2014-08-26 14:24:14,751 - MainThread - util - ERROR - number of occurrence is out of limit: 0 not in <1:1>; .*/var/lib/pgsql/pgstartup.log
14:24:14 2014-08-26 14:24:14,752 - MainThread - util - ERROR - number of occurrence is out of limit: 0 not in <1:1>; .*/var/lib/pgsql/data
14:24:14 2014-08-26 14:24:14,753 - MainThread - util - ERROR - number of occurrence is out of limit: 0 not in <3:3>; .*/var/lib/pgsql/data/.*[.]conf
14:24:14 2014-08-26 14:24:14,753 - MainThread - util - ERROR - number of occurrence is out of limit: 0 not in <1:1>; .*/var/lib/pgsql/data/pg_log

Version-Release number of selected component (if applicable):
ovirt-log-collector-3.5.0-0.1.master.el6.noarch

How reproducible:


Steps to Reproduce:
1. engine-log-collector collect
2. inspect collected archive
3.

Actual results:
/var/lib/pgsql not collected

Expected results:
/var/lib/pgsql collected

Additional info:
Discovered by automated test

Comment 1 Jiri Belka 2014-10-21 14:54:59 UTC
[root@jb-rhevm34 ~]# rpm -qf `which engine-log-collector`
rhevm-log-collector-3.4.5-1.el6ev.noarch
[root@jb-rhevm34 ~]# tar -xJOf /tmp/sosreport-LogCollector-20141021164943.tar.xz './log-collector-data/postgresql-sosreport-qaredhat.com-20141021164924-e358.tar.xz' | tar tJvf -
drwx------ root/root         0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/
drwxr-xr-x root/root         0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/var/
drwxr-xr-x root/root         0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/var/lib/
drwxr-xr-x root/root         0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/var/lib/pgsql/
-rw------- postgres/postgres 1152 2013-02-22 13:02 jb-rhevm34-2014102116491413902959/var/lib/pgsql/pgstartup.log
drwxr-xr-x root/root            0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/
-rw------- postgres/postgres    4 2013-02-22 13:02 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/PG_VERSION
-rw------- postgres/postgres   57 2014-10-19 17:31 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/postmaster.opts
drwxr-xr-x root/root            0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/
-rw------- postgres/postgres    0 2014-10-21 00:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Tue.log
-rw------- postgres/postgres    0 2014-10-20 00:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Mon.log
-rw------- postgres/postgres  171 2014-10-17 17:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Fri.log
-rw------- postgres/postgres    0 2014-10-16 00:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Thu.log
-rw------- postgres/postgres    0 2014-10-15 00:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Wed.log
-rw------- postgres/postgres    0 2014-10-11 00:00 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Sat.log
-rw------- postgres/postgres  151 2014-10-19 17:31 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_log/postgresql-Sun.log
-rw------- postgres/postgres 16970 2014-03-04 17:02 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/postgresql.conf
-rw------- postgres/postgres  1631 2013-02-22 13:02 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_ident.conf
-rw------- postgres/postgres  4563 2014-04-25 10:41 jb-rhevm34-2014102116491413902959/var/lib/pgsql/data/pg_hba.conf
drwxr-xr-x root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/sos_logs/
-rw-r--r-- root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/sos_logs/sos.log
drwxr-xr-x root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/sos_reports/
-rw-r--r-- root/root          2151 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/sos_reports/sosreport.html
drwxr-xr-x root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/sos_commands/
drwxr-xr-x root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/tmp/
drwxr-xr-x root/root             0 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/tmp/tmpP6ZJxZ/
-rw-r--r-- root/root       4565504 2014-10-21 16:49 jb-rhevm34-2014102116491413902959/tmp/tmpP6ZJxZ/sos_pgdump.tar

Comment 2 Petr Beňas 2014-10-22 11:13:18 UTC
Pretty weird, got the same version of collector, but the DB wasn't collected. 
[root@pb-rh34 ~]#  tar -xJOf /tmp/sosreport-LogCollector-20141022104209.tar.xz './log-collector-data/postgresql-sosreport-admin-20141022104030-53ba.tar.xz' | tar tJvf -
drwx------ root/root         0 2014-10-22 10:40 pb-rh34-2014102210401413967229/
drwxr-xr-x root/root         0 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_logs/
-rw-r--r-- root/root         0 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_logs/sos.log
-rw-r--r-- root/root       527 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_logs/sosreport-plugin-errors.txt
drwxr-xr-x root/root         0 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_reports/
-rw-r--r-- root/root       666 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_reports/sosreport.html
drwxr-xr-x root/root         0 2014-10-22 10:40 pb-rh34-2014102210401413967229/sos_commands/
[root@pb-rh34 ~]# rpm -qa | grep collector
rhevm-log-collector-3.4.5-1.el6ev.noarch

Comment 3 Simone Tiraboschi 2014-10-30 09:50:11 UTC
I tried to reproduce with rhevm-log-collector-3.4.5-1.el6ev.noarch on RHEL6 but it works as expected.

Petr, do you have any hint about how to reproduce?

Comment 4 Petr Beňas 2014-10-31 09:15:36 UTC
Still reproducible in vt8 with our automation job. Feel free to contact me directly for the test code details. 
http://jenkins.qa.lab.tlv.redhat.com:8080/view/RhevmCore/view/3.5-ALL/job/3.5-git-rhevmCore-infra_tools_log_collector_nfs/51/consoleFull

Comment 5 Sandro Bonazzola 2014-11-14 11:55:58 UTC
(In reply to Petr Beňas from comment #4)
> Still reproducible in vt8 with our automation job. Feel free to contact me
> directly for the test code details. 
> http://jenkins.qa.lab.tlv.redhat.com:8080/view/RhevmCore/view/3.5-ALL/job/3.
> 5-git-rhevmCore-infra_tools_log_collector_nfs/51/consoleFull

Petr, can you change the job at this line:
23:21:34 2014-10-30 23:21:34,129 - MainThread - rhevm_tools - INFO - executed command: ['echo', 'Y', '|', '/usr/bin/rhevm-log-collector', '--data-center=newdc', 'collect'] ; rc: 0, out: This command will collect system configuration and diagnostic

for adding --verbose to command line?
without that I can't debug what's wrong there and I can't reproduce on my system.

Comment 6 Petr Beňas 2014-11-14 12:09:45 UTC
Hi Sandro, there already is a test case running with the --verbose option. Search for "LogCollectorSingleDC.test_log_collector_collect_verbosity". I see there an error "err: ERROR: Failed to collect logs from: 10.35.160.33; Could not parse sosreport output to determine filename".

Comment 7 Sandro Bonazzola 2014-11-14 12:39:48 UTC
23:27:59 Traceback (most recent call last):
23:27:59   File "/usr/lib/python2.6/site-packages/sos/sosreport.py", line 736, in sosreport
23:27:59     plug.setup()
23:27:59   File "/usr/lib/python2.6/site-packages/sos/plugins/postgresql.py", line 98, in setup
23:27:59     self.pg_dump()
23:27:59 TypeError: putenv() argument 2 must be string, not int
23:27:59   Running plugins. Please wait 

Opened a bug on sos report: https://github.com/sosreport/sos/issues/433

will solve adding a str

Comment 8 Sandro Bonazzola 2014-11-14 12:41:34 UTC
it affects only numerical passwords

Comment 9 Bryn M. Reeves 2014-11-14 13:16:52 UTC
Adding a coercion to string in the plugin is a workaround; unless this is highly urgent I'd suggest holding out for a fix from upstream.

Comment 10 Sandro Bonazzola 2014-11-14 13:37:43 UTC
Bryn, just let me know when the plugin is fixed upstream and I'll be happy to sync. Also because this change is on el6 only on log-collector side: sent a patch to sos-devel with the same fix used here.

Comment 11 Bryn M. Reeves 2014-11-14 14:27:48 UTC
Will do - most likely it'll require a fix in sosreport.py::_set_tunables() so it'll need to be fixed in the base sos package (if that's going to cause scheduling problems then the workaround of coercing to str in the plugin is an acceptable short-term fix).

Comment 13 Sandro Bonazzola 2015-01-15 15:09:21 UTC
This is an automated message: 
This bug should be fixed in oVirt 3.5.1 RC1, moving to QA

Comment 14 Sandro Bonazzola 2015-01-21 16:04:28 UTC
oVirt 3.5.1 has been released. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.