Bug 1060383

The fix must include a unit test for the plugin, so catch similar issues earlier.

Setting target release to current version for consideration and review. please
do not push non-RFE bugs to an undefined target release to make sure bugs are
reviewed for relevancy, fix, closure, etc.

This is an automated message.
Re-targeting all non-blocker bugs still open on 3.4.0 to 3.4.1.

I should test this with logcollector?

(In reply to Jiri Belka from comment #4)
> I should test this with logcollector?

Yes, please.

fail, vdsm3-4.14.6-0.el7ev.x86_64

# engine-log-collector --no-postgresql -H 10.34.63.222 -v
...snip...
INFO: collecting information from 10.34.63.222
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root.63.222', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.0)

)
DEBUG: STDERR(Error reading response length from authentication socket.^M
no such option "all_logs" for plugin (general)
)
DEBUG: message(list index out of range)
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/bin/engine-log-collector", line 530, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

ERROR: Failed to collect logs from: 10.34.63.222; Could not parse sosreport output to determine filename
DEBUG: Traceback (most recent call last):
DEBUG:   File "/usr/bin/engine-log-collector", line 709, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:   File "/usr/bin/engine-log-collector", line 566, in parse_sosreport_stdout
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG: Exception: Could not parse sosreport output to determine filename
...snip...
DEBUG: STDOUT(2c81e0afb74f4002eb4708a5c9c62e58  /tmp/sosreport-LogCollector-20140415170959.tar.xz
)
DEBUG: STDERR()
INFO: Log files have been collected and placed in /tmp/sosreport-LogCollector-20140415170959.tar.xz.
The MD5 for this file is 2c81e0afb74f4002eb4708a5c9c62e58 and its size is 43.5M
# echo $?
2

missing data from host:

# tar tJvf /tmp/sosreport-LogCollector-20140415170959.tar.xz | grep '10.34.63.222.*tar.xz'
#

Created attachment 886516 [details]
logcollector output

(In reply to Jiri Belka from comment #6)
> fail, vdsm3-4.14.6-0.el7ev.x86_64
> 
> # engine-log-collector --no-postgresql -H 10.34.63.222 -v
> ...snip...
> INFO: collecting information from 10.34.63.222
> DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i',
> '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no',
> '-oServerAliveInterval=600', 'root.63.222', "\nVERSION=`/bin/rpm -q
> --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 22 ];
> then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o
> libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,
> selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n   
> /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o
> vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo
> No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
> DEBUG: returncode(0)
> DEBUG: STDOUT(
> sosreport (version 3.0)
> 
> )
> DEBUG: STDERR(Error reading response length from authentication socket.^M
> no such option "all_logs" for plugin (general)
> )
> DEBUG: message(list index out of range)
> DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
>   File "/usr/bin/engine-log-collector", line 530, in parse_sosreport_stdout
>     if fileAry[0] is not None and len(fileAry) > 0:
> IndexError: list index out of range
> 
> ERROR: Failed to collect logs from: 10.34.63.222; Could not parse sosreport
> output to determine filename
> DEBUG: Traceback (most recent call last):
> DEBUG:   File "/usr/bin/engine-log-collector", line 709, in run
> DEBUG:     self.parse_sosreport_stdout(stdout)
> DEBUG:   File "/usr/bin/engine-log-collector", line 566, in
> parse_sosreport_stdout
> DEBUG:     "Could not parse sosreport output to determine filename"
> DEBUG: Exception: Could not parse sosreport output to determine filename
> ...snip...
> DEBUG: STDOUT(2c81e0afb74f4002eb4708a5c9c62e58 
> /tmp/sosreport-LogCollector-20140415170959.tar.xz
> )
> DEBUG: STDERR()
> INFO: Log files have been collected and placed in
> /tmp/sosreport-LogCollector-20140415170959.tar.xz.
> The MD5 for this file is 2c81e0afb74f4002eb4708a5c9c62e58 and its size is
> 43.5M
> # echo $?
> 2
> 
> missing data from host:
> 
> # tar tJvf /tmp/sosreport-LogCollector-20140415170959.tar.xz | grep
> '10.34.63.222.*tar.xz'
> #

Seems an issue with engine-log-collector not with vdsm sos plugin.
Sandro, can you please provide your thoughts?

Thanks!

I started working on an oVirt engine plugin upstream a few weeks back, based on suggestions and patches from Sandro. You can find that work here:

  https://github.com/sosreport/sos/tree/bmr-add-ovirt-plugin

  https://github.com/sosreport/sos/blob/bmr-add-ovirt-plugin/sos/plugins/ovirt.py

Please review and comment on github - I'm just waiting for enough feedback before I merge this (and the dwh/reports bits).

In reply to comment #8 - it's not really clear what's happening here. Clearly engine-log-collector got something it didn't expect in the output from the remote sos command but without seeing what the output was it's hard to say what may have caused it.

Note that there's been some reorganisation of plugins in 3.0 - e.g. general.all_logs no longer exists as all syslog data is now captured in the 'logs' plugin (there is a corresponding logs.all_logs option now).

Hello Jiri,

It seems a different bug not related to vdsm sos plugin. Please open a bug on engine-log-collector with the output that Bryn requested in comment #10, we can also add it into Depends On of this bug. I am moving back this bug to ON_QA.

OK looking at the output in the attachment:

https://bugzilla.redhat.com/attachment.cgi?id=886516

It seems to be the problem I suggested in comment #10 - the "all_logs" option has moved from the general plugin to the logs plugin:

DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.0)

)
DEBUG: STDERR(Error reading response length from authentication socket.

no such option "all_logs" for plugin (general)
)

It might be useful to teach the log collector some of the common errors sosreport can spit out so that it can react to this better.

BZ1088172 create to address the issue reported in #6. Meanwhile making this BZ depending on BZ1088172.

(In reply to Jiri Belka from comment #13)
> BZ1088172 create to address the issue reported in #6. Meanwhile making this
> BZ depending on BZ1088172.

I'm sorry, I don't understand one thing.
This bug is targeted to oVirt 3.4.1 but log-collector is not expected to support sos 3.0 before oVirt 3.5.0 release.

ovirt-log-collector from master can't generate the issue described in comment #6.

This oVirt bug now depend on a rhev-m bug which has its own life cycle.
We also don't support el7 at all in oVirt 3.4.

This is an automated message

oVirt 3.4.1 has been released:
 * should fix your issue
 * should be available at your local mirror within two days.

If problems still persist, please make note of it in this bug report.