Bug 1460734 - traceback on hosts sos report failure
traceback on hosts sos report failure
Status: CLOSED CURRENTRELEASE
Product: ovirt-log-collector
Classification: oVirt
Component: Core (Show other bugs)
---
Unspecified Unspecified
high Severity high (vote)
: ovirt-4.2.0
: 4.2.0
Assigned To: Sandro Bonazzola
David Necpal
:
Depends On:
Blocks: 1456888
  Show dependency treegraph
 
Reported: 2017-06-12 10:36 EDT by Sandro Bonazzola
Modified: 2017-12-20 05:51 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-12-20 05:51:39 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Integration
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.2+
rule-engine: planning_ack+
sbonazzo: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 78534 master MERGED CollectorBase: fix list index out of range 2017-06-22 12:18 EDT

  None (edit)
Description Sandro Bonazzola 2017-06-12 10:36:45 EDT
Due to bug #1460732 sos 3.4 is failing while executed on hosts.
On ovirt-log-collector side, the failure cause a traceback:

About to collect information from 2 hypervisors. Continue? (Y/n): 
INFO: Gathering information from selected hypervisors...
INFO: collecting information from lago-rhv-suite-4-1-host1
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host1', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
INFO: collecting information from lago-rhv-suite-4-1-host0
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host0', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host0,192.168.201.3' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: message(list index out of range)
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG: Traceback (most recent call last):
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG: Configuration for lago-rhv-suite-4-1-host0:
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host1,192.168.201.2' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG: message(list index out of range)
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'engine': 'engine:443',
DEBUG: Traceback (most recent call last):
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host0',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:  'hosts': set([('test-dc',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG:                ('test-dc',
DEBUG: Configuration for lago-rhv-suite-4-1-host1:
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG:  'insecure': False,
DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'kerberos': False,
DEBUG:  'engine': 'engine:443',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host1',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'hosts': set([('test-dc',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:                ('test-dc',
DEBUG:  'max_connections': 10,
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'no_hypervisor': False,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG:  'no_postgresql': False,
DEBUG:  'output': '/dev/shm',
DEBUG:  'passwd': '123',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'insecure': False,
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'kerberos': False,
DEBUG:  'pg_dbport': '5432',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:  'pg_user': 'engine',
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:  'quiet': False,
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_port': 22,
DEBUG:  'max_connections': 10,
DEBUG:  'time_only': False,
DEBUG:  'no_hypervisor': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'no_postgresql': False,
DEBUG:  'verbose': True}
DEBUG:  'output': '/dev/shm',
INFO: finished collecting information from lago-rhv-suite-4-1-host0
DEBUG:  'passwd': '123',
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'pg_dbport': '5432',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'pg_user': 'engine',
DEBUG:  'quiet': False,
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_port': 22,
DEBUG:  'time_only': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'verbose': True}
INFO: finished collecting information from lago-rhv-suite-4-1-host1
Comment 1 David Necpal 2017-09-04 12:48:05 EDT
Verified on version: 
ovirt-log-collector-4.2.0-0.0.master.20170903141131.gitbd2607f.el7.centos.noarch
Comment 2 Sandro Bonazzola 2017-12-20 05:51:39 EST
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.

Note You need to log in before you can comment on or make changes to this bug.