Bug 1460734 - traceback on hosts sos report failure
Summary: traceback on hosts sos report failure
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-log-collector
Classification: oVirt
Component: Core
Version: ---
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.2.0
: 4.2.0
Assignee: Sandro Bonazzola
QA Contact: David Necpal
URL:
Whiteboard:
Depends On:
Blocks: 1456888
TreeView+ depends on / blocked
 
Reported: 2017-06-12 14:36 UTC by Sandro Bonazzola
Modified: 2017-12-20 10:51 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-12-20 10:51:39 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-4.2+
rule-engine: planning_ack+
sbonazzo: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 78534 0 'None' MERGED CollectorBase: fix list index out of range 2020-06-18 03:56:14 UTC

Description Sandro Bonazzola 2017-06-12 14:36:45 UTC
Due to bug #1460732 sos 3.4 is failing while executed on hosts.
On ovirt-log-collector side, the failure cause a traceback:

About to collect information from 2 hypervisors. Continue? (Y/n): 
INFO: Gathering information from selected hypervisors...
INFO: collecting information from lago-rhv-suite-4-1-host1
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host1', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
INFO: collecting information from lago-rhv-suite-4-1-host0
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host0', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host0,192.168.201.3' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: message(list index out of range)
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG: Traceback (most recent call last):
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG: Configuration for lago-rhv-suite-4-1-host0:
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host1,192.168.201.2' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG: message(list index out of range)
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'engine': 'engine:443',
DEBUG: Traceback (most recent call last):
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host0',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:  'hosts': set([('test-dc',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG:                ('test-dc',
DEBUG: Configuration for lago-rhv-suite-4-1-host1:
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG:  'insecure': False,
DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'kerberos': False,
DEBUG:  'engine': 'engine:443',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host1',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'hosts': set([('test-dc',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:                ('test-dc',
DEBUG:  'max_connections': 10,
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'no_hypervisor': False,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG:  'no_postgresql': False,
DEBUG:  'output': '/dev/shm',
DEBUG:  'passwd': '123',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'insecure': False,
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'kerberos': False,
DEBUG:  'pg_dbport': '5432',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:  'pg_user': 'engine',
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:  'quiet': False,
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_port': 22,
DEBUG:  'max_connections': 10,
DEBUG:  'time_only': False,
DEBUG:  'no_hypervisor': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'no_postgresql': False,
DEBUG:  'verbose': True}
DEBUG:  'output': '/dev/shm',
INFO: finished collecting information from lago-rhv-suite-4-1-host0
DEBUG:  'passwd': '123',
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'pg_dbport': '5432',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'pg_user': 'engine',
DEBUG:  'quiet': False,
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_port': 22,
DEBUG:  'time_only': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'verbose': True}
INFO: finished collecting information from lago-rhv-suite-4-1-host1

Comment 1 David Necpal 2017-09-04 16:48:05 UTC
Verified on version: 
ovirt-log-collector-4.2.0-0.0.master.20170903141131.gitbd2607f.el7.centos.noarch

Comment 2 Sandro Bonazzola 2017-12-20 10:51:39 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.