Bug 1460734 - traceback on hosts sos report failure
traceback on hosts sos report failure
Status: VERIFIED
Product: ovirt-log-collector
Classification: oVirt
Component: Core (Show other bugs)
---
Unspecified Unspecified
high Severity high (vote)
: ovirt-4.2.0
: 4.2.0
Assigned To: Sandro Bonazzola
David Necpal
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-12 10:36 EDT by Sandro Bonazzola
Modified: 2017-09-04 12:48 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Integration
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.2+
rule-engine: planning_ack+
sbonazzo: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 78534 master MERGED CollectorBase: fix list index out of range 2017-06-22 12:18 EDT

  None (edit)
Description Sandro Bonazzola 2017-06-12 10:36:45 EDT
Due to bug #1460732 sos 3.4 is failing while executed on hosts.
On ovirt-log-collector side, the failure cause a traceback:

About to collect information from 2 hypervisors. Continue? (Y/n): 
INFO: Gathering information from selected hypervisors...
INFO: collecting information from lago-rhv-suite-4-1-host1
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host1', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
INFO: collecting information from lago-rhv-suite-4-1-host0
DEBUG: calling(['/usr/bin/ssh', '-n', '-p', '22', '-i', '/etc/pki/ovirt-engine/keys/engine_id_rsa', '-oStrictHostKeyChecking=no', '-oServerAliveInterval=600', 'root@lago-rhv-suite-4-1-host0', "\nVERSION=`/bin/rpm -q --qf '[%{VERSION}]' sos | /bin/sed 's/\\.//'`;\nif [ $VERSION -ge 32 ]; then\n    /usr/sbin/sosreport  --batch --all-logs         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd,ovirt_hosted_engine\nelif [ $VERSION -ge 30 ]; then\n    /usr/sbin/sosreport  --batch -k logs.all_logs=True         -o logs,libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm,processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd\nelif [ $VERSION -ge 22 ]; then\n    /usr/sbin/sosreport  --batch -k general.all_logs=True         -o libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm\nelif [ $VERSION -ge 17 ]; then\n    /usr/sbin/sosreport  --no-progressbar -k general.all_logs=True         -o vdsm,general,networking,hardware,process,yum,filesys\nelse\n    /bin/echo No", 'valid', 'version', 'of', 'sosreport', 'found. 1>&2\n    exit 1\nfi\n'])
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host0,192.168.201.3' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: message(list index out of range)
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG: Traceback (most recent call last):
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG: Configuration for lago-rhv-suite-4-1-host0:
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 3.4)

)
DEBUG: STDERR(Warning: Permanently added 'lago-rhv-suite-4-1-host1,192.168.201.2' (ECDSA) to the list of known hosts.
plugin vdsm does not install, skipping: type object 'Plugin' has no attribute 'add_copy_spec_limit'
a non-existing plugin (vdsm) was specified in the command line
)
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG: message(list index out of range)
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG: parse_sosreport_stdout: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 554, in parse_sosreport_stdout
    if fileAry[0] is not None and len(fileAry) > 0:
IndexError: list index out of range

DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'engine': 'engine:443',
DEBUG: Traceback (most recent call last):
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host0',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 755, in run
DEBUG:     self.parse_sosreport_stdout(stdout)
DEBUG:  'hosts': set([('test-dc',
DEBUG:   File "/usr/lib/python2.7/site-packages/ovirt_log_collector/__main__.py", line 590, in parse_sosreport_stdout
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:     "Could not parse sosreport output to determine filename"
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG: Exception: Could not parse sosreport output to determine filename
DEBUG:                ('test-dc',
DEBUG: Configuration for lago-rhv-suite-4-1-host1:
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG: {'bc_reports': 'vdsm,general,networking,hardware,process,yum,filesys',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'cert_file': '/etc/pki/ovirt-engine/ca.pem',
DEBUG:  'insecure': False,
DEBUG:  'conf_file': '/root/ovirt-log-collector.conf',
DEBUG:  'kerberos': False,
DEBUG:  'engine': 'engine:443',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'hostname': 'lago-rhv-suite-4-1-host1',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'hosts': set([('test-dc',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:                 'lago-rhv-suite-4-1-host0'),
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:                ('test-dc',
DEBUG:  'max_connections': 10,
DEBUG:                 <ovirt_log_collector.helper.hypervisors.Cluster object at 0x26bd290>,
DEBUG:  'no_hypervisor': False,
DEBUG:                 'lago-rhv-suite-4-1-host1')]),
DEBUG:  'no_postgresql': False,
DEBUG:  'output': '/dev/shm',
DEBUG:  'passwd': '123',
DEBUG:  'include_sensitive_data': False,
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'insecure': False,
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'kerberos': False,
DEBUG:  'pg_dbport': '5432',
DEBUG:  'key_file': '/etc/pki/ovirt-engine/keys/engine_id_rsa',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'local_scratch_dir': '/dev/shm/log/working/log-collector-data',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'local_tmp_dir': '/dev/shm/log',
DEBUG:  'pg_user': 'engine',
DEBUG:  'local_working_dir': '/dev/shm/log/working',
DEBUG:  'quiet': False,
DEBUG:  'log_file': '/var/log/ovirt-engine/ovirt-log-collector/ovirt-log-collector-20170612082725.log',
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host0',
DEBUG:  'ssh_port': 22,
DEBUG:  'max_connections': 10,
DEBUG:  'time_only': False,
DEBUG:  'no_hypervisor': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'no_postgresql': False,
DEBUG:  'verbose': True}
DEBUG:  'output': '/dev/shm',
INFO: finished collecting information from lago-rhv-suite-4-1-host0
DEBUG:  'passwd': '123',
DEBUG:  'pg_dbhost': 'localhost',
DEBUG:  'pg_dbname': 'engine',
DEBUG:  'pg_dbport': '5432',
DEBUG:  'pg_pass': 'MiQPXNuVwbkiOaPG1zXeNH',
DEBUG:  'pg_ssh_user': 'root',
DEBUG:  'pg_user': 'engine',
DEBUG:  'quiet': False,
DEBUG:  'reports': 'libvirt,vdsm,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,memory,rpm',
DEBUG:  'reports3': 'processor,pci,md,block,scsi,multipath,systemd,sanlock,lvm2,chrony,systemd',
DEBUG:  'scp_cmd': '/usr/bin/scp -P 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_cmd': '/usr/bin/ssh -n -p 22 -i /etc/pki/ovirt-engine/keys/engine_id_rsa -oStrictHostKeyChecking=no -oServerAliveInterval=600 root@lago-rhv-suite-4-1-host1',
DEBUG:  'ssh_port': 22,
DEBUG:  'time_only': False,
DEBUG:  'user': 'admin@internal',
DEBUG:  'verbose': True}
INFO: finished collecting information from lago-rhv-suite-4-1-host1
Comment 1 David Necpal 2017-09-04 12:48:05 EDT
Verified on version: 
ovirt-log-collector-4.2.0-0.0.master.20170903141131.gitbd2607f.el7.centos.noarch

Note You need to log in before you can comment on or make changes to this bug.