Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1048270

Summary: Can not collect sosreports from hosts after upgrading to a 6.5 RHEV-H image
Product: Red Hat Enterprise Virtualization Manager Reporter: wdaniel
Component: ovirt-engine-log-collectorAssignee: Sandro Bonazzola <sbonazzo>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: urgent    
Version: 3.2.0CC: acathrow, benglish, iheim, Rhev-m-bugs, sbonazzo, wdaniel, yeylon
Target Milestone: ---Keywords: Reopened
Target Release: ---Flags: wdaniel: Triaged?
Hardware: x86_64   
OS: Linux   
Whiteboard: integration
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-22 14:40:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
result of '# sosreport -vvv' on 6.5 hypervisor none

Description wdaniel 2014-01-03 14:58:17 UTC
Created attachment 844977 [details]
result of '# sosreport -vvv' on 6.5 hypervisor

Description of problem:
After customers have installed/upgraded to a 6.5 host their sosreport tool reports errors collecting files or running commands. Using 'sosreport -vvv' shows that 'could not run command', 'IOError', and 'file already exists' occur very frequently.


Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor release 6.5 (20131115.0.3.2.el6_5)
sos-2.2-47.el6.noarch

How reproducible:
Very

Steps to Reproduce:
1. Upgrade hypervisors to a 6.5 image
2. Attempt to generate a log collector including the 6.5 based hypervisors
3.

Actual results:
rhevm-log-collector returns saying that there were errors collecting sosreports

Expected results:
rhevm-log-collector generates full sosreports for all specified hosts

Additional info:

These are a sample of the errors seen in the sos.log file from hypervisor:

2013-12-10 10:42:33,863 INFO: could not run command: /sbin/multipath -v4 -ll
2013-12-10 10:42:33,863 INFO: could not run command: /usr/bin/systool -v -c -b scsi
2013-12-10 10:42:42,106 INFO: error copying file /sys/bus/scsi/uevent (IOError)
2013-12-10 10:42:42,107 INFO: error copying file /sys/bus/scsi/drivers/sd/uevent (IOError)
2013-12-10 10:42:42,108 INFO: error copying file /sys/bus/scsi/drivers/sd/unbind (IOError)
2013-12-10 10:42:42,108 INFO: error copying file /sys/bus/scsi/drivers/sd/bind (IOError)
2013-12-10 10:42:42,108 INFO: error copying file /sys/bus/scsi/drivers/sr/uevent (IOError)
2013-12-10 10:42:42,109 INFO: error copying file /sys/bus/scsi/drivers/sr/unbind (IOError)
2013-12-10 10:42:42,109 INFO: error copying file /sys/bus/scsi/drivers/sr/bind (IOError)
2013-12-10 10:42:42,109 INFO: error copying file /sys/bus/scsi/drivers/ses/uevent (IOError)
2013-12-10 10:42:42,110 INFO: error copying file /sys/bus/scsi/drivers/ses/unbind (IOError)
2013-12-10 10:42:42,110 INFO: error copying file /sys/bus/scsi/drivers/ses/bind (IOError)
2013-12-10 10:42:42,110 INFO: error copying file /sys/bus/scsi/drivers_probe (IOError)
2013-12-10 10:42:42,408 INFO: could not run command: /usr/bin/cpufreq-info
2013-12-10 10:42:42,854 INFO: could not run command: /usr/bin/systool -c fc_host -v
2013-12-10 10:42:42,854 INFO: could not run command: /usr/bin/systool -c scsi_host -v

Comment 1 Sandro Bonazzola 2014-01-07 09:52:05 UTC
I'm not sure to follow.
The bug summary and description talks about rhevm-log-collector failing, but command line and logs are only from sos command execution.

Looking at the logs it seems that IOErrors are only on sockets or special files in /proc and /sys and AFAIK it was the same in 6.4.

About the "could not run command", I think it's desired behavior in sos tool: they're marked as INFO not as ERROR to just report about missing commands / tools.

So, I'm closing this as not a bug for log-collector.
Seems more something to be reported against sos package but not sure it's a bug for it too.
If you're reopening this bug against log-collector, please provide log collector log file when ran using -v flag, thanks.

Comment 2 wdaniel 2014-01-10 16:45:24 UTC
(In reply to Sandro Bonazzola from comment #1)
> I'm not sure to follow.
> The bug summary and description talks about rhevm-log-collector failing, but
> command line and logs are only from sos command execution.
> 
> Looking at the logs it seems that IOErrors are only on sockets or special
> files in /proc and /sys and AFAIK it was the same in 6.4.
> 
> About the "could not run command", I think it's desired behavior in sos
> tool: they're marked as INFO not as ERROR to just report about missing
> commands / tools.
> 
> So, I'm closing this as not a bug for log-collector.
> Seems more something to be reported against sos package but not sure it's a
> bug for it too.
> If you're reopening this bug against log-collector, please provide log
> collector log file when ran using -v flag, thanks.

Sandro,

I wanted to pass along the customer's response when I asked for some clarification:

However, to clarify, the original issue was that rhevm-log-collector was NOT collecting logs from any hypervisors. It said that it was collecting logs, it looked as if it was running through the collection process, but it errored out and failed. If you look at the log-collector that is attached to the case, it is only 19MB. A successful run is usually >1.5GB. I was able tpo log into the hypervisors individually and run sosreport manually and collect the logs one by one. 

He has also stated that he has upgraded the log collector package and it appears to have solved these issues, save for some odd line breaks and TTY issues. As such, I doubt his running 'rhevm-log-collector -v collect' will add much to the conversation at this point.

I'm going to look into recent errata releases related to that log collector package to see what was addressed, but is there anything else that needs to be investigated/clarified/tested for this bug?

Comment 3 Sandro Bonazzola 2014-01-15 07:51:14 UTC
(In reply to wdaniel from comment #2)

> He has also stated that he has upgraded the log collector package and it
> appears to have solved these issues, save for some odd line breaks and TTY
> issues. As such, I doubt his running 'rhevm-log-collector -v collect' will
> add much to the conversation at this point.

I agree. So, original issue has been solved by upgrading to latest rhevm-log-collector? If so, please move this to closed.

> I'm going to look into recent errata releases related to that log collector
> package to see what was addressed, but is there anything else that needs to
> be investigated/clarified/tested for this bug?

No, I don't need anything else.

Comment 4 Sandro Bonazzola 2014-01-22 14:40:18 UTC
Moving to closed, current release as per comment #3.