Bug 1560845

Summary: sosreport does not collect openvswtich container
Product: Red Hat Enterprise Linux 7 Reporter: Eran Kuris <ekuris>
Component: sosAssignee: Pavel Moravec <pmoravec>
Status: CLOSED ERRATA QA Contact: Miroslav HradĂ­lek <mhradile>
Severity: high Docs Contact:
Priority: high    
Version: 7.4CC: agk, bmr, dalvarez, ekuris, gavin, mhradile, nusiddiq, plambri, pmoravec, sbradley
Target Milestone: pre-dev-freezeKeywords: OtherQA, Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: sos-3.5-8.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1580525 (view as bug list) Environment:
Last Closed: 2018-10-30 10:31:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1503796, 1580525    

Description Eran Kuris 2018-03-27 06:19:39 UTC
Description of problem:
During OVN testing I noticed that  sosreport do not collect openvswtich container  logs.
/sosreport-controller-0-20180326122314/var/log/containers/
/sosreport-compute-1-20180326122358/var/log/containers/

The problem reproduce on controller & compute nodes. 

Version-Release number of selected component (if applicable):
OSP13 -p 2018-03-02.2

How reproducible:
always

Steps to Reproduce:
1.Deploy OSP13-OVN HA setup 
2. create a setup with network, router & few instances so we have some inputs in logs. 
3. run sosreport on the host & check the file that created

Actual results:
openvswtich container logs do not exist in the sosreport file.

Expected results:
all openvswtich container logs  should be exist 

Additional info:
adding sosreports files.

Comment 5 Pavel Moravec 2018-04-04 11:34:25 UTC
Checking what sos collects now (*), my understanding is you should rather require logs from some subdirectory like /var/log/containers/openvswitch or similar?

Since the directory /var/log/containers can have very rich content, much of it irrelevant to openvswitch, I expect.

(*) https://github.com/sosreport/sos/search?utf8=%E2%9C%93&q=%22%2Fvar%2Flog%2Fcontainers%22&type=

Further, like in bz1560849, is whole directory /var/log/containers/{some-subdir?} required to collect, or is it sufficient to:

- collect /var/log/containers/{some-subdir?}*.log by default
- if --all-logs option provided, whole dir collected

?

(and again, automatic prerequisite to collect these logs will be presence of openvswitch or openvswitch-dpdk package, I assume)

Comment 6 Eran Kuris 2018-04-08 05:40:39 UTC
(In reply to Pavel Moravec from comment #5)
> Checking what sos collects now (*), my understanding is you should rather
> require logs from some subdirectory like /var/log/containers/openvswitch or
> similar?
> 
> Since the directory /var/log/containers can have very rich content, much of
> it irrelevant to openvswitch, I expect.
> 
> (*)
> https://github.com/sosreport/sos/
> search?utf8=%E2%9C%93&q=%22%2Fvar%2Flog%2Fcontainers%22&type=
> 
> Further, like in bz1560849, is whole directory
> /var/log/containers/{some-subdir?} required to collect, or is it sufficient
> to:
> 
> - collect /var/log/containers/{some-subdir?}*.log by default
> - if --all-logs option provided, whole dir collected
> 
> ?
> 
> (and again, automatic prerequisite to collect these logs will be presence of
> openvswitch or openvswitch-dpdk package, I assume)

When I enter to my setup I see the openvswitch logs under  var/log/containers/openvswitch, my expectation is when I  collect the logs with sos report I will see the logs with the same path of : var/log/containers/openvswitch

Comment 7 Pavel Moravec 2018-04-08 12:41:43 UTC
(In reply to Eran Kuris from comment #6)
> (In reply to Pavel Moravec from comment #5)
> > Checking what sos collects now (*), my understanding is you should rather
> > require logs from some subdirectory like /var/log/containers/openvswitch or
> > similar?
> > 
> > Since the directory /var/log/containers can have very rich content, much of
> > it irrelevant to openvswitch, I expect.
> > 
> > (*)
> > https://github.com/sosreport/sos/
> > search?utf8=%E2%9C%93&q=%22%2Fvar%2Flog%2Fcontainers%22&type=
> > 
> > Further, like in bz1560849, is whole directory
> > /var/log/containers/{some-subdir?} required to collect, or is it sufficient
> > to:
> > 
> > - collect /var/log/containers/{some-subdir?}*.log by default
> > - if --all-logs option provided, whole dir collected
> > 
> > ?
> > 
> > (and again, automatic prerequisite to collect these logs will be presence of
> > openvswitch or openvswitch-dpdk package, I assume)
> 
> When I enter to my setup I see the openvswitch logs under 
> var/log/containers/openvswitch, my expectation is when I  collect the logs
> with sos report I will see the logs with the same path of :
> var/log/containers/openvswitch

Thanks for clarification. 

Wrt. bz1560849 / PR1259 I forgot to ask: when these logs shall be automatically collected? Presence of what package should trigger those logs collection? (or in sos terminology: what plugin should collect them)?

- plugin openvswitch is automatically enabled by presence of either 'openvswitch', or 'openvswitch-dpdk' package
- proposed plugin ovn_central will be automatically enabled by presence of openvswitch-ovn-central package
- proposed plugin ovn_host will be automatically enabled by presence of openvswitch-ovn-host package

What package / plugin is responsible for the (container) logs, please?

Comment 8 Eran Kuris 2018-04-08 13:11:14 UTC
Numan can you answer that ^ 
I am not sure about it.

Comment 9 Daniel Alvarez Sanchez 2018-04-17 12:54:22 UTC
Just to clarify, what we need is that the OVN sosreport plugin collects the logs from [0] (which defaults to /var/log/containers/openvswitch).

[0] https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/pacemaker/ovn_dbs_bundle.pp#L130

Comment 13 Numan Siddique 2018-05-02 09:46:14 UTC
(In reply to Pavel Moravec from comment #7)
> (In reply to Eran Kuris from comment #6)
> > (In reply to Pavel Moravec from comment #5)
> > > Checking what sos collects now (*), my understanding is you should rather
> > > require logs from some subdirectory like /var/log/containers/openvswitch or
> > > similar?
> > > 
> > > Since the directory /var/log/containers can have very rich content, much of
> > > it irrelevant to openvswitch, I expect.
> > > 
> > > (*)
> > > https://github.com/sosreport/sos/
> > > search?utf8=%E2%9C%93&q=%22%2Fvar%2Flog%2Fcontainers%22&type=
> > > 
> > > Further, like in bz1560849, is whole directory
> > > /var/log/containers/{some-subdir?} required to collect, or is it sufficient
> > > to:
> > > 
> > > - collect /var/log/containers/{some-subdir?}*.log by default
> > > - if --all-logs option provided, whole dir collected
> > > 
> > > ?
> > > 
> > > (and again, automatic prerequisite to collect these logs will be presence of
> > > openvswitch or openvswitch-dpdk package, I assume)
> > 
> > When I enter to my setup I see the openvswitch logs under 
> > var/log/containers/openvswitch, my expectation is when I  collect the logs
> > with sos report I will see the logs with the same path of :
> > var/log/containers/openvswitch
> 
> Thanks for clarification. 
> 
> Wrt. bz1560849 / PR1259 I forgot to ask: when these logs shall be
> automatically collected? Presence of what package should trigger those logs
> collection? (or in sos terminology: what plugin should collect them)?
> 
> - plugin openvswitch is automatically enabled by presence of either
> 'openvswitch', or 'openvswitch-dpdk' package
> - proposed plugin ovn_central will be automatically enabled by presence of
> openvswitch-ovn-central package
> - proposed plugin ovn_host will be automatically enabled by presence of
> openvswitch-ovn-host package
> 
> What package / plugin is responsible for the (container) logs, please?

Sorry for the late response.

I would say the propose plugin ovn_central should also collect the container logs (when openvswitch-ovn-central package) is enabled. I am not sure if proposed ovn_central plugin is doing it.

Comment 14 Numan Siddique 2018-05-02 09:50:22 UTC
The pull request here [1] which adds the ovn_central plugin does collect the logs as well.

[1] - https://github.com/sosreport/sos/pull/1259/files

Comment 16 Pavel Moravec 2018-05-02 11:32:06 UTC
For 7.5.2: 
- GSSApproved for the z-stream

- please confirm OtherQE / somebody from OSP / OVN people will verify the sos build once ready
(as 2 plugins would be backported whereas they provide bit more functionality beyond RHEL platform what would need more time to our QE to verify it)

Comment 18 Eran Kuris 2018-05-03 05:08:47 UTC
(In reply to Pavel Moravec from comment #16)
> For 7.5.2: 
> - GSSApproved for the z-stream
> 
> - please confirm OtherQE / somebody from OSP / OVN people will verify the
> sos build once ready
> (as 2 plugins would be backported whereas they provide bit more
> functionality beyond RHEL platform what would need more time to our QE to
> verify it)

Once the fix will be part of OSP puddle I will verify the issue.

Comment 21 Pavel Moravec 2018-05-30 11:15:29 UTC
*** Bug 1560849 has been marked as a duplicate of this bug. ***

Comment 26 errata-xmlrpc 2018-10-30 10:31:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:3144