Bug 1174855

Summary: CFME 3.1 Zone level log collection not working as expected
Product: Red Hat CloudForms Management Engine Reporter: Thomas Hennessy <thenness>
Component: ApplianceAssignee: Joe Rafaniello <jrafanie>
Status: CLOSED ERRATA QA Contact: Jan Krocil <jkrocil>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 5.3.0CC: jhardy, jocarter, jrafanie, mfeifer, xlecauch
Target Milestone: GA   
Target Release: 5.4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: 5.4.0.0.11 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1182782 (view as bug list) Environment:
Last Closed: 2015-06-16 12:46:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1182782    

Description Thomas Hennessy 2014-12-16 15:46:29 UTC
Description of problem: When zone level log collection is initiated, log collection is initiated "N" times for one appliance instead of once for each of the "N" appliances in the zone.


Version-Release number of selected component (if applicable): Version: 5.3.1.10 Build:   20141112120709_c36c424 


How reproducible: 


Steps to Reproduce:
1. Create a multi-appliance single zone  CFME 3.1  environment (eg, 'default')
2. use the UI interface to create a zone-level depot for the zone created above (eg "redhat dropbox")
3. after log depot has been defined and saved, issue log collection for 'collect all'
4- after a few minutes, examine the depot to determine if logs have been uploaded for more than one appliance. (there hasn't)

Actual results: logs are uploaded for only one appliance, apparently the appliance from which the request has been issued.


Expected results: a full log set for each of the appliances within the zone for which log collection has been initiated.


Additional info:

Key log information from UI worker on appliance where log collection was initiated.  This zone has two appliances named "CFME-311-DB" AND "CFME-311-NONDB".  Notice in the log sequence that while two log collection message puts are issued, they are both for the same "CFME-311.DB" appliance, instead of one collection for each different appliance in the zone.
=====
[----] I, [2014-12-16T14:42:07.814943 #2295:c8d82c]  INFO -- : MIQ(LogFile-logs_from_server) Queueing the request by userid: [admin] for logs from server: [CFME-311-DB default MiqServer 440311000000000001]
[----] I, [2014-12-16T14:42:07.831897 #2295:c8d82c]  INFO -- : MIQ(MiqQueue.put)        Message id: [440311000000383352],  id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [LogFile._request_logs], Timeout: [1800], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:support_case=>"tch-Dec16", :taskid=>440311000000008629, :klass=>"MiqServer", :id=>440311000000000001}]
[----] I, [2014-12-16T14:42:07.832154 #2295:c8d82c]  INFO -- : MIQ(MiqTask.update_status) Task: [440311000000008629] [Queued] [Ok] [Queued the request for logs from server: [CFME-311-DB default MiqServer 440311000000000001]]
[----] I, [2014-12-16T14:42:07.843262 #2295:c8d82c]  INFO -- : MIQ(LogFile-logs_from_server) Task: [440311000000008629] Queued the request for logs from server: [CFME-311-DB default MiqServer 440311000000000001]
[----] I, [2014-12-16T14:42:07.843698 #2295:c8d82c]  INFO -- : MIQ(LogFile-logs_from_server) Queueing the request by userid: [admin] for logs from server: [CFME-311-DB default MiqServer 440311000000000001]
[----] I, [2014-12-16T14:42:07.861982 #2295:c8d82c]  INFO -- : MIQ(MiqQueue.put)        Message id: [440311000000383353],  id: [], Zone: [default], Role: [], Server: [], Ident: [generic], Target id: [], Instance id: [], Task id: [], Command: [LogFile._request_logs], Timeout: [1800], Priority: [20], State: [ready], Deliver On: [], Data: [], Args: [{:support_case=>"tch-Dec16", :taskid=>440311000000008630, :klass=>"MiqServer", :id=>440311000000000001}]
[----] I, [2014-12-16T14:42:07.862233 #2295:c8d82c]  INFO -- : MIQ(MiqTask.update_status) Task: [440311000000008630] [Queued] [Ok] [Queued the request for logs from server: [CFME-311-DB default MiqServer 440311000000000001]]
[----] I, [2014-12-16T14:42:07.868481 #2295:c8d82c]  INFO -- : MIQ(LogFile-logs_from_server) Task: [440311000000008630] Queued the request for logs from server: [CFME-311-DB default MiqServer 440311000000000001] 
======

Comment 2 Joe Rafaniello 2014-12-18 23:50:56 UTC
Thanks for the great description!

Work in progress branch: https://github.com/jrafanie/manageiq/compare/zone_collecting_only_one_server_logs_n_times

Need to add some tests to verify the new behavior works as originally intended.

Comment 3 Joe Rafaniello 2014-12-23 14:59:47 UTC
Upstream pull request: https://github.com/ManageIQ/manageiq/pull/1277

Comment 4 CFME Bot 2015-01-13 20:36:05 UTC
New commit detected on manageiq/master:
https://github.com/ManageIQ/manageiq/commit/4511cac601dc80cb0c70b390eb8f2e9a50e4bd78

commit 4511cac601dc80cb0c70b390eb8f2e9a50e4bd78
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Mon Dec 22 17:18:58 2014 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Mon Dec 22 17:29:30 2014 -0500

    Move specific setup out of the general LogManagement context.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1174855

 vmdb/spec/models/miq_server/log_management_spec.rb | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

Comment 5 CFME Bot 2015-01-13 20:36:09 UTC
New commit detected on manageiq/master:
https://github.com/ManageIQ/manageiq/commit/e3fd51da79beadc98597081fc7c08177aa2d9a09

commit e3fd51da79beadc98597081fc7c08177aa2d9a09
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Thu Dec 18 18:41:44 2014 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Tue Dec 23 15:29:46 2014 -0500

    Pass the server receiving the synchronize_logs message.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1174855
    
    If initiated from a zone, we want to create one miq_queue and miq_tasks row for each server in the Zone.
    
    Without this change, it will create N number of rows for the server receiving the web request, MiqServer.my_server, where N is the number of active servers in the zone.  It will never request logs for the other servers in the zone.
    
    This is fairly easy to recreate:
    1) Setup a Zone with 2 or more "active" appliances (miq_servers)
    2) Request log collection for the Zone
    3) Look at the log depot and find only the logs from the server receiving the web request.

 vmdb/app/models/miq_server/log_management.rb       |  4 +++-
 vmdb/spec/models/miq_server/log_management_spec.rb | 20 +++++++++++++++++++-
 vmdb/spec/models/shared_examples/log_collection.rb |  6 +++---
 3 files changed, 25 insertions(+), 5 deletions(-)

Comment 6 CFME Bot 2015-01-15 21:56:59 UTC
New commit detected on cfme/5.3.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=3fa2063c234ac5ec9994dbb839e33b6744253e3c

commit 3fa2063c234ac5ec9994dbb839e33b6744253e3c
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Mon Dec 22 17:18:58 2014 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Thu Jan 15 16:47:30 2015 -0500

    Move specific setup out of the general LogManagement context.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1174855

 vmdb/spec/models/miq_server/log_management_spec.rb | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

Comment 7 CFME Bot 2015-01-15 21:57:02 UTC
New commit detected on cfme/5.3.z:
https://code.engineering.redhat.com/gerrit/gitweb?p=cfme.git;a=commitdiff;h=6031ef8866183e4828e645dd6a3fe53b18e7067f

commit 6031ef8866183e4828e645dd6a3fe53b18e7067f
Author:     Joe Rafaniello <jrafanie>
AuthorDate: Thu Dec 18 18:41:44 2014 -0500
Commit:     Joe Rafaniello <jrafanie>
CommitDate: Thu Jan 15 16:47:36 2015 -0500

    Pass the server receiving the synchronize_logs message.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1174855
    
    If initiated from a zone, we want to create one miq_queue and miq_tasks row for each server in the Zone.
    
    Without this change, it will create N number of rows for the server receiving the web request, MiqServer.my_server, where N is the number of active servers in the zone.  It will never request logs for the other servers in the zone.
    
    This is fairly easy to recreate:
    1) Setup a Zone with 2 or more "active" appliances (miq_servers)
    2) Request log collection for the Zone
    3) Look at the log depot and find only the logs from the server receiving the web request.

 vmdb/app/models/miq_server/log_management.rb       |  4 +++-
 vmdb/spec/models/miq_server/log_management_spec.rb | 20 +++++++++++++++++++-
 vmdb/spec/models/shared_examples/log_collection.rb |  6 +++---
 3 files changed, 25 insertions(+), 5 deletions(-)

Comment 9 Jan Krocil 2015-06-05 12:56:00 UTC
Verified fixed in 5.4.0.4 (5.4.0.4.20150528174021_88cc76c).

With current behavior, appliance log depot must be unset for zone log depot to be used during zone log collection, otherwise appliance log depot is prioritized.

More information can be found here https://github.com/ManageIQ/manageiq/issues/1962 in the "Existing behavior" section.

Comment 11 errata-xmlrpc 2015-06-16 12:46:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1100.html