Bug 1098029

Summary: "rhsc-log-collector" takes too long even to prompt for the REST API password before it actually start collecting information from selected servers
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Prasanth <pprakash>
Component: rhsc-log-collectorAssignee: Darshan <dnarayan>
Status: CLOSED ERRATA QA Contact: Prasanth <pprakash>
Severity: high Docs Contact:
Priority: medium    
Version: rhgs-3.0CC: dnarayan, dpati, kmayilsa, kroberts, nlevinki, pprakash, rhs-bugs, rhsc-qe-bugs, sbonazzo
Target Milestone: ---   
Target Release: RHGS 3.0.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: integration
Fixed In Version: rhsc-log-collector-3.0.0-3.0.el6rhs Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-09-22 19:09:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1122965    

Description Prasanth 2014-05-15 06:37:13 UTC
Description of problem:

"rhsc-log-collector" takes too long even to prompt for the REST API password before it actually start collecting information from selected servers. 

----
# rhsc-log-collector 
INFO: Gathering oVirt Engine information...
INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost...
Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip):
----


Version-Release number of selected component (if applicable): rhsc-log-collector-3.0.0-2.0.el6rhs


How reproducible: 100%


Steps to Reproduce:
1. Execute # rhsc-log-collector from the Engine server and see the time it takes for the password prompt.
2.
3.

Actual results: It takes more than a minute even to prompt for the password before it actually starts collecting information from selected servers. See the relevant logs below:

----------
014-05-15 11:39:59::INFO::rhsc-log-collector::1295::root:: Gathering oVirt Engine information...
2014-05-15 11:40:46::INFO::rhsc-log-collector::1269::root:: Gathering PostgreSQL the oVirt Engine database and log files from localhost...
2014-05-15 11:41:56::INFO::rhsc-log-collector::1222::root:: Gathering information from selected servers...
2014-05-15 11:41:56::INFO::rhsc-log-collector::706::root:: collecting information from 10.70.43.98
2014-05-15 11:41:56::INFO::rhsc-log-collector::674::root:: Gluster logs will be collected from 10.70.43.98
2014-05-15 11:41:56::INFO::rhsc-log-collector::706::root:: collecting information from 10.70.43.35
2014-05-15 11:41:56::INFO::rhsc-log-collector::674::root:: Gluster logs will be collected from 10.70.43.35
2014-05-15 11:42:52::INFO::rhsc-log-collector::764::root:: finished collecting information from 10.70.43.35
2014-05-15 11:42:53::INFO::rhsc-log-collector::764::root:: finished collecting information from 10.70.43.98
2014-05-15 11:42:59::INFO::rhsc-log-collector::1652::root:: Log files have been collected and placed in /tmp/sosreport-LogCollector-20140515114253.tar.xz.
The MD5 for this file is 5ecf0faa48dedc7c62a87c9954ac090f and its size is 23.0M
----------


Expected results: It should'nt take too much time. Also, this was so quick in the previous builds.


Additional info:

Comment 1 Darshan 2014-05-20 12:31:35 UTC
   It collects the log from the rhsc-engine first before asking for password, so it takes that much time.

   To elaborate, it executes the following command to collect the logs related to engine, because of which there is a delay:

sosreport --batch --report --build --tmp-dir=/tmp/logcollector-Rk5etx -o engine,rpm,libvirt,general,networking,hardware,process,yum,filesys,devicemapper,selinux,kernel,apache,memory,ovirt_engine_dwh,ovirt_engine_reports -k rpm.rpmva=off -k general.all_logs=True -k apache.log=True -k engine.sensitive_keys=:ENGINE_DB_PASSWORD:ENGINE_PKI_TRUST_STORE_PASSWORD:ENGINE_PKI_ENGINE_STORE_PASSWORD

Comment 2 Prasanth 2014-05-21 12:06:31 UTC
(In reply to Darshan from comment #1)
>    It collects the log from the rhsc-engine first before asking for
> password, so it takes that much time.
> 
>    To elaborate, it executes the following command to collect the logs
> related to engine, because of which there is a delay:
> 
> sosreport --batch --report --build --tmp-dir=/tmp/logcollector-Rk5etx -o
> engine,rpm,libvirt,general,networking,hardware,process,yum,filesys,
> devicemapper,selinux,kernel,apache,memory,ovirt_engine_dwh,
> ovirt_engine_reports -k rpm.rpmva=off -k general.all_logs=True -k
> apache.log=True -k
> engine.sensitive_keys=:ENGINE_DB_PASSWORD:ENGINE_PKI_TRUST_STORE_PASSWORD:
> ENGINE_PKI_ENGINE_STORE_PASSWORD


If this is the current behaviour, then it has to fixed as it doesn't seems to be the right way. It should either prompt for the password before it start collecting the engine logs or it should show the progress of gathering the logs. I prefer the first one, i.e; prompting for the password before start gathering the engine logs due to the following condition:

* Lets assume that log-collector succesfully gathered all the engine logs
* Then iIt prompted for the REST API password
* User gave a wrong passsword 
So 
In the above case, the whole command fails at the middle of the process. Is it not better and makes more sense if it prompts for the password first and fails right away on an invalid entry?? If not, what do you suggest??

Comment 3 Sandro Bonazzola 2014-05-22 10:41:16 UTC
(In reply to Prasanth from comment #2)

> * Lets assume that log-collector succesfully gathered all the engine logs
> * Then iIt prompted for the REST API password
> * User gave a wrong passsword 
> So 
> In the above case, the whole command fails at the middle of the process. Is
> it not better and makes more sense if it prompts for the password first and
> fails right away on an invalid entry?? If not, what do you suggest??

I agree, make sense ask password before start log collection.
Note that if engine is down we need to be sure that log collector can collect engine host logs in any case. So a failure connecting to the engine should not cause log collector to exit.

Comment 4 Keith Robertson 2014-06-02 16:24:50 UTC
(In reply to Sandro Bonazzola from comment #3)
> (In reply to Prasanth from comment #2)
> 
> > * Lets assume that log-collector succesfully gathered all the engine logs
> > * Then iIt prompted for the REST API password
> > * User gave a wrong passsword 
> > So 
> > In the above case, the whole command fails at the middle of the process. Is
> > it not better and makes more sense if it prompts for the password first and
> > fails right away on an invalid entry?? If not, what do you suggest??
> 
> I agree, make sense ask password before start log collection.
> Note that if engine is down we need to be sure that log collector can
> collect engine host logs in any case. So a failure connecting to the engine
> should not cause log collector to exit.

Sandro and Prasanth, 

I originally designed the LC to explicitly not ask for the password upon launch because you don't need the PW to collect logs from the RHEV-M.  Hence, I didn't want to gate the collection of these logs on a non-responsive RHEV-M.  Translation... some logs are better than no logs.

I have no problem with this behavior changing as long as a non-responsive RHEV-M will not force the user to supply the 'no-hypervisors' option.

Keith

Comment 8 Prasanth 2014-06-12 09:18:29 UTC
Verified in rhsc-log-collector-3.0.0-3.0.el6rhs

------------
# rhsc-log-collector 
Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to skip): 
INFO: Gathering oVirt Engine information...
INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost...
About to collect information from 3 servers. Continue? (Y/n): y
INFO: Gathering information from selected servers...
INFO: collecting information from 10.70.43.69
INFO: Gluster logs will be collected from 10.70.43.69
INFO: collecting information from 10.70.42.241
INFO: Gluster logs will be collected from 10.70.42.241
INFO: collecting information from 10.70.42.167
INFO: Gluster logs will be collected from 10.70.42.167
INFO: finished collecting information from 10.70.43.69
INFO: finished collecting information from 10.70.42.167
INFO: finished collecting information from 10.70.42.241
Creating compressed archive...
INFO: Log files have been collected and placed in /tmp/sosreport-LogCollector-20140612130910.tar.xz.
The MD5 for this file is 5c5e3a39a229a9d8c07dc03fd6cefde3 and its size is 54.1M
------------

Comment 10 errata-xmlrpc 2014-09-22 19:09:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1277.html