Bug 1122926 - [RFE][engine-log-collector] make engine-log-collector do all job via engine (no direct access to db host, hypervisors)
Summary: [RFE][engine-log-collector] make engine-log-collector do all job via engine (...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: oVirt
Classification: Retired
Component: ovirt-engine-logcollector
Version: 3.5
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 3.5.0
Assignee: Sandro Bonazzola
QA Contact: Jiri Belka
URL:
Whiteboard: integration
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-24 12:26 UTC by Jiri Belka
Modified: 2014-12-15 15:06 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-08-08 11:04:52 UTC
oVirt Team: ---


Attachments (Terms of Use)

Description Jiri Belka 2014-07-24 12:26:50 UTC
Description of problem:

make engine-log-collector do all job via engine (no direct access to db host, hypervisors).

engine-log-collector needs a user to access to engine (like admin@internal).

IMHO engine-log-collector should instruct its (tbd) counter-part on engine host to take care of everything - that is: to load /etc/ovirt-engine/engine.conf.d/10-setup-database.conf transparently, make dump/query of the database itself, collect all data from hypervisors.
Then the output should be delivered to the user via engine-log-collector.

FYI engine already knows DB credentials, engine already has its ssh public keys on hosts so engine is the king here and has access where it needs.

there should not be need for direct access from place where engine-log-collector is executed (for example RHEV admin laptop) to DB host (if a separated host is used for the database) and hosts themselves.

in some enterprise environments there's split of scopes and thus direct access can be limited (db admins only to db hosts, no direct access from RHEV admins to hypervisors etc...).

in ideal world i'd like to see something like this:

1. engine-log-collector -r $my_engine -u admin@internal -> *.tar.xz finishes on my laptop (no direct access to hosts/db hosts)
2. we have Log Viewer but log-collector should have it's GUI part as well, just click in GUI and you end with a tarball offered for download

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Jiri Belka 2014-07-24 12:29:56 UTC
Is following part executed on engine host (if it would be remote host) or from the place where engine-log-collector was executed? If the latter this would conflict with this RFE :)

engine-log-collector -r jb-ovirt35.rhev.lab.eng.brq.redhat.com -u admin@internal --conf-file=/dev/null -vvv
...snip...
DEBUG: STDERR()
INFO: Gathering PostgreSQL the oVirt Engine database and log files from localhost...
DEBUG: calling(['/usr/sbin/sosreport', '--batch', '-o', 'postgresql', '--tmp-dir=/tmp/logcollector-Frg3zd/log-collector-data', '--report', '-k',
 'postgresql.dbname=engine', '-k', 'postgresql.dbhost=localhost', '-k', 'postgresql.dbport=5432', '-k', 'postgresql.username=engine', '-k', 'pos
tgresql.password=wFhA3QYlu9T2yt7pVnC4ef'])
DEBUG: returncode(0)
DEBUG: STDOUT(
sosreport (version 2.2)
...snip...

Comment 2 Sandro Bonazzola 2014-07-25 14:01:06 UTC
Adding some more people to the discussion since the RFE requires a work out of log-collector package and impacts on other projects.

Comment 3 Sandro Bonazzola 2014-07-25 14:02:01 UTC
(In reply to Jiri Belka from comment #1)
> Is following part executed on engine host (if it would be remote host) or
> from the place where engine-log-collector was executed?

It's executed on the host running engine-log-collector, which usually is the engine host.

Comment 4 Itamar Heim 2014-08-03 20:59:47 UTC
I thought that when the engine collects the logs, it actually runs log collector code?

Comment 5 Keith Robertson 2014-08-04 14:48:40 UTC
(In reply to Itamar Heim from comment #4)
> I thought that when the engine collects the logs, it actually runs log
> collector code?

I think we are talking about two separate scenarios.  I am going to try to address both below:

=== Scenario 1: Red Hat Access Plug-in ===
If the suggestion is for the Red Hat Access plug-in to call to the LC to get logs then I will say this is a bad idea.  The Red Hat Access plug-in uses native Java SSH libraries to collect logs from the hypervisors so that we have excellent feedback without having to deal with a shell forked process.  Running shell scripts from a Java process is generally a bad idea and should be avoided in favor of native calls and this is what we have done.  Same is also true for dumping the DB as we use a JDBC driver to to this and not pg_dump.

As it relates to the discovery of IP addresses, passwords, etc., the Red Hat Access plug-in either uses engine's code to do this or the REST API.  We do not write custom code if the engine does what we need via an API.

=== Scenario 2: The LC Should Proxy All Calls through Engine ===
I think Jiri is suggesting in this BZ that the LC should use the engine exclusively to vacuum up logs from hypervisors, dump the DB, and collect SoS reports from the RHEV-M host.

If this is what Jiri is asking for then I think this is a terrible idea because the LC must operate when the engine is completely down.  The LC is the method of last resort for our users to collect logs and give them to support.  As such, it should have as few dependencies on JBoss actually running as possible.

- Today, the engine does not need to be on-line for the LC to collect logs from a hypervisor.  Users can type in an IP address if they want and the LC will SFTP them.
- Today, the engine does not need to be on-line to dump the DB.  The LC will use pg_dump to do this.
- Today, the engine does not need to be on-line to collect SoS reports from the RHEV-M.

Comment 6 Sandro Bonazzola 2014-08-08 11:04:52 UTC
As per comment #5, both described scenario are not something we want.
Closing as wontfix accordingly.


Note You need to log in before you can comment on or make changes to this bug.