Bug 1376370

Summary: Optimizer: outdated data received due to connection re-use making the optimizer ineffective
Product: [Retired] ovirt-optimizer Reporter: Shira Maximov <mshira>
Component: GeneralAssignee: Martin Sivák <msivak>
Status: CLOSED CURRENTRELEASE QA Contact: Shira Maximov <mshira>
Severity: high Docs Contact:
Priority: high    
Version: 0.10CC: amarchuk, bugs, dfediuck, mavital, mgoldboi, mshira, msivak
Target Milestone: ovirt-4.0.4Keywords: TestOnly
Target Release: ---Flags: rule-engine: ovirt-4.0.z+
rule-engine: ovirt-4.1+
mgoldboi: planning_ack+
dfediuck: devel_ack+
mavital: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-17 13:06:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1376402    
Bug Blocks: 1358794    
Attachments:
Description Flags
server and optimizer logs.
none
engine log
none
logs none

Description Shira Maximov 2016-09-15 08:55:19 UTC
Created attachment 1201162 [details]
server and optimizer logs.

Description of problem:
After installing the optimizer the status is:
Could not connect to the optimizer service [status code: 404]

in the server.log: 
016-09-14 10:58:25,927 ERROR [stderr] (Thread-78) Exception in thread "Thread-78" org.ovirt.engine.sdk4.Error: Failed to send request
2016-09-14 10:58:25,929 ERROR [stderr] (Thread-78)      at org.ovirt.engine.sdk4.internal.HttpConnection.send(HttpConnection.java:212)
2016-09-14 10:58:25,930 ERROR [stderr] (Thread-78)      at org.ovirt.engine.sdk4.internal.services.AffinityLabelsServiceImpl$ListRequestImpl.send(AffinityLabelsServiceImpl.java:152)
2016-09-14 10:58:25,930 ERROR [stderr] (Thread-78)      at org.ovirt.engine.sdk4.internal.services.AffinityLabelsServiceImpl$ListRequestImpl.send(AffinityLabelsServiceImpl.java:122)
2016-09-14 10:58:25,931 ERROR [stderr] (Thread-78)      at org.ovirt.optimizer.solver.thread.ClusterInfoUpdater.run(ClusterInfoUpdater.java:97)
2016-09-14 10:58:25,931 ERROR [stderr] (Thread-78)      at java.lang.Thread.run(Thread.java:745)
2016-09-14 10:58:25,932 ERROR [stderr] (Thread-78) Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
2016-09-14 10:58:25,932 ERROR [stderr] (Thread-78) Make sure to release the connection before allocating another one.
2016-09-14 10:58:25,933 ERROR [stderr] (Thread-78)      at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:162)
2016-09-14 10:58:25,934 ERROR [stderr] (Thread-78)      at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:139)
2016-09-14 10:58:25,934 ERROR [stderr] (Thread-78)      at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:456)
2016-09-14 10:58:25,935 ERROR [stderr] (Thread-78)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
2016-09-14 10:58:25,935 ERROR [stderr] (Thread-78)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
2016-09-14 10:58:25,958 ERROR [stderr] (Thread-78)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
2016-09-14 10:58:25,958 ERROR [stderr] (Thread-78)      at org.ovirt.engine.sdk4.internal.HttpClient42.execute(HttpClient42.java:26)
2016-09-14 10:58:25,959 ERROR [stderr] (Thread-78)      at org.ovirt.engine.sdk4.internal.HttpConnection.send(HttpConnection.java:209)
2016-09-14 10:58:25,959 ERROR [stderr] (Thread-78)      ... 4 more
2016-09-14 11:24:58,774 ERROR [stderr] (Thread-82) Exception in thread "Thread-82" org.ovirt.engine.sdk4.Error: Failed to send request
2016-09-14 11:24:58,775 ERROR [stderr] (Thread-82)      at org.ovirt.engine.sdk4.internal.HttpConnection.send(HttpConnection.java:212)
2016-09-14 11:24:58,775 ERROR [stderr] (Thread-82)      at org.ovirt.engine.sdk4.internal.services.AffinityLabelsServiceImpl$ListRequestImpl.send(AffinityLabelsServiceImpl.java:152)
2016-09-14 11:24:58,776 ERROR [stderr] (Thread-82)      at org.ovirt.engine.sdk4.internal.services.AffinityLabelsServiceImpl$ListRequestImpl.send(AffinityLabelsServiceImpl.java:122)
2016-09-14 11:24:58,776 ERROR [stderr] (Thread-82)      at org.ovirt.optimizer.solver.thread.ClusterInfoUpdater.run(ClusterInfoUpdater.java:97)
2016-09-14 11:24:58,776 ERROR [stderr] (Thread-82)      at java.lang.Thread.run(Thread.java:745)
2016-09-14 11:24:58,777 ERROR [stderr] (Thread-82) Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
2016-09-14 11:24:58,777 ERROR [stderr] (Thread-82) Make sure to release the connection before allocating another one.
2016-09-14 11:24:58,777 ERROR [stderr] (Thread-82)      at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:162)
2016-09-14 11:24:58,777 ERROR [stderr] (Thread-82)      at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:139)
2016-09-14 11:24:58,778 ERROR [stderr] (Thread-82)      at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:456)
2016-09-14 11:24:58,778 ERROR [stderr] (Thread-82)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
2016-09-14 11:24:58,778 ERROR [stderr] (Thread-82)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
2016-09-14 11:24:58,778 ERROR [stderr] (Thread-82)      at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
2016-09-14 11:24:58,779 ERROR [stderr] (Thread-82)      at org.ovirt.engine.sdk4.internal.HttpClient42.execute(HttpClient42.java:26)
2016-09-14 11:24:58,779 ERROR [stderr] (Thread-82)      at org.ovirt.engine.sdk4.internal.HttpConnection.send(HttpConnection.java:209)
2016-09-14 11:24:58,779 ERROR [stderr] (Thread-82)      ... 4 more



Version-Release number of selected component (if applicable):
ovirt-optimizer-0.11-1.el7ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. install the optimizer
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Martin Sivák 2016-09-15 11:32:46 UTC
This will be test only once the SDK issue is resolved.

Comment 2 Shira Maximov 2016-09-25 07:43:06 UTC
failed to verify on :
ovirt-engine-4.0.4.4-0.1.el7ev.noarch
optaplanner-6.4.0-6.Final_redhat_4.1.el7.noarch

Martin, the sdk issue seems to be solved but the optimizer still doesn't work, 
after installing the optimizer and restarting the ovirt engine, the service ovirt-websocket-proxy failed to run.. 
the error :
SSL_ONLY is set but SSL_CERTIFICATE or SSL_KEY file not found.
 
I saw this error : 
Caused by: java.io.FileNotFoundException: /etc/pki/ovirt-engine/keys/engine.p12 (Permission denied)

In engine logs, the permissions for the file seems like in other envs :
[root@mshira40 ~]# ll /etc/pki/ovirt-engine/keys/engine.p12
-rw-------. 1 ovirt root 2813 Jul  4 12:52 /etc/pki/ovirt-engine/keys/engine.p12

note- the optimizer installation was done with the ansible script

Comment 3 Shira Maximov 2016-09-25 07:44:57 UTC
Created attachment 1204491 [details]
engine log

Comment 4 Red Hat Bugzilla Rules Engine 2016-09-25 08:50:33 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 5 Martin Sivák 2016-09-26 15:23:09 UTC
Shira - ovirt websocket proxy has nothing to do with the optimizer. If the engine can't start then you hit a completely different issue.

The ansible script configures the ui plugin on the engine machine and the optimizer machine. It does not touch the engine itself.

Comment 7 Shira Maximov 2016-09-28 11:28:15 UTC
Created attachment 1205469 [details]
logs

Comment 10 Shira Maximov 2016-09-29 13:31:03 UTC
moving to verify