Bug 996510

Summary: RHEVM cpu utilization is very high
Product: Red Hat Enterprise Virtualization Manager Reporter: Nikhil Gupta <ngupta>
Component: ovirt-engine-dwhAssignee: Yaniv Lavi <ylavi>
Status: CLOSED INSUFFICIENT_DATA QA Contact: David Botzer <dbotzer>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.1.3CC: acathrow, iheim, lzelkha, ngupta, pstehlik, Rhev-m-bugs, yeylon
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-18 14:26:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nikhil Gupta 2013-08-13 10:23:47 UTC
Description of problem:
RHEVM cpu utilization is very high and getting following error in ovirt-engine-dwhd.log:
<snip>
2013-08-13 12:10:00|EIcH09|PrZXdj|EmhjU1|OVIRT_ENGINE_DWH|SampleTimeKeepingJob|Default|6|Java Exception|tJava_1|javax.net.ssl.SSLHandshakeException:java.security.cert.CertificateException: No name matching xyz found|1
</snip>

Version-Release number of selected component (if applicable):
rhevm-3.1.0-40.el6ev.noarch
rhevm-dwh-3.1.3-2.el6ev.noarch
rhevm-setup-3.1.0-40.el6ev.noarch

How reproducible:
1) Added below entry in /etc/hosts:

1.2.3.4    xyz.example.com    xyz

2) Changing "Default.properties" in "/etc/ovirt-engine/ovirt-engine-dwh/"

from 
ovirtEnginePortalAddress=xyz.example.com 

to 
ovirtEnginePortalAddress=xyz

3) Logs in "/var/log/ovirt-engine/ovirt-engine-dwhd.log":

2013-08-13 12:09:35|dLupD0|dLupD0|dLupD0|OVIRT_ENGINE_DWH|HistoryETL|Default|6|Java Exception|tJDBCOutput_2|org.postgresql.util.PSQLException:FATAL: terminating connection due to administrator command|1
2013-08-13 12:09:44|ETL Service Started
Exception in component tJava_1
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching xyz found
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
        at ovirt_engine_dwh.sampletimekeepingjob_3_1.SampleTimeKeepingJob.tJava_1Process(SampleTimeKeepingJob.java:1123)
        at ovirt_engine_dwh.sampletimekeepingjob_3_1.SampleTimeKeepingJob$1.run(SampleTimeKeepingJob.java:4242)
Caused by: java.security.cert.CertificateException: No name matching xyz found
        at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:208)
        at sun.security.util.HostnameChecker.match(HostnameChecker.java:93)
        at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:347)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:203)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323)
        ... 12 more
2013-08-13 12:10:00|EIcH09|PrZXdj|EmhjU1|OVIRT_ENGINE_DWH|SampleTimeKeepingJob|Default|6|Java Exception|tJava_1|javax.net.ssl.SSLHandshakeException:java.security.cert.CertificateException: No name matching xyz found|1


Actual results:
CPU utilization is very high.

Expected results:
CPU utilization should be normal.

Comment 2 Yaniv Lavi 2013-08-13 10:27:09 UTC
What process is causing the high CPU utilization?



Yaniv

Comment 5 Yaniv Lavi 2013-08-13 11:37:15 UTC
What are the server specs?
It's definitely not a dwh issue, since according to the errors in the log there can be not data collection. 


Yaniv

Comment 6 Itamar Heim 2013-08-13 12:36:55 UTC
there are some perf improvements in 3.2 (and actually many perf improvements coming in 3.3).

please upgrade engine *and* hosts to 3.2.2 before we do further analysis

Comment 8 Yaniv Lavi 2013-08-18 14:26:44 UTC
I'll close this issue. Reopen if needed.


Yaniv