Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1351166 - hawkular pod in metrics deployment OOMs but POD stays in status 'running'
hawkular pod in metrics deployment OOMs but POD stays in status 'running'
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular (Show other bugs)
3.2.0
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Matt Wringe
chunchen
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-06-29 07:48 EDT by Miheer Salunke
Modified: 2016-10-06 03:00 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-09-27 05:38:59 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1933 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.3 Release Advisory 2016-09-27 09:24:36 EDT

  None (edit)
Comment 7 Matt Wringe 2016-08-05 09:47:50 EDT
This should be fixed for 3.3. Setting ON_QA
Comment 8 chunchen 2016-08-08 03:58:41 EDT
It's fixed, checked with latest logging 3.3 images. I simulate this situation via stopping the java process [1] in the hawkular container, after that, though the hawkular pod status is still *Running*, but the container in the hawkular pod is marked as not *Ready*, please refer to the below results:

The test results:
1. Check the hawkular pod and container status before stopping java process
[chunchen@F17-CCY daily]$ oc get pod
NAME READY STATUS RESTARTS AGE
hawkular-cassandra-1-iq1e5 1/1 Running 0 11m
hawkular-metrics-4e62i 1/1 Running 0 11m
heapster-36bvw 1/1 Running 3 11m
metrics-deployer-1b1lb 0/1 Completed 0 13m

2. Stop the java process [1] in hawkular container
sh-4.2$ kill -9 JAVA-PROCESS-PID

3. Check the hawkular pod and container status again
[chunchen@F17-CCY daily]$ oc get pod
NAME                         READY     STATUS      RESTARTS   AGE
hawkular-cassandra-1-iq1e5   1/1       Running     0          25m
hawkular-metrics-4e62i       0/1       Running     2          25m
heapster-36bvw               1/1       Running     3          25m
metrics-deployer-1b1lb       0/1       Completed   0          27m


[1] java process:
/usr/lib/jvm/java-1.8.0/bin/java -D[Standalone] -server -verbose:gc -Xloggc:/opt/eap/standalone/log/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,org.jboss.byteman -Djava.awt.headless=true -Xbootclasspath/p:/opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.0.3.Final-redhat-1.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/ext/main/javax.json-1.0.4.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/ext/main/jboss-logmanager-ext-1.0.0.Alpha2-redhat-1.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -javaagent:/opt/eap/jolokia.jar=port=8778,protocol=https,caCert=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt,clientPrincipal=cn=system:master-proxy,useSslClientAuthentication=true,extraClientCheck=true,host=0.0.0.0,discoveryEnabled=false -Djava.security.egd=file:/dev/./urandom -Dorg.jboss.boot.log.file=/opt/eap/standalone/log/server.log -Dlogging.configuration=file:/opt/eap/standalone/configuration/logging.properties -jar /opt/eap/jboss-modules.jar -mp /opt/eap/modules org.jboss.as.standalone -Djboss.home.dir=/opt/eap -Djboss.server.base.dir=/opt/eap/standalone -Djavax.net.ssl.keyStore=/opt/hawkular/auth/hawkular-metrics.keystore -Djavax.net.ssl.keyStorePassword=5-Vr9QReUImUocS -Djavax.net.ssl.trustStore=/opt/hawkular/auth/hawkular-metrics.truststore -Djavax.net.ssl.trustStorePassword=5Vj4Q9h1580X6HF -b 0.0.0.0 -Dhawkular-metrics.cassandra-nodes=hawkular-cassandra -Dhawkular-metrics.cassandra-use-ssl -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true -Dhawkular-metrics.openshift.auth-methods=openshift-oauth,htpasswd -Dhawkular-metrics.openshift.htpasswd-file=/secrets/hawkular-metrics.htpasswd.file -Dhawkular.metrics.allowed-cors-access-control-allow-headers=authorization -Dhawkular.metrics.default-ttl=7 -DKUBERNETES_MASTER_URL=https://openshift-123.lab.sjc.redhat.com:8443 -DUSER_WRITE_ACCESS=true
Comment 10 errata-xmlrpc 2016-09-27 05:38:59 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933

Note You need to log in before you can comment on or make changes to this bug.