Bug 1611941 - [3.6]hawkular-metrics pod failed to start up due to unsuccessful version check
Summary: [3.6]hawkular-metrics pod failed to start up due to unsuccessful version check
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Hawkular
Version: 3.6.1
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.6.z
Assignee: Ruben Vargas Palma
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks: 1612648 1612813 1619497
TreeView+ depends on / blocked
 
Reported: 2018-08-03 06:29 UTC by Junqi Zhao
Modified: 2019-01-30 15:12 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1612648 1612813 1619497 (view as bug list)
Environment:
Last Closed: 2019-01-30 15:12:36 UTC
Target Upstream Version:


Attachments (Terms of Use)
metrics pods log (19.47 KB, application/x-gzip)
2018-08-03 06:29 UTC, Junqi Zhao
no flags Details
metrics logs - metrics-hawkular-metrics:v3.6.173.0.130-2 (34.70 KB, text/plain)
2018-09-20 04:47 UTC, Junqi Zhao
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0101 0 None None None 2019-01-30 15:12:39 UTC

Description Junqi Zhao 2018-08-03 06:29:24 UTC
Created attachment 1472883 [details]
metrics pods log

Description of problem:
Deploy metrics 3.6,hawkular-metrics pod failed to start up due to unsuccessful version check, this blocks metrics installation.

# oc get po
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-hvhr0   1/1       Running   0          14m
hawkular-metrics-2r82k       0/1       Running   4          14m
heapster-8996q               0/1       Running   1          13m

logs in hawkular-metrics pod 
2018-08-03 05:13:24,048 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-03 05:13:24,048 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-03 05:13:34,052 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-03 05:13:34,052 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-03 05:13:44,053 FATAL [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The schema version check failed. Start up cannot proceed.: org.hawkular.metrics.api.jaxrs.util.SchemaVersionCheckException: Version check unsuccessful after 30 attempts
	at org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker.waitForSchemaUpdates(SchemaVersionChecker.java:72)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.doSchemaVersionCheck(MetricsServiceLifecycle.java:522)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.startMetricsService(MetricsServiceLifecycle.java:362)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Although "Keyspace hawkular_metrics does not exist" shows, actually it exsits
# oc rsh hawkular-cassandra-1-hvhr0
sh-4.2$ cqlsh --ssl -e "select table_name from system_schema.tables where keyspace_name = 'hawkular_metrics'"

 table_name
--------------------
 active_time_slices
           cassalog
               data
    data_compressed
  finished_jobs_idx
               jobs
             leases
              locks
        metrics_idx
   metrics_tags_idx
     retentions_idx
 scheduled_jobs_idx
         sys_config
              tasks
            tenants

(15 rows)

Version-Release number of selected component (if applicable):
metrics-cassandra-v3.6.173.0.128-2
metrics-hawkular-metrics-v3.6.173.0.128-3
metrics-heapster-v3.6.173.0.128-2

# openshift version
openshift v3.6.173.0.128
kubernetes v1.6.1+5115d708d7
etcd 3.2.1


How reproducible:
Always

Steps to Reproduce:
1. Deploy Metrics 3.6, parameters see the [Additional info] part
2.
3.

Actual results:
hawkular-metrics pod failed to start up due to unsuccessful version check 

Expected results:


Additional info:
openshift_metrics_install_metrics=true
openshift_metrics_image_prefix=registry.reg-aws.openshift.com:443/openshift3/
openshift_metrics_image_version=v3.6.173.0.128
openshift_metrics_cassandra_storage_type=dynamic

Comment 1 John Sanda 2018-08-03 13:26:12 UTC
https://github.com/openshift/openshift-ansible/pull/8961 will be backported to 3.6 as well. That is holding things at the moment.

Comment 3 Junqi Zhao 2018-08-06 03:20:55 UTC
Changed to ON_QA by errata, change back to MODIFIED again

Comment 4 Junqi Zhao 2018-08-06 03:52:28 UTC
(In reply to John Sanda from comment #1)
> https://github.com/openshift/openshift-ansible/pull/8961 will be backported
> to 3.6 as well. That is holding things at the moment.

I don't think we should backport hawkular_schema_job to 3.6 or 3.7, it will bring us efforts to test metrics, since it is one regression bug, we should not change too much.

Comment 5 Junqi Zhao 2018-08-06 04:43:43 UTC
Tested again with the same images,Keyspace hawkular_metrics does not exist actually.
metrics-cassandra-v3.6.173.0.128-2
metrics-hawkular-metrics-v3.6.173.0.128-3
metrics-heapster-v3.6.173.0.128-2

# oc get pod
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-bf8wp   1/1       Running   0          6m
hawkular-metrics-kk24s       0/1       Running   0          6m
heapster-nkp8h               0/1       Running   0          5m
# oc rsh hawkular-cassandra-1-bf8wp
sh-4.2$ cqlsh --ssl -e "select table_name from system_schema.tables where keyspace_name = 'hawkular_metrics'"

 table_name
------------

(0 rows)

Comment 6 John Sanda 2018-08-06 17:57:40 UTC
(In reply to Junqi Zhao from comment #4)
> (In reply to John Sanda from comment #1)
> > https://github.com/openshift/openshift-ansible/pull/8961 will be backported
> > to 3.6 as well. That is holding things at the moment.
> 
> I don't think we should backport hawkular_schema_job to 3.6 or 3.7, it will
> bring us efforts to test metrics, since it is one regression bug, we should
> not change too much.

We decided to backport to address bug 1564681, which is linked to a customer case. At the time of our decision, the customer was on 3.6 and has subsequently upgraded to 3.7.

These changes also make hawkular-metrics more stable and should consequently help reduce maintenance. Prior to the schema installer, starting multiple replicas of hawkular-metrics can result in inconsistent state that has sometimes required manual intervention to fix.

Comment 8 Junqi Zhao 2018-08-07 05:52:10 UTC
Tested with, 
metrics-cassandra-v3.6.173.0.128-10
metrics-hawkular-metrics-v3.6.173.0.128-10
metrics-heapster-v3.6.173.0.128-10


it is still "Version check unsuccessful after 30 attempts"
# oc get pod -n openshift-infra
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-gkp68   1/1       Running   0          16m
hawkular-metrics-438hl       0/1       Running   2          16m
heapster-3fzkd               0/1       Running   1          16m


logs in hawkular-metrics pods
******************************************************************
2018-08-07 05:45:40,958 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-07 05:45:40,958 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-07 05:45:50,965 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-07 05:45:50,966 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-07 05:46:00,970 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-07 05:46:00,970 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-07 05:46:10,975 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-07 05:46:10,975 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-07 05:46:20,976 FATAL [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The schema version check failed. Start up cannot proceed.: org.hawkular.metrics.api.jaxrs.util.SchemaVersionCheckException: Version check unsuccessful after 30 attempts
	at org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker.waitForSchemaUpdates(SchemaVersionChecker.java:72)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.doSchemaVersionCheck(MetricsServiceLifecycle.java:522)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.startMetricsService(MetricsServiceLifecycle.java:362)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
*** JBossAS process (462) received TERM signal ***
2018-08-07 05:46:49,525 INFO  [org.jboss.as.server] (Thread-3) WFLYSRV0220: Server shutdown has been requested via an OS signal
2018-08-07 05:46:49,650 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 64) WFLYUT0022: Unregistered web context: /hawkular/metrics
2018-08-07 05:46:49,695 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0019: Host default-host stopping
2018-08-07 05:46:49,705 INFO  [org.jboss.weld.deployer] (MSC service thread 1-6) WFLYWELD0010: Stopping weld service for deployment hawkular-metrics.war
2018-08-07 05:46:49,762 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-6) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
2018-08-07 05:46:49,790 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-8) WFLYSRV0028: Stopped deployment activemq-rar.rar (runtime-name: activemq-rar.rar) in 251ms
2018-08-07 05:46:49,815 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) WFLYJCA0019: Stopped Driver service with driver-name = h2
2018-08-07 05:46:49,826 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0008: Undertow HTTPS listener https suspending
2018-08-07 05:46:49,830 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTP listener default suspending
2018-08-07 05:46:49,831 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080
2018-08-07 05:46:49,836 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443
2018-08-07 05:46:49,837 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0004: Undertow 1.3.31.Final-redhat-1 stopping
2018-08-07 05:46:49,904 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment hawkular-metrics.war (runtime-name: hawkular-metrics.war) in 364ms
2018-08-07 05:46:49,911 INFO  [org.jboss.as] (MSC service thread 1-7) WFLYSRV0050: JBoss EAP 7.0.8.GA (WildFly Core 2.1.18.Final-redhat-1) stopped in 368ms
*** JBossAS process (462) received TERM signal ***
******************************************************************

Keyspace hawkular_metrics does not exist
# oc exec hawkular-cassandra-1-gkp68 -n openshift-infra -- cqlsh --ssl -e "select table_name from system_schema.tables where keyspace_name = 'hawkular_metrics'"

 table_name
------------

(0 rows)

Comment 9 Junqi Zhao 2018-08-08 05:58:35 UTC
tested with
metrics-cassandra-v3.6.173.0.128-11
metrics-hawkular-metrics-v3.6.173.0.128-11
metrics-heapster-v3.6.173.0.128-11

Issue is fixed, and metrics works well
Please change to ON_QA

Comment 11 Junqi Zhao 2018-08-22 04:19:23 UTC
Issue is not fixed with images:
metrics-cassandra-v3.6.173.0.129-2
metrics-hawkular-metrics-v3.6.173.0.129-2
metrics-heapster-v3.6.173.0.129-2

# oc get pod
NAME                         READY     STATUS    RESTARTS   AGE
hawkular-cassandra-1-b7mrf   1/1       Running   0          44m
hawkular-metrics-c8kd4       0/1       Running   7          44m
heapster-k38ch               0/1       Running   5          44m

# oc logs hawkular-metrics-c8kd4
2018-08-22 03:29:08,034 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-22 03:29:18,037 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-22 03:29:18,037 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-22 03:29:28,040 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist
2018-08-22 03:29:28,041 INFO  [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms
2018-08-22 03:29:38,041 FATAL [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The schema version check failed. Start up cannot proceed.: org.hawkular.metrics.api.jaxrs.util.SchemaVersionCheckException: Version check unsuccessful after 30 attempts
	at org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker.waitForSchemaUpdates(SchemaVersionChecker.java:72)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.doSchemaVersionCheck(MetricsServiceLifecycle.java:522)
	at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.startMetricsService(MetricsServiceLifecycle.java:362)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2018-08-22 03:30:02,443 INFO  [org.jboss.as.server] (Thread-5) WFLYSRV0220: Server shutdown has been requested via an OS signal
*** JBossAS process (463) received TERM signal ***
2018-08-22 03:30:02,487 INFO  [org.wildfly.extension.undertow] (ServerService Thread Pool -- 66) WFLYUT0022: Unregistered web context: /hawkular/metrics
2018-08-22 03:30:02,531 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0019: Host default-host stopping
2018-08-22 03:30:02,546 INFO  [org.jboss.weld.deployer] (MSC service thread 1-4) WFLYWELD0010: Stopping weld service for deployment hawkular-metrics.war
2018-08-22 03:30:02,570 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
2018-08-22 03:30:02,583 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-3) WFLYJCA0019: Stopped Driver service with driver-name = h2
2018-08-22 03:30:02,603 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-6) WFLYSRV0028: Stopped deployment activemq-rar.rar (runtime-name: activemq-rar.rar) in 151ms
2018-08-22 03:30:02,604 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0008: Undertow HTTP listener default suspending
2018-08-22 03:30:02,605 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow HTTPS listener https suspending
2018-08-22 03:30:02,605 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080
2018-08-22 03:30:02,605 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443
2018-08-22 03:30:02,606 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0004: Undertow 1.3.31.Final-redhat-1 stopping
2018-08-22 03:30:02,642 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-5) WFLYSRV0028: Stopped deployment hawkular-metrics.war (runtime-name: hawkular-metrics.war) in 191ms
2018-08-22 03:30:02,645 INFO  [org.jboss.as] (MSC service thread 1-1) WFLYSRV0050: JBoss EAP 7.0.8.GA (WildFly Core 2.1.18.Final-redhat-1) stopped in 194ms
*** JBossAS process (463) received TERM signal ***

Comment 13 Junqi Zhao 2018-09-14 05:55:43 UTC
Issue is not fixed with images:
metrics-cassandra:v3.6.173.0.129-2
metrics-hawkular-metrics:v3.6.173.0.129-2
metrics-heapster:v3.6.173.0.129-2

Comment 14 John Sanda 2018-09-17 19:40:33 UTC
The latest 3.6 image is v3.6.173.0.129 and the problem still exist because that image uses Hawkular Metrics 0.27.8.

The schema installer was introduced in OCP 3.10. We wanted to back port it to 3.6. The changes were introduced upstream in Hawkular Metrics 0.27.8. Because of the ongoing problems we have had with trying to back port, we are going to update the 3.6 image to use Hawkular Metrics 0.27.7.

Comment 15 Junqi Zhao 2018-09-20 04:43:13 UTC
Tested metrics-hawkular-metrics-v3.6.173.0.130-2 with
metrics-cassandra-v3.6.173.0.130-1
metrics-heapster-v3.6.173.0.130-1

hawkular-metrics pods failed to start up, error in pod logs in under below
We need to add back the infinispan configurtation in standalone.xml in the image  as did for 3.7 and 3.9 images
**********************************************************************
2018-09-20 04:26:49,352 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "hawkular-metrics.war")]) - failure description: {"WFLYCTL0180: Services with missing/unavailable dependencies" => [
    "jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.cache.locks is missing [jboss.naming.context.java.jboss.infinispan.cache.hawkular-metrics.locks]",
    "jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.container.hawkular-metrics is missing [jboss.naming.context.java.jboss.infinispan.container.hawkular-metrics]"
]}
2018-09-20 04:26:49,368 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "hawkular-metrics.war" (runtime-name : "hawkular-metrics.war")
2018-09-20 04:26:49,369 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "activemq-rar.rar" (runtime-name : "activemq-rar.rar")
2018-09-20 04:26:49,373 INFO  [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0183: Service status report
WFLYCTL0184:    New missing/unsatisfied dependencies:
      service jboss.naming.context.java.jboss.infinispan.cache.hawkular-metrics.locks (missing) dependents: [service jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.cache.locks] 
      service jboss.naming.context.java.jboss.infinispan.container.hawkular-metrics (missing) dependents: [service jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.container.hawkular-metrics] 
2018-09-20 04:26:49,453 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
2018-09-20 04:26:49,453 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
2018-09-20 04:26:49,453 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026: JBoss EAP 7.0.8.GA (WildFly Core 2.1.18.Final-redhat-1) started (with errors) in 5114ms - Started 387 of 706 services (22 services failed or missing dependencies, 394 services are lazy, passive or on-demand)

Comment 16 Junqi Zhao 2018-09-20 04:47:02 UTC
Created attachment 1485008 [details]
metrics logs - metrics-hawkular-metrics:v3.6.173.0.130-2

Comment 17 Junqi Zhao 2018-09-21 03:28:42 UTC
Issue is fixed with metrics-hawkular-metrics:v3.6.173.0.130-3

other images
metrics-cassandra-v3.6.173.0.130-1
metrics-heapster-v3.6.173.0.130-1

new Bug 1631598 is filed, but not a test blocker

Comment 18 Junqi Zhao 2018-09-21 11:26:30 UTC
please change to ON_QA, so we can close it.

Comment 19 Junqi Zhao 2018-10-17 01:04:43 UTC
Per Comment 17, move to VERIFIED.

Comment 21 errata-xmlrpc 2019-01-30 15:12:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0101


Note You need to log in before you can comment on or make changes to this bug.