Bug 1613095
| Summary: | [3.9]hawkular-metrics pod failed to start up due to unsuccessful version check | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Junqi Zhao <juzhao> | ||||
| Component: | Hawkular | Assignee: | Ruben Vargas Palma <rvargasp> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Junqi Zhao <juzhao> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | high | ||||||
| Version: | 3.9.0 | CC: | ahaile, anli, aos-bugs, ggore, jsanda, ricardo.arguello, rvargasp, vlaad, wsun | ||||
| Target Milestone: | --- | Keywords: | Regression, TestBlocker | ||||
| Target Release: | 3.9.z | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | 1612648 | Environment: | |||||
| Last Closed: | 2018-12-13 19:27:05 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1612813, 1619497 | ||||||
| Bug Blocks: | 1599241, 1610733 | ||||||
| Attachments: |
|
||||||
|
Comment 1
Junqi Zhao
2018-08-07 01:11:52 UTC
Tested with, metrics-cassandra-v3.9.40-10 metrics-hawkular-metrics-v3.9.40-10 metrics-heapster-v3.9.40-10 it is still "Version check unsuccessful after 30 attempts" # oc get po -n openshift-infra NAME READY STATUS RESTARTS AGE hawkular-cassandra-1-sxc22 1/1 Running 0 13m hawkular-metrics-5cj5q 0/1 Running 4 12m heapster-nwmsb 0/1 Running 1 12m logs in hawkular-metrics pods ****************************************************************** 2018-08-07 05:49:15,204 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:49:15,205 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:49:25,214 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:49:25,215 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:49:35,230 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:49:35,231 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:49:45,239 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:49:45,240 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:49:55,270 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:49:55,271 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:50:05,280 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Version check failed: Keyspace hawkular_metrics does not exist 2018-08-07 05:50:05,280 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-07 05:50:15,282 FATAL [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The schema version check failed. Start up cannot proceed.: org.hawkular.metrics.api.jaxrs.util.SchemaVersionCheckException: Version check unsuccessful after 30 attempts at org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker.waitForSchemaUpdates(SchemaVersionChecker.java:73) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.doSchemaVersionCheck(MetricsServiceLifecycle.java:531) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.startMetricsService(MetricsServiceLifecycle.java:368) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-08-07 05:50:41,434 INFO [org.jboss.as.server] (Thread-8) WFLYSRV0220: Server shutdown has been requested via an OS signal *** JBossAS process (386) received TERM signal *** 2018-08-07 05:50:41,664 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 62) WFLYUT0022: Unregistered web context: '/hawkular/metrics' from server 'default-server' 2018-08-07 05:50:41,858 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0019: Host default-host stopping 2018-08-07 05:50:41,863 ERROR [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (MSC service thread 1-2) HAWKMETRICS200009: Unexcepted exception while shutting down: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at com.google.common.util.concurrent.Futures.wrapAndThrowExceptionOrError(Futures.java:1464) at com.google.common.util.concurrent.Futures.get(Futures.java:1453) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.destroy(MetricsServiceLifecycle.java:670) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:97) at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.preDestroy(DefaultLifecycleCallbackInvoker.java:90) at org.jboss.weld.injection.producer.BasicInjectionTarget.preDestroy(BasicInjectionTarget.java:127) at org.jboss.weld.bean.ManagedBean.destroy(ManagedBean.java:191) at org.jboss.weld.util.bean.IsolatedForwardingBean.destroy(IsolatedForwardingBean.java:50) at org.jboss.weld.context.AbstractContext.destroyContextualInstance(AbstractContext.java:139) at org.jboss.weld.context.AbstractContext.destroy(AbstractContext.java:153) at org.jboss.weld.context.AbstractSharedContext.destroy(AbstractSharedContext.java:61) at org.jboss.weld.context.AbstractSharedContext.invalidate(AbstractSharedContext.java:56) at org.jboss.weld.bootstrap.WeldRuntime.shutdown(WeldRuntime.java:56) at org.jboss.weld.bootstrap.WeldBootstrap.shutdown(WeldBootstrap.java:113) at org.jboss.as.weld.WeldStartService.stop(WeldStartService.java:128) at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2150) at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2101) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.stopServices(MetricsServiceLifecycle.java:682) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ... 3 more 2018-08-07 05:50:41,977 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 65) WFLYCLINF0003: Stopped client-mappings cache from ejb container 2018-08-07 05:50:42,072 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0028: Stopped deployment activemq-rar.rar (runtime-name: activemq-rar.rar) in 573ms 2018-08-07 05:50:42,101 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS] 2018-08-07 05:50:42,147 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-7) WFLYJCA0019: Stopped Driver service with driver-name = h2 2018-08-07 05:50:42,203 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0008: Undertow HTTPS listener https suspending 2018-08-07 05:50:42,208 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTP listener default suspending 2018-08-07 05:50:42,210 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080 2018-08-07 05:50:42,210 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443 2018-08-07 05:50:42,222 INFO [org.wildfly.extension.undertow] (MSC service thread 1-6) WFLYUT0004: Undertow 1.4.18.SP5-redhat-1 stopping 2018-08-07 05:50:42,402 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0028: Stopped deployment hawkular-metrics.war (runtime-name: hawkular-metrics.war) in 923ms 2018-08-07 05:50:42,420 INFO [org.jboss.as] (MSC service thread 1-5) WFLYSRV0050: JBoss EAP 7.1.2.GA (WildFly Core 3.0.13.Final-redhat-1) stopped in 939ms *** JBossAS process (386) received TERM signal *** ****************************************************************** Keyspace hawkular_metrics does not exist # oc exec hawkular-cassandra-1-sxc22 -n openshift-infra -- cqlsh --ssl -e "select table_name from system_schema.tables where keyspace_name = 'hawkular_metrics'" table_name ------------ (0 rows) tested with metrics-cassandra-v3.9.40-11 metrics-hawkular-metrics-v3.9.40-11 metrics-heapster-v3.9.40-11 metrics-hawkular-openshift-agent-v3.9.40-11 Issue is fixed, and metrics works well New regression issue: Bug 1613656, it is not test blocker add TestBlocker back,issue is reproduced with metrics-hawkular-metrics-v3.9.41-1 metrics-cassandra-v3.9.41-1 metrics-heapster-v3.9.41-1 # oc logs -f hawkular-metrics-w956g 2018-08-22 08:20:35,859 INFO [org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker] (metricsservice-lifecycle-thread) Trying again in 10000 ms 2018-08-22 08:20:45,860 FATAL [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The schema version check failed. Start up cannot proceed.: org.hawkular.metrics.api.jaxrs.util.SchemaVersionCheckException: Version check unsuccessful after 30 attempts at org.hawkular.metrics.api.jaxrs.util.SchemaVersionChecker.waitForSchemaUpdates(SchemaVersionChecker.java:73) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.doSchemaVersionCheck(MetricsServiceLifecycle.java:531) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.startMetricsService(MetricsServiceLifecycle.java:368) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) *** JBossAS process (375) received TERM signal *** 2018-08-22 08:21:12,623 INFO [org.jboss.as.server] (Thread-8) WFLYSRV0220: Server shutdown has been requested via an OS signal 2018-08-22 08:21:12,684 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 66) WFLYUT0022: Unregistered web context: '/hawkular/metrics' from server 'default-server' 2018-08-22 08:21:12,737 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0019: Host default-host stopping 2018-08-22 08:21:12,743 ERROR [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (MSC service thread 1-8) HAWKMETRICS200009: Unexcepted exception while shutting down: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException at com.google.common.util.concurrent.Futures.wrapAndThrowExceptionOrError(Futures.java:1464) at com.google.common.util.concurrent.Futures.get(Futures.java:1453) at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.destroy(MetricsServiceLifecycle.java:670) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:97) at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.preDestroy(DefaultLifecycleCallbackInvoker.java:90) at org.jboss.weld.injection.producer.BasicInjectionTarget.preDestroy(BasicInjectionTarget.java:127) at org.jboss.weld.bean.ManagedBean.destroy(ManagedBean.java:191) at org.jboss.weld.util.bean.IsolatedForwardingBean.destroy(IsolatedForwardingBean.java:50) at org.jboss.weld.context.AbstractContext.destroyContextualInstance(AbstractContext.java:139) at org.jboss.weld.context.AbstractContext.destroy(AbstractContext.java:153) at org.jboss.weld.context.AbstractSharedContext.destroy(AbstractSharedContext.java:61) at org.jboss.weld.context.AbstractSharedContext.invalidate(AbstractSharedContext.java:56) at org.jboss.weld.bootstrap.WeldRuntime.shutdown(WeldRuntime.java:56) at org.jboss.weld.bootstrap.WeldBootstrap.shutdown(WeldBootstrap.java:113) at org.jboss.as.weld.WeldStartService.stop(WeldStartService.java:128) at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2150) at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2101) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.NullPointerException at org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle.stopServices(MetricsServiceLifecycle.java:682) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ... 3 more 2018-08-22 08:21:12,762 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0003: Stopped client-mappings cache from ejb container 2018-08-22 08:21:12,831 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-7) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS] 2018-08-22 08:21:12,836 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-6) WFLYJCA0019: Stopped Driver service with driver-name = h2 2018-08-22 08:21:12,844 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0028: Stopped deployment activemq-rar.rar (runtime-name: activemq-rar.rar) in 204ms 2018-08-22 08:21:12,861 INFO [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0008: Undertow HTTPS listener https suspending 2018-08-22 08:21:12,861 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0008: Undertow HTTP listener default suspending 2018-08-22 08:21:12,862 INFO [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 0.0.0.0:8443 2018-08-22 08:21:12,871 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080 2018-08-22 08:21:12,871 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0004: Undertow 1.4.18.SP5-redhat-1 stopping 2018-08-22 08:21:12,908 INFO [org.jboss.as.server.deployment] (MSC service thread 1-6) WFLYSRV0028: Stopped deployment hawkular-metrics.war (runtime-name: hawkular-metrics.war) in 271ms 2018-08-22 08:21:12,917 INFO [org.jboss.as] (MSC service thread 1-8) WFLYSRV0050: JBoss EAP 7.1.2.GA (WildFly Core 3.0.13.Final-redhat-1) stopped in 279ms *** JBossAS process (375) received TERM signal *** Issue is not fixed with images: metrics-cassandra:v3.9.43-2 metrics-hawkular-metrics:v3.9.43-2 metrics-heapster:v3.9.43-2 The latest 3.9 image is 3.9.44 and the problem still exist because that image uses Hawkular Metrics 0.29.2. The schema installer was introduced in OCP 3.10. We wanted to back port it to 3.9. The changes were introduced upstream in Hawkular Metrics 0.29.2. Because og the ongoing problems we have had with trying to back port, we are going to update the 3.9 image to use Hawkular Metrics 0.29.1. HAWKULAR_METRICS_VERSION=0.29.2.Final-redhat-1 in metrics-hawkular-metrics-v3.9.43-3 Tested with metrics-hawkular-metrics-v3.9.43-4, hawkular-metrics pods failed to start up, there is error in pod logs.
other images
metrics-cassandra-v3.9.43-3
metrics-heapster-v3.9.43-3
# oc get pod -n openshift-infra
NAME READY STATUS RESTARTS AGE
hawkular-cassandra-1-qchg4 1/1 Running 0 17m
hawkular-metrics-rs4gr 0/1 Running 1 17m
heapster-g2n62 0/1 Running 2 17m
2018-09-19 05:12:29,290 INFO [org.jboss.weld.Version] (MSC service thread 1-7) WELD-000900: 2.4.7 (redhat)
2018-09-19 05:12:29,587 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "hawkular-metrics.war")]) - failure description: {
"WFLYCTL0412: Required services that are not installed:" => [
"jboss.naming.context.java.jboss.infinispan.container.hawkular-metrics",
"jboss.naming.context.java.jboss.infinispan.cache.hawkular-metrics.locks"
],
"WFLYCTL0180: Services with missing/unavailable dependencies" => [
"jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.cache.locks is missing [jboss.naming.context.java.jboss.infinispan.cache.hawkular-metrics.locks]",
"jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.container.hawkular-metrics is missing [jboss.naming.context.java.jboss.infinispan.container.hawkular-metrics]"
]
}
2018-09-19 05:12:29,611 INFO [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "hawkular-metrics.war" (runtime-name : "hawkular-metrics.war")
2018-09-19 05:12:29,612 INFO [org.jboss.as.server] (ServerService Thread Pool -- 34) WFLYSRV0010: Deployed "activemq-rar.rar" (runtime-name : "activemq-rar.rar")
2018-09-19 05:12:29,616 INFO [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0183: Service status report
WFLYCTL0184: New missing/unsatisfied dependencies:
service jboss.naming.context.java.jboss.infinispan.cache.hawkular-metrics.locks (missing) dependents: [service jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.cache.locks]
service jboss.naming.context.java.jboss.infinispan.container.hawkular-metrics (missing) dependents: [service jboss.naming.context.java.module.hawkular-metrics.hawkular-metrics.env.container.hawkular-metrics]
2018-09-19 05:12:29,763 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
2018-09-19 05:12:29,766 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
2018-09-19 05:12:29,766 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
2018-09-19 05:12:29,766 ERROR [org.jboss.as] (Controller Boot Thread) WFLYSRV0026: JBoss EAP 7.1.2.GA (WildFly Core 3.0.13.Final-redhat-1) started (with errors) in 10478ms - Started 422 of 689 services (22 services failed or missing dependencies, 372 services are lazy, passive or on-demand)
Created attachment 1484573 [details]
metrics logs - metrics-hawkular-metrics:v3.9.43-4
Ruben, I bet I know what the problem is. I think we left out another, necessary change in addition to reverting back to hawkular-metrics 0.29.1. We need to add back the infinispan configurtation in standalone.xml in the image. fixed with metrics-hawkular-metrics:v3.9.43-5 other images metrics-cassandra-v3.9.43-3 metrics-heapster-v3.9.43-3 # oc get pod -n openshift-infra NAME READY STATUS RESTARTS AGE hawkular-cassandra-1-4k9hl 1/1 Running 0 32m hawkular-metrics-55mrs 1/1 Running 0 32m heapster-6zvzt 1/1 Running 0 32m please change to ON_QA, so we can close it. Per Comment 15, set it to VERIFIED. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3748 |