Login
Log in using an SSO provider:
Fedora Account System
Red Hat Associate
Red Hat Customer
Login using a Red Hat Bugzilla account
Forgot Password
Create an Account
Red Hat Bugzilla – Attachment 1340164 Details for
Bug 1500471
3.6.1 White spaces in the cert prevents Origin Metrics from starting
Home
New
Search
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh92 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
[?]
This site requires JavaScript to be enabled to function correctly, please enable it.
hawkular-metrics 3.6 pod log
hawkular-metrics.txt (text/plain), 233.13 KB, created by
Junqi Zhao
on 2017-10-18 12:41:22 UTC
(
hide
)
Description:
hawkular-metrics 3.6 pod log
Filename:
MIME Type:
Creator:
Junqi Zhao
Created:
2017-10-18 12:41:22 UTC
Size:
233.13 KB
patch
obsolete
>2017-10-18 12:16:08 Starting Hawkular Metrics >The service account has read permissions for its project. Proceeding >The service account has permission to watch namespaces. Proceeding >Creating the Hawkular Metrics keystore from the Secret's cert data >Converting the PKCS12 keystore into a Java Keystore >Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss >Entry for alias hawkular-metrics successfully imported. >Import command completed: 1 entries successfully imported, 0 entries failed or cancelled >[Storing /opt/hawkular/auth/hawkular-metrics.keystore] >Building the trust store >Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss >Certificate was added to keystore >Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss >Certificate was added to keystore >Splitting up the Kubernetes CA into individual certificates >Adding the Kubernetes CAs into the trust store >Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss >Certificate was added to keystore >-Xms1192m -Xmx1192m -XX:+UseParallelGC -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=100m -XX:+ExitOnOutOfMemoryError >========================================================================= > JBoss Bootstrap Environment > JBOSS_HOME: /opt/eap > JAVA: /usr/lib/jvm/java-1.8.0/bin/java > JAVA_OPTS: -server -verbose:gc -Xloggc:"/opt/eap/standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1192m -Xmx1192m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api -Djava.awt.headless=true -javaagent:/opt/jolokia/jolokia.jar=config=/opt/jolokia/etc/jolokia.properties -Xbootclasspath/p:/opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/.overlays/layer-base-jboss-eap-7.0.8.CP/org/jboss/logmanager/main/jboss-logmanager-2.0.7.Final-redhat-1.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/ext/main/jboss-logmanager-ext-1.0.0.Alpha2-redhat-1.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -XX:+UseParallelGC -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump >========================================================================= >Picked up JAVA_TOOL_OPTIONS: -Duser.home=/home/jboss -Duser.name=jboss >[0m2017-10-18 12:16:16,229 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.4.Final-redhat-1 >[0mCould not start Jolokia agent: java.lang.IllegalStateException: Cannot use keystore for https communication: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- >[0m2017-10-18 12:16:16,820 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.7.SP1-redhat-1 >[0m[0m2017-10-18 12:16:16,960 INFO [org.jboss.as] (MSC service thread 1-6) WFLYSRV0049: JBoss EAP 7.0.8.GA (WildFly Core 2.1.18.Final-redhat-1) starting >[0m[32m2017-10-18 12:16:16,961 DEBUG [org.jboss.as.config] (MSC service thread 1-6) Configured system properties: > KUBERNETES_MASTER_URL = https://kubernetes.default.svc > USER_WRITE_ACCESS = False > [Standalone] = > awt.toolkit = sun.awt.X11.XToolkit > com.datastax.driver.FORCE_NIO = true > file.encoding = ANSI_X3.4-1968 > file.encoding.pkg = sun.io > file.separator = / > hawkular-alerts.cassandra-nodes = hawkular-cassandra > hawkular-alerts.cassandra-use-ssl = true > hawkular.alerts.allowed-cors-access-control-allow-headers = authorization > hawkular.alerts.openshift.auth-methods = openshift-oauth,htpasswd > hawkular.alerts.openshift.htpasswd-file = /hawkular-account/hawkular-metrics.htpasswd > hawkular.metrics.admin-tenant = _hawkular_admin > hawkular.metrics.allowed-cors-access-control-allow-headers = authorization > hawkular.metrics.cassandra.nodes = hawkular-cassandra > hawkular.metrics.cassandra.use-ssl = true > hawkular.metrics.default-ttl = 7 > hawkular.metrics.openshift.auth-methods = openshift-oauth,htpasswd > hawkular.metrics.openshift.htpasswd-file = /hawkular-account/hawkular-metrics.htpasswd > java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment > java.awt.headless = true > java.awt.printerjob = sun.print.PSPrinterJob > java.class.path = /opt/eap/jboss-modules.jar:/opt/jolokia/jolokia.jar > java.class.version = 52.0 > java.endorsed.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/endorsed > java.ext.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/ext:/usr/java/packages/lib/ext > java.home = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre > java.io.tmpdir = /tmp > java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib > java.net.preferIPv4Stack = true > java.runtime.name = OpenJDK Runtime Environment > java.runtime.version = 1.8.0_141-b16 > java.security.egd = file:/dev/./urandom > java.specification.name = Java Platform API Specification > java.specification.vendor = Oracle Corporation > java.specification.version = 1.8 > java.util.logging.manager = org.jboss.logmanager.LogManager > java.vendor = Oracle Corporation > java.vendor.url = http://java.oracle.com/ > java.vendor.url.bug = http://bugreport.sun.com/bugreport/ > java.version = 1.8.0_141 > java.vm.info = mixed mode > java.vm.name = OpenJDK 64-Bit Server VM > java.vm.specification.name = Java Virtual Machine Specification > java.vm.specification.vendor = Oracle Corporation > java.vm.specification.version = 1.8 > java.vm.vendor = Oracle Corporation > java.vm.version = 25.141-b16 > javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder > javax.net.ssl.keyStore = /opt/hawkular/auth/hawkular-metrics.keystore > javax.net.ssl.keyStorePassword = <redacted> > javax.net.ssl.trustStore = /opt/hawkular/auth/hawkular-metrics.truststore > javax.net.ssl.trustStorePassword = <redacted> > javax.xml.datatype.DatatypeFactory = __redirected.__DatatypeFactory > javax.xml.parsers.DocumentBuilderFactory = __redirected.__DocumentBuilderFactory > javax.xml.parsers.SAXParserFactory = __redirected.__SAXParserFactory > javax.xml.stream.XMLEventFactory = __redirected.__XMLEventFactory > javax.xml.stream.XMLInputFactory = __redirected.__XMLInputFactory > javax.xml.stream.XMLOutputFactory = __redirected.__XMLOutputFactory > javax.xml.transform.TransformerFactory = __redirected.__TransformerFactory > javax.xml.validation.SchemaFactory:http://www.w3.org/2001/XMLSchema = __redirected.__SchemaFactory > javax.xml.xpath.XPathFactory:http://java.sun.com/jaxp/xpath/dom = __redirected.__XPathFactory > jboss.bind.address = 0.0.0.0 > jboss.bind.address.private = 10.129.0.54 > jboss.home.dir = /opt/eap > jboss.host.name = hawkular-metrics-n15zd > jboss.modules.dir = /opt/eap/modules > jboss.modules.system.pkgs = org.jboss.logmanager,jdk.nashorn.api > jboss.node.name = hawkular-metrics-n15zd > jboss.qualified.host.name = hawkular-metrics-n15zd > jboss.server.base.dir = /opt/eap/standalone > jboss.server.config.dir = /opt/eap/standalone/configuration > jboss.server.data.dir = /opt/eap/standalone/data > jboss.server.deploy.dir = /opt/eap/standalone/data/content > jboss.server.log.dir = /opt/eap/standalone/log > jboss.server.name = hawkular-metrics-n15zd > jboss.server.persist.config = true > jboss.server.temp.dir = /opt/eap/standalone/tmp > jgroups.password = <redacted> > line.separator = > logging.configuration = file:/opt/eap/standalone/configuration/logging.properties > module.path = /opt/eap/modules > org.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH = true > org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH = true > org.jboss.boot.log.file = /opt/eap/standalone/log/server.log > org.jboss.resolver.warning = true > org.xml.sax.driver = __redirected.__XMLReaderFactory > os.arch = amd64 > os.name = Linux > os.version = 3.10.0-693.el7.x86_64 > path.separator = : > sun.arch.data.model = 64 > sun.boot.class.path = /opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/.overlays/layer-base-jboss-eap-7.0.8.CP/org/jboss/logmanager/main/jboss-logmanager-2.0.7.Final-redhat-1.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/ext/main/jboss-logmanager-ext-1.0.0.Alpha2-redhat-1.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/classes > sun.boot.library.path = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.141-2.b16.el7_4.x86_64/jre/lib/amd64 > sun.cpu.endian = little > sun.cpu.isalist = > sun.io.unicode.encoding = UnicodeLittle > sun.java.command = /opt/eap/jboss-modules.jar -mp /opt/eap/modules org.jboss.as.standalone -Djboss.home.dir=/opt/eap -Djboss.server.base.dir=/opt/eap/standalone -Djavax.net.ssl.keyStore=/opt/hawkular/auth/hawkular-metrics.keystore -Djavax.net.ssl.trustStore=/opt/hawkular/auth/hawkular-metrics.truststore -Djboss.node.name=hawkular-metrics-n15zd -b 10.129.0.54 -bprivate 10.129.0.54 -P /opt/hawkular/server.properties -b 0.0.0.0 -Dhawkular.metrics.cassandra.nodes=hawkular-cassandra -Dhawkular.metrics.cassandra.use-ssl -Dhawkular.metrics.openshift.auth-methods=openshift-oauth,htpasswd -Dhawkular.metrics.openshift.htpasswd-file=/hawkular-account/hawkular-metrics.htpasswd -Dhawkular.metrics.allowed-cors-access-control-allow-headers=authorization -Dhawkular.metrics.default-ttl=7 -Dhawkular.metrics.admin-tenant=_hawkular_admin -Dhawkular-alerts.cassandra-nodes=hawkular-cassandra -Dhawkular-alerts.cassandra-use-ssl -Dhawkular.alerts.openshift.auth-methods=openshift-oauth,htpasswd -Dhawkular.alerts.openshift.htpasswd-file=/hawkular-account/hawkular-metrics.htpasswd -Dhawkular.alerts.allowed-cors-access-control-allow-headers=authorization -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true -Dcom.datastax.driver.FORCE_NIO=true -DKUBERNETES_MASTER_URL=https://kubernetes.default.svc -DUSER_WRITE_ACCESS=False > sun.java.launcher = SUN_STANDARD > sun.jnu.encoding = ANSI_X3.4-1968 > sun.management.compiler = HotSpot 64-Bit Tiered Compilers > sun.os.patch.level = unknown > user.country = US > user.dir = /home/jboss > user.home = /home/jboss > user.language = en > user.name = jboss > user.timezone = UTC >[0m[32m2017-10-18 12:16:16,961 DEBUG [org.jboss.as.config] (MSC service thread 1-6) VM Arguments: -Duser.home=/home/jboss -Duser.name=jboss -D[Standalone] -verbose:gc -Xloggc:/opt/eap/standalone/log/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1192m -Xmx1192m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api -Djava.awt.headless=true -javaagent:/opt/jolokia/jolokia.jar=config=/opt/jolokia/etc/jolokia.properties -Xbootclasspath/p:/opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/.overlays/layer-base-jboss-eap-7.0.8.CP/org/jboss/logmanager/main/jboss-logmanager-2.0.7.Final-redhat-1.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/ext/main/jboss-logmanager-ext-1.0.0.Alpha2-redhat-1.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -XX:+UseParallelGC -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heapdump -Dorg.jboss.boot.log.file=/opt/eap/standalone/log/server.log -Dlogging.configuration=file:/opt/eap/standalone/configuration/logging.properties >[0m[0m2017-10-18 12:16:19,679 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 26) WFLYDR0001: Content added at location /opt/eap/standalone/data/content/93/bd76dce5eb352c86229fcd0c44da1db68fa456/content >[0m[0m2017-10-18 12:16:20,091 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 26) WFLYDR0001: Content added at location /opt/eap/standalone/data/content/77/b90484af86b80ac4015b1d8a5366d484bdf845/content >[0m[0m2017-10-18 12:16:20,111 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http) >[0m[0m2017-10-18 12:16:20,156 INFO [org.xnio] (MSC service thread 1-4) XNIO version 3.4.6.Final-redhat-1 >[0m[0m2017-10-18 12:16:20,190 INFO [org.xnio.nio] (MSC service thread 1-4) XNIO NIO Implementation Version 3.4.6.Final-redhat-1 >[0m[0m2017-10-18 12:16:20,314 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 40) WFLYIO001: Worker 'default' has auto-configured to 8 core threads with 64 task threads based on your 4 available processors >[0m[0m2017-10-18 12:16:20,382 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 41) WFLYCLINF0001: Activating Infinispan subsystem. >[0m[0m2017-10-18 12:16:20,395 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 48) WFLYJSF0007: Activated the following JSF Implementations: [main] >[0m[0m2017-10-18 12:16:20,417 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 45) WFLYCLJG0001: Activating JGroups subsystem. >[0m[0m2017-10-18 12:16:20,444 INFO [org.jboss.as.webservices] (ServerService Thread Pool -- 62) WFLYWS0002: Activating WebServices Extension >[0m[0m2017-10-18 12:16:20,488 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 51) WFLYNAM0001: Activating Naming Subsystem >[0m[0m2017-10-18 12:16:20,478 INFO [org.jboss.remoting] (MSC service thread 1-4) JBoss Remoting version 4.0.24.Final-redhat-1 >[0m[0m2017-10-18 12:16:20,474 INFO [org.jboss.as.connector] (MSC service thread 1-7) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.7.Final-redhat-1) >[0m[0m2017-10-18 12:16:20,459 INFO [org.jboss.as.security] (ServerService Thread Pool -- 58) WFLYSEC0002: Activating Security Subsystem >[0m[0m2017-10-18 12:16:20,563 INFO [org.jboss.as.security] (MSC service thread 1-7) WFLYSEC0001: Current PicketBox version=4.9.8.Final-redhat-1 >[0m[0m2017-10-18 12:16:20,583 INFO [org.jboss.as.naming] (MSC service thread 1-8) WFLYNAM0003: Starting Naming Service >[0m[0m2017-10-18 12:16:20,594 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0003: Undertow 1.3.31.Final-redhat-1 starting >[0m[0m2017-10-18 12:16:20,595 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 61) WFLYUT0003: Undertow 1.3.31.Final-redhat-1 starting >[0m[0m2017-10-18 12:16:20,599 INFO [org.jboss.as.mail.extension] (MSC service thread 1-2) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default] >[0m[0m2017-10-18 12:16:20,659 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 36) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3) >[0m[0m2017-10-18 12:16:20,664 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = h2 >[0m[0m2017-10-18 12:16:20,950 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 61) WFLYUT0014: Creating file handler for path '/opt/eap/welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]'] >[0m[0m2017-10-18 12:16:20,964 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0012: Started server default-server. >[0m[0m2017-10-18 12:16:20,980 INFO [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0018: Host default-host starting >[0m[0m2017-10-18 12:16:20,989 INFO [org.jboss.as.ejb3] (MSC service thread 1-7) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 64 (per class), which is derived from thread worker pool sizing. >[0m[0m2017-10-18 12:16:20,990 INFO [org.jboss.as.ejb3] (MSC service thread 1-6) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 16 (per class), which is derived from the number of CPUs on this host. >[0m[0m2017-10-18 12:16:21,060 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080 >[0m[0m2017-10-18 12:16:21,061 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0006: Undertow AJP listener ajp listening on 0.0.0.0:8009 >[0m[0m2017-10-18 12:16:21,110 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 64) MODCLUSTER000001: Initializing mod_cluster version 1.3.3.Final-redhat-1 >[0m[0m2017-10-18 12:16:21,179 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 64) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364 >[0m[0m2017-10-18 12:16:21,374 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS] >[0m[0m2017-10-18 12:16:21,531 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0027: Starting deployment of "activemq-rar.rar" (runtime-name: "activemq-rar.rar") >[0m[0m2017-10-18 12:16:21,531 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-metrics.ear" (runtime-name: "hawkular-metrics.ear") >[0m[0m2017-10-18 12:16:21,557 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-5) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/eap/standalone/deployments >[0m[0m2017-10-18 12:16:21,694 INFO [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0006: Undertow HTTPS listener https listening on 0.0.0.0:8443 >[0m[0m2017-10-18 12:16:22,426 INFO [org.jboss.ws.common.management] (MSC service thread 1-4) JBWS022052: Starting JBossWS 5.1.8.Final-redhat-1 (Apache CXF 3.1.10.redhat-1) >[0m[0m2017-10-18 12:16:22,893 INFO [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-8) IJ020001: Required license terms for file:/opt/eap/standalone/tmp/vfs/temp/temp6da3d6bcf769a2d9/content-b44979ea00bef212/contents/ >[0m[0m2017-10-18 12:16:22,962 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts.war") >[0m[0m2017-10-18 12:16:22,963 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-email.war") >[0m[0m2017-10-18 12:16:22,964 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-metrics-alerter.war") >[0m[0m2017-10-18 12:16:22,964 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-elasticsearch-alerter.war") >[0m[0m2017-10-18 12:16:22,964 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-elasticsearch.war") >[0m[0m2017-10-18 12:16:22,964 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-metrics.war") >[0m[0m2017-10-18 12:16:22,971 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-webhook.war") >[0m[33m2017-10-18 12:16:23,347 WARN [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-8) IJ020017: Invalid archive: file:/opt/eap/standalone/tmp/vfs/temp/temp6da3d6bcf769a2d9/content-b44979ea00bef212/contents/ >[0m[33m2017-10-18 12:16:24,549 WARN [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0001: /content/hawkular-metrics.ear/hawkular-metrics.war/WEB-INF/jboss-deployment-structure.xml in subdeployment ignored. jboss-deployment-structure.xml is only parsed for top level deployments. >[0m[33m2017-10-18 12:16:25,851 WARN [org.jboss.as.dependency.private] (MSC service thread 1-7) WFLYSRV0018: Deployment "deployment.hawkular-metrics.ear" is using a private module ("org.infinispan:main") which may be changed or removed in future versions without notice. >[0m[33m2017-10-18 12:16:25,921 WARN [org.jboss.as.dependency.private] (MSC service thread 1-3) WFLYSRV0018: Deployment "deployment.hawkular-metrics.ear.hawkular-alerts.war" is using a private module ("com.google.guava:main") which may be changed or removed in future versions without notice. >[0m[0m2017-10-18 12:16:25,958 INFO [org.jboss.weld.deployer] (MSC service thread 1-2) WFLYWELD0003: Processing weld deployment hawkular-metrics.ear >[0m[0m2017-10-18 12:16:26,092 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-2) HV000001: Hibernate Validator 5.2.5.Final-redhat-2 >[0m[0m2017-10-18 12:16:26,309 INFO [org.jboss.weld.deployer] (MSC service thread 1-8) WFLYWELD0003: Processing weld deployment hawkular-metrics.war >[0m[0m2017-10-18 12:16:26,321 INFO [org.jboss.weld.deployer] (MSC service thread 1-7) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-elasticsearch.war >[0m[0m2017-10-18 12:16:26,325 INFO [org.jboss.weld.deployer] (MSC service thread 1-3) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-webhook.war >[0m[0m2017-10-18 12:16:26,326 INFO [org.jboss.weld.deployer] (MSC service thread 1-5) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-email.war >[0m[0m2017-10-18 12:16:26,327 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0003: Processing weld deployment hawkular-metrics-alerter.war >[0m[0m2017-10-18 12:16:26,333 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'ConditionManager' in deployment unit 'subdeployment "hawkular-metrics-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-metrics-alerter/ConditionManager!org.hawkular.metrics.alerter.ConditionManager > java:app/hawkular-metrics-alerter/ConditionManager!org.hawkular.metrics.alerter.ConditionManager > java:module/ConditionManager!org.hawkular.metrics.alerter.ConditionManager > java:global/hawkular-metrics/hawkular-metrics-alerter/ConditionManager > java:app/hawkular-metrics-alerter/ConditionManager > java:module/ConditionManager >[0m[0m2017-10-18 12:16:26,339 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-5) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-email.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts-action-email/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:app/hawkular-alerts-action-email/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:global/hawkular-metrics/hawkular-alerts-action-email/StandaloneActionPluginRegister > java:app/hawkular-alerts-action-email/StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister >[0m[0m2017-10-18 12:16:26,346 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-3) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-webhook.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts-action-webhook/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:app/hawkular-alerts-action-webhook/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:global/hawkular-metrics/hawkular-alerts-action-webhook/StandaloneActionPluginRegister > java:app/hawkular-alerts-action-webhook/StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister >[0m[0m2017-10-18 12:16:26,333 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'GroupTriggerManager' in deployment unit 'subdeployment "hawkular-metrics-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-metrics-alerter/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager > java:app/hawkular-metrics-alerter/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager > java:module/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager > java:global/hawkular-metrics/hawkular-metrics-alerter/GroupTriggerManager > java:app/hawkular-metrics-alerter/GroupTriggerManager > java:module/GroupTriggerManager >[0m[0m2017-10-18 12:16:26,369 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-7) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-elasticsearch.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:app/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister > java:global/hawkular-metrics/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister > java:app/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister > java:module/StandaloneActionPluginRegister >[0m[0m2017-10-18 12:16:26,380 INFO [org.jboss.weld.deployer] (MSC service thread 1-2) WFLYWELD0003: Processing weld deployment hawkular-elasticsearch-alerter.war >[0m[0m2017-10-18 12:16:26,381 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2) WFLYEJB0473: JNDI bindings for session bean named 'ElasticsearchAlerter' in deployment unit 'subdeployment "hawkular-elasticsearch-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-elasticsearch-alerter/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter > java:app/hawkular-elasticsearch-alerter/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter > java:module/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter > java:global/hawkular-metrics/hawkular-elasticsearch-alerter/ElasticsearchAlerter > java:app/hawkular-elasticsearch-alerter/ElasticsearchAlerter > java:module/ElasticsearchAlerter >[0m[0m2017-10-18 12:16:26,527 INFO [org.jboss.weld.deployer] (MSC service thread 1-4) WFLYWELD0003: Processing weld deployment hawkular-alerts.war >[0m[0m2017-10-18 12:16:26,532 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'DroolsRulesEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine > java:app/hawkular-alerts/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine > java:module/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine > java:global/hawkular-metrics/hawkular-alerts/DroolsRulesEngineImpl > java:app/hawkular-alerts/DroolsRulesEngineImpl > java:module/DroolsRulesEngineImpl >[0m[0m2017-10-18 12:16:26,532 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'CepEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine > java:app/hawkular-alerts/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine > java:module/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine > java:global/hawkular-metrics/hawkular-alerts/CepEngineImpl > java:app/hawkular-alerts/CepEngineImpl > java:module/CepEngineImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'PropertiesServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService > java:app/hawkular-alerts/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService > java:module/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService > java:global/hawkular-metrics/hawkular-alerts/PropertiesServiceImpl > java:app/hawkular-alerts/PropertiesServiceImpl > java:module/PropertiesServiceImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'IncomingDataManagerImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager > java:app/hawkular-alerts/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager > java:module/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager > java:global/hawkular-metrics/hawkular-alerts/IncomingDataManagerImpl > java:app/hawkular-alerts/IncomingDataManagerImpl > java:module/IncomingDataManagerImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'AlertsContext' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext > java:app/hawkular-alerts/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext > java:module/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext > java:global/hawkular-metrics/hawkular-alerts/AlertsContext > java:app/hawkular-alerts/AlertsContext > java:module/AlertsContext >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'AlertsEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine > java:app/hawkular-alerts/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine > java:module/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine > java:global/hawkular-metrics/hawkular-alerts/AlertsEngineImpl > java:app/hawkular-alerts/AlertsEngineImpl > java:module/AlertsEngineImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'CassActionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService > java:app/hawkular-alerts/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService > java:module/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService > java:global/hawkular-metrics/hawkular-alerts/CassActionsServiceImpl > java:app/hawkular-alerts/CassActionsServiceImpl > java:module/CassActionsServiceImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'PartitionManagerImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager > java:app/hawkular-alerts/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager > java:module/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager > java:global/hawkular-metrics/hawkular-alerts/PartitionManagerImpl > java:app/hawkular-alerts/PartitionManagerImpl > java:module/PartitionManagerImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'CassAlertsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService > java:app/hawkular-alerts/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService > java:module/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService > java:global/hawkular-metrics/hawkular-alerts/CassAlertsServiceImpl > java:app/hawkular-alerts/CassAlertsServiceImpl > java:module/CassAlertsServiceImpl >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'StreamWatcher' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher > java:app/hawkular-alerts/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher > java:module/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher > java:global/hawkular-metrics/hawkular-alerts/StreamWatcher > java:app/hawkular-alerts/StreamWatcher > java:module/StreamWatcher >[0m[0m2017-10-18 12:16:26,533 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'StatusServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService > java:app/hawkular-alerts/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService > java:module/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService > java:global/hawkular-metrics/hawkular-alerts/StatusServiceImpl > java:app/hawkular-alerts/StatusServiceImpl > java:module/StatusServiceImpl >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'PublishCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager > java:app/hawkular-alerts/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager > java:module/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager > java:global/hawkular-metrics/hawkular-alerts/PublishCacheManager > java:app/hawkular-alerts/PublishCacheManager > java:module/PublishCacheManager >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'DataDrivenGroupCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager > java:app/hawkular-alerts/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager > java:module/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager > java:global/hawkular-metrics/hawkular-alerts/DataDrivenGroupCacheManager > java:app/hawkular-alerts/DataDrivenGroupCacheManager > java:module/DataDrivenGroupCacheManager >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'ExtensionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService > java:app/hawkular-alerts/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService > java:module/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService > java:global/hawkular-metrics/hawkular-alerts/ExtensionsServiceImpl > java:app/hawkular-alerts/ExtensionsServiceImpl > java:module/ExtensionsServiceImpl >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'EventsAggregationExtension' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension > java:app/hawkular-alerts/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension > java:module/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension > java:global/hawkular-metrics/hawkular-alerts/EventsAggregationExtension > java:app/hawkular-alerts/EventsAggregationExtension > java:module/EventsAggregationExtension >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'CassCluster' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/CassCluster!org.hawkular.alerts.engine.impl.CassCluster > java:app/hawkular-alerts/CassCluster!org.hawkular.alerts.engine.impl.CassCluster > java:module/CassCluster!org.hawkular.alerts.engine.impl.CassCluster > java:global/hawkular-metrics/hawkular-alerts/CassCluster > java:app/hawkular-alerts/CassCluster > java:module/CassCluster >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'CassDefinitionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService > java:app/hawkular-alerts/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService > java:module/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService > java:global/hawkular-metrics/hawkular-alerts/CassDefinitionsServiceImpl > java:app/hawkular-alerts/CassDefinitionsServiceImpl > java:module/CassDefinitionsServiceImpl >[0m[0m2017-10-18 12:16:26,534 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'ActionsCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: > java:global/hawkular-metrics/hawkular-alerts/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager > java:app/hawkular-alerts/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager > java:module/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager > java:global/hawkular-metrics/hawkular-alerts/ActionsCacheManager > java:app/hawkular-alerts/ActionsCacheManager > java:module/ActionsCacheManager >[0m[0m2017-10-18 12:16:26,548 INFO [org.jboss.weld.deployer] (MSC service thread 1-4) WFLYWELD0006: Starting Services for CDI deployment: hawkular-metrics.ear >[0m[0m2017-10-18 12:16:26,591 INFO [org.jboss.weld.Version] (MSC service thread 1-4) WELD-000900: 2.3.3 (redhat) >[0m[0m2017-10-18 12:16:26,608 INFO [org.jboss.weld.deployer] (MSC service thread 1-5) WFLYWELD0009: Starting weld service for deployment hawkular-metrics.ear >[0m[0m2017-10-18 12:16:27,079 INFO [org.jgroups.protocols.openshift.KUBE_PING] (MSC service thread 1-1) namespace [openshift-infra] set; clustering enabled >[0m[0m2017-10-18 12:16:27,334 INFO [org.jgroups.protocols.openshift.KUBE_PING] (MSC service thread 1-1) Starting UndertowServer on port 8888 for channel address: hawkular-metrics-n15zd >[0m[0m2017-10-18 12:16:27,360 INFO [org.jgroups.protocols.openshift.KUBE_PING] (MSC service thread 1-1) UndertowServer started. >[0m[31m2017-10-18 12:16:27,411 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (MSC service thread 1-1) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:385) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3.down(MERGE3.java:254) > at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:371) > at org.jgroups.protocols.FD.down(FD.java:318) > at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92) > at org.jgroups.protocols.EncryptBase.down(EncryptBase.java:135) > at org.jgroups.protocols.ASYM_ENCRYPT.down(ASYM_ENCRYPT.java:107) > at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:582) > at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:675) > at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:347) > at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:76) > at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:41) > at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1082) > at org.jgroups.protocols.FlowControl.down(FlowControl.java:347) > at org.jgroups.protocols.FRAG2.down(FRAG2.java:136) > at org.jgroups.protocols.AUTH.down(AUTH.java:192) > at org.jgroups.protocols.FORK.down(FORK.java:101) > at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1039) > at org.jgroups.JChannel.down(JChannel.java:790) > at org.jgroups.JChannel._connect(JChannel.java:564) > at org.jgroups.JChannel.connect(JChannel.java:294) > at org.jgroups.JChannel.connect(JChannel.java:279) > at org.wildfly.clustering.jgroups.spi.service.ChannelConnectorBuilder.start(ChannelConnectorBuilder.java:94) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:2032) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1955) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 45 more >[0m[31m2017-10-18 12:16:28,433 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (MSC service thread 1-1) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:385) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3.down(MERGE3.java:254) > at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:371) > at org.jgroups.protocols.FD.down(FD.java:318) > at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92) > at org.jgroups.protocols.EncryptBase.down(EncryptBase.java:135) > at org.jgroups.protocols.ASYM_ENCRYPT.down(ASYM_ENCRYPT.java:107) > at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:582) > at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:675) > at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:347) > at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:76) > at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:41) > at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1082) > at org.jgroups.protocols.FlowControl.down(FlowControl.java:347) > at org.jgroups.protocols.FRAG2.down(FRAG2.java:136) > at org.jgroups.protocols.AUTH.down(AUTH.java:192) > at org.jgroups.protocols.FORK.down(FORK.java:101) > at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1039) > at org.jgroups.JChannel.down(JChannel.java:790) > at org.jgroups.JChannel._connect(JChannel.java:564) > at org.jgroups.JChannel.connect(JChannel.java:294) > at org.jgroups.JChannel.connect(JChannel.java:279) > at org.wildfly.clustering.jgroups.spi.service.ChannelConnectorBuilder.start(ChannelConnectorBuilder.java:94) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:2032) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1955) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 45 more >[0m[31m2017-10-18 12:16:29,435 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (MSC service thread 1-1) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:385) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3.down(MERGE3.java:254) > at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:371) > at org.jgroups.protocols.FD.down(FD.java:318) > at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:92) > at org.jgroups.protocols.EncryptBase.down(EncryptBase.java:135) > at org.jgroups.protocols.ASYM_ENCRYPT.down(ASYM_ENCRYPT.java:107) > at org.jgroups.protocols.pbcast.NAKACK2.down(NAKACK2.java:582) > at org.jgroups.protocols.UNICAST3.down(UNICAST3.java:675) > at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:347) > at org.jgroups.protocols.pbcast.ClientGmsImpl.joinInternal(ClientGmsImpl.java:76) > at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:41) > at org.jgroups.protocols.pbcast.GMS.down(GMS.java:1082) > at org.jgroups.protocols.FlowControl.down(FlowControl.java:347) > at org.jgroups.protocols.FRAG2.down(FRAG2.java:136) > at org.jgroups.protocols.AUTH.down(AUTH.java:192) > at org.jgroups.protocols.FORK.down(FORK.java:101) > at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:1039) > at org.jgroups.JChannel.down(JChannel.java:790) > at org.jgroups.JChannel._connect(JChannel.java:564) > at org.jgroups.JChannel.connect(JChannel.java:294) > at org.jgroups.JChannel.connect(JChannel.java:279) > at org.wildfly.clustering.jgroups.spi.service.ChannelConnectorBuilder.start(ChannelConnectorBuilder.java:94) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:2032) > at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1955) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 45 more >[0m[33m2017-10-18 12:16:30,436 WARN [org.jgroups.protocols.openshift.KUBE_PING] (MSC service thread 1-1) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[0m2017-10-18 12:16:33,912 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000078: Starting JGroups channel hawkular-metrics >[0m[0m2017-10-18 12:16:33,914 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel server >[0m[0m2017-10-18 12:16:33,916 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000078: Starting JGroups channel web >[0m[0m2017-10-18 12:16:33,916 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups channel ejb >[0m[0m2017-10-18 12:16:33,917 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000078: Starting JGroups channel hibernate >[0m[0m2017-10-18 12:16:33,917 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000078: Starting JGroups channel hawkular-alerts >[0m[0m2017-10-18 12:16:33,923 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel server: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,925 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000094: Received new cluster view for channel ejb: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,925 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000094: Received new cluster view for channel web: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,926 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000094: Received new cluster view for channel hawkular-metrics: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,926 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000094: Received new cluster view for channel hibernate: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,926 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000094: Received new cluster view for channel hawkular-alerts: [hawkular-metrics-n15zd|0] (1) [hawkular-metrics-n15zd] >[0m[0m2017-10-18 12:16:33,929 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Channel ejb local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:33,935 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-3) ISPN000079: Channel web local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:33,936 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Mahou' 8.1.8.Final-redhat-1 >[0m[0m2017-10-18 12:16:33,937 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-8) ISPN000079: Channel hawkular-metrics local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:33,945 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel server local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:33,946 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-5) ISPN000079: Channel hibernate local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:33,950 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-6) ISPN000079: Channel hawkular-alerts local address is hawkular-metrics-n15zd, physical addresses are [10.129.0.54:7600] >[0m[0m2017-10-18 12:16:34,172 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started dataIds cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,187 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 72) WFLYCLINF0002: Started partition cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,188 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 71) WFLYCLINF0002: Started globalActions cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,190 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 70) WFLYCLINF0002: Started publish cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,191 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 68) WFLYCLINF0002: Started schema cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,191 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 66) WFLYCLINF0002: Started data cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,197 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 69) WFLYCLINF0002: Started triggers cache from hawkular-alerts container >[0m[0m2017-10-18 12:16:34,201 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 64) WFLYCLINF0002: Started client-mappings cache from ejb container >[0m[0m2017-10-18 12:16:34,307 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 76) HAWKALERT220023: Init Actions Cache >[0m[33m2017-10-18 12:16:34,336 WARN [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 70) HAWKALERT220021: Clear Publish Cache >[0m[0m2017-10-18 12:16:34,336 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 70) HAWKALERT220020: Init Publish Cache >[0m[0m2017-10-18 12:16:34,337 INFO [org.hawkular.alerts.extensions.EventsAggregationExtension] (ServerService Thread Pool -- 64) Registering Distributed Trigger listener >[0m[0m2017-10-18 12:16:34,427 INFO [com.datastax.driver.core.SystemProperties] (ServerService Thread Pool -- 74) com.datastax.driver.FORCE_NIO is defined, using value true >[0m[0m2017-10-18 12:16:34,438 INFO [com.datastax.driver.core.NettyUtil] (ServerService Thread Pool -- 74) Found Netty's native epoll transport in the classpath, but NIO was forced through the FORCE_NIO system property. >[0m[33m2017-10-18 12:16:35,832 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Could not connect to Cassandra cluster - assuming is not up yet. Cause: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: hawkular-cassandra/172.30.162.177:9042 (com.datastax.driver.core.exceptions.TransportException: [hawkular-cassandra/172.30.162.177:9042] Cannot connect)) >[0m[33m2017-10-18 12:16:35,834 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) [15] Retrying connecting to Cassandra cluster in [3000]ms... >[0m[33m2017-10-18 12:16:41,860 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Could not connect to Cassandra cluster - assuming is not up yet. Cause: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: hawkular-cassandra/172.30.162.177:9042 (com.datastax.driver.core.exceptions.TransportException: [hawkular-cassandra/172.30.162.177:9042] Cannot connect)) >[0m[33m2017-10-18 12:16:41,862 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) [14] Retrying connecting to Cassandra cluster in [3000]ms... >[0m[33m2017-10-18 12:16:47,899 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Could not connect to Cassandra cluster - assuming is not up yet. Cause: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: hawkular-cassandra/172.30.162.177:9042 (com.datastax.driver.core.exceptions.TransportException: [hawkular-cassandra/172.30.162.177:9042] Cannot connect)) >[0m[33m2017-10-18 12:16:47,899 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) [13] Retrying connecting to Cassandra cluster in [3000]ms... >[0m[31m2017-10-18 12:16:51,329 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:16:52,330 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:16:53,331 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:16:53,924 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Could not connect to Cassandra cluster - assuming is not up yet. Cause: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: hawkular-cassandra/172.30.162.177:9042 (com.datastax.driver.core.exceptions.TransportException: [hawkular-cassandra/172.30.162.177:9042] Cannot connect)) >[0m[33m2017-10-18 12:16:53,924 WARN [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) [12] Retrying connecting to Cassandra cluster in [3000]ms... >[0m[33m2017-10-18 12:16:54,332 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[0m2017-10-18 12:16:57,432 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (ServerService Thread Pool -- 74) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) >[0m[0m2017-10-18 12:16:57,433 INFO [com.datastax.driver.core.Cluster] (ServerService Thread Pool -- 74) New Cassandra host hawkular-cassandra/172.30.162.177:9042 added >[0m[0m2017-10-18 12:16:57,518 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Checking Schema existence for keyspace: hawkular_alerts >[0m[0m2017-10-18 12:16:58,341 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Executing [script:vfs:/content/hawkular-metrics.ear/hawkular-alerts.war/WEB-INF/lib/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar/org/hawkular/alerts/schema/cassalog.groovy, tags:[], vars:[keyspace:hawkular_alerts, reset:false, session:com.datastax.driver.core.SessionManager@57a017a1, logger:org.jboss.logging.JBossLogManagerLogger@355a2fdb]] >[0m[0m2017-10-18 12:16:58,721 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table triggers >[0m[0m2017-10-18 12:16:58,921 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table triggers_actions >[0m[0m2017-10-18 12:16:59,052 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table conditions >[0m[0m2017-10-18 12:16:59,277 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table dampenings >[0m[0m2017-10-18 12:16:59,442 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table action_plugins >[0m[0m2017-10-18 12:16:59,512 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_definitions >[0m[0m2017-10-18 12:16:59,598 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_history >[0m[0m2017-10-18 12:16:59,712 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_history_actions >[0m[0m2017-10-18 12:16:59,813 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_history_alerts >[0m[0m2017-10-18 12:16:59,886 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_history_ctimes >[0m[0m2017-10-18 12:16:59,983 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table actions_history_results >[0m[0m2017-10-18 12:17:00,059 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table tags >[0m[0m2017-10-18 12:17:00,129 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts >[0m[0m2017-10-18 12:17:00,195 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts_triggers >[0m[0m2017-10-18 12:17:00,290 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts_ctimes >[0m[0m2017-10-18 12:17:00,383 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts_statuses >[0m[0m2017-10-18 12:17:00,471 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts_severities >[0m[0m2017-10-18 12:17:00,535 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table alerts_lifecycle >[0m[0m2017-10-18 12:17:00,627 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table events >[0m[0m2017-10-18 12:17:00,723 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table events_triggers >[0m[0m2017-10-18 12:17:00,807 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table events_ctimes >[0m[0m2017-10-18 12:17:00,876 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Creating table events_categories >[0m[0m2017-10-18 12:17:01,119 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Creating change log table hawkular_alerts.cassalog >[0m[33m2017-10-18 12:17:01,318 WARN [com.datastax.driver.core.Cluster] (cluster5-worker-2) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='UPDATE hawkular_alerts.cassalog SET applied_at = ? WHERE bucket = ? AND revision = ?' >[0m[0m2017-10-18 12:17:01,342 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:01,388 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:01,397 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 1.0 >CREATE TABLE sys_config ( > config_id text, > name text, > value text, > PRIMARY KEY (config_id, name) >) >-- >[0m[0m2017-10-18 12:17:01,494 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:01,510 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 2.0 >ALTER TABLE conditions ADD interval bigint >-- >[0m[0m2017-10-18 12:17:01,656 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:01,661 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 3.0 >DROP TABLE alerts_statuses >-- >[0m[0m2017-10-18 12:17:01,782 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 3.1 >DROP TABLE alerts_severities >-- >[0m[0m2017-10-18 12:17:01,867 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:01,875 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 4.0 >ALTER TABLE conditions ADD activeRules set<text> >-- >[0m[0m2017-10-18 12:17:01,959 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 4.1 >ALTER TABLE conditions ADD sampleSize int >-- >[0m[0m2017-10-18 12:17:02,051 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: set-keyspace >USE hawkular_alerts >-- >[0m[0m2017-10-18 12:17:02,059 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 74) Applying ChangeSet >-- version: 5.0 >CREATE TABLE alerts_stimes ( > tenantId text, > alertId text, > stime bigint, > PRIMARY KEY (tenantId, stime, alertId) >) >-- >[0m[0m2017-10-18 12:17:02,319 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 74) Done creating Schema for keyspace: hawkular_alerts >[0m[0m2017-10-18 12:17:02,363 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 68) HAWKALERT220015: Hawkular Alerts deployed in distributed mode >[0m[0m2017-10-18 12:17:02,367 INFO [org.hawkular.alerts.engine.impl.ExtensionsServiceImpl] (ServerService Thread Pool -- 64) Adding EventExtension org.hawkular.alerts.extensions.EventsAggregationExtension@1197e8bc >[0m[0m2017-10-18 12:17:02,377 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject] (EE-ManagedExecutorService-default-Thread-1) Found kmodule: vfs:/content/hawkular-metrics.ear/hawkular-alerts.war/WEB-INF/lib/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar/META-INF/kmodule.xml >[0m[0m2017-10-18 12:17:02,378 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject] (EE-ManagedExecutorService-default-Thread-1) Virtual file physical path = /opt/eap/standalone/tmp/vfs/deployment/deploymentb51992fb3793d2a6/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar-6276b742442841c5/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar >[0m[0m2017-10-18 12:17:02,499 INFO [org.hawkular.openshift.auth.OpenshiftAuthServletExtension] (ServerService Thread Pool -- 66) Enabling the OpenShift Namespace Filter. >[0m[0m2017-10-18 12:17:02,579 INFO [org.drools.compiler.kie.builder.impl.KieRepositoryImpl] (EE-ManagedExecutorService-default-Thread-1) KieModule was added: ZipKieModule[releaseId=org.hawkular.alerts:hawkular-alerts-engine:1.7.0.Final-redhat-1,file=/opt/eap/standalone/tmp/vfs/deployment/deploymentb51992fb3793d2a6/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar-6276b742442841c5/hawkular-alerts-engine-1.7.0.Final-redhat-1.jar] >[0m[0m2017-10-18 12:17:02,583 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 66) Initializing Mojarra 2.2.12.SP4 for context '/hawkular/alerts' >[0m[0m2017-10-18 12:17:04,047 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 66) Creating NamespaceHandler >[0m[0m2017-10-18 12:17:04,048 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 66) Received token from /var/run/secrets/kubernetes.io/serviceaccount/token >[0m[0m2017-10-18 12:17:04,176 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (ServerService Thread Pool -- 66) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) >[0m[0m2017-10-18 12:17:04,180 INFO [com.datastax.driver.core.Cluster] (ServerService Thread Pool -- 66) New Cassandra host hawkular-cassandra/172.30.162.177:9042 added >[0m[0m2017-10-18 12:17:04,618 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 66) Received namespace overrides: {} >[0m[0m2017-10-18 12:17:05,161 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 66) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.alerts.rest.HawkularAlertsApp$Proxy$_$$_WeldClientProxy >[0m[0m2017-10-18 12:17:05,544 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 66) WFLYUT0021: Registered web context: /hawkular/alerts >[0m[0m2017-10-18 12:17:05,661 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 66) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@2dcaab75] registered >[0m[0m2017-10-18 12:17:05,661 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 66) HAWKALERT270001: Action plugin [elasticsearch] registered >[0m[0m2017-10-18 12:17:05,701 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 66) WFLYUT0021: Registered web context: /hawkular/actions/elasticsearch >[0m[0m2017-10-18 12:17:05,939 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 76) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@77d209a3] registered >[0m[0m2017-10-18 12:17:05,940 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 76) HAWKALERT270001: Action plugin [email] registered >[0m[0m2017-10-18 12:17:05,987 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 76) WFLYUT0021: Registered web context: /hawkular/actions/email >[0m[0m2017-10-18 12:17:06,046 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 76) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@207fbf08] registered >[0m[0m2017-10-18 12:17:06,046 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 76) HAWKALERT270001: Action plugin [webhook] registered >[0m[0m2017-10-18 12:17:06,103 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 76) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.alerts.actions.webhook.WebHookApp >[0m[0m2017-10-18 12:17:06,114 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 76) WFLYUT0021: Registered web context: /hawkular/actions/webhook >[0m[0m2017-10-18 12:17:06,176 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 76) Initializing Mojarra 2.2.12.SP4 for context '/hawkular/__alerter-elasticsearch' >[0m[0m2017-10-18 12:17:06,420 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 76) WFLYUT0021: Registered web context: /hawkular/__alerter-elasticsearch >[0m[0m2017-10-18 12:17:06,468 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 76) WFLYCLINF0002: Started locks cache from hawkular-metrics container >[0m[0m2017-10-18 12:17:06,491 INFO [org.hawkular.openshift.auth.OpenshiftAuthServletExtension] (ServerService Thread Pool -- 75) Enabling the OpenShift Namespace Filter. >[0m[0m2017-10-18 12:17:06,517 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (ServerService Thread Pool -- 75) Hawkular Metrics version: 0.27.6.Final-redhat-1 >[0m[0m2017-10-18 12:17:06,520 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) HAWKMETRICS200002: Initializing metrics service >[0m[0m2017-10-18 12:17:06,522 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 75) Initializing Mojarra 2.2.12.SP4 for context '/hawkular/metrics' >[0m[0m2017-10-18 12:17:06,636 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (metricsservice-lifecycle-thread) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) >[0m[0m2017-10-18 12:17:06,636 INFO [com.datastax.driver.core.Cluster] (metricsservice-lifecycle-thread) New Cassandra host hawkular-cassandra/172.30.162.177:9042 added >[0m[0m2017-10-18 12:17:06,945 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Executing [script:vfs:/content/hawkular-metrics.ear/hawkular-alerts.war/WEB-INF/lib/hawkular-metrics-schema-0.27.6.Final-redhat-1.jar/org/hawkular/schema/cassalog.groovy, tags:[0.15.x, 0.18.x, 0.19.x, 0.20.x, 0.21.x, 0.23.x, 0.26.x, 0.27.x], vars:[keyspace:hawkular_metrics, reset:false, session:com.datastax.driver.core.SessionManager@70fbda3c, replicationFactor:1, logger:org.jboss.logging.JBossLogManagerLogger@4ee3df9a]] >[0m[0m2017-10-18 12:17:07,400 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 75) Creating NamespaceHandler >[0m[0m2017-10-18 12:17:07,400 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 75) Received token from /var/run/secrets/kubernetes.io/serviceaccount/token >[0m[0m2017-10-18 12:17:07,406 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tenants >[0m[0m2017-10-18 12:17:07,516 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (ServerService Thread Pool -- 75) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) >[0m[0m2017-10-18 12:17:07,516 INFO [com.datastax.driver.core.Cluster] (ServerService Thread Pool -- 75) New Cassandra host hawkular-cassandra/172.30.162.177:9042 added >[0m[0m2017-10-18 12:17:07,519 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type aggregation_template >[0m[0m2017-10-18 12:17:07,704 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tenants_by_time >[0m[0m2017-10-18 12:17:07,771 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type aggregate_data >[0m[0m2017-10-18 12:17:07,832 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table data >[0m[0m2017-10-18 12:17:07,906 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table metrics_tags_idx >[0m[0m2017-10-18 12:17:08,021 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table metrics_idx >[0m[0m2017-10-18 12:17:08,061 INFO [org.hawkular.openshift.auth.org.hawkular.openshift.namespace.NamespaceHandler] (ServerService Thread Pool -- 75) Received namespace overrides: {} >[0m[0m2017-10-18 12:17:08,078 INFO [org.hawkular.metrics.api.jaxrs.HawkularMetricsRestApp] (ServerService Thread Pool -- 75) HAWKMETRICS200001: Hawkular Metrics starting >[0m[0m2017-10-18 12:17:08,078 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 75) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.metrics.api.jaxrs.HawkularMetricsRestApp >[0m[0m2017-10-18 12:17:08,189 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table retentions_idx >[0m[0m2017-10-18 12:17:08,353 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type trigger_def >[0m[0m2017-10-18 12:17:08,391 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 75) WFLYUT0021: Registered web context: /hawkular/metrics >[0m[0m2017-10-18 12:17:08,407 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tasks >[0m[0m2017-10-18 12:17:08,510 INFO [org.hawkular.metrics.alerter.groups.GroupTriggerManager] (ServerService Thread Pool -- 70) Starting Hawkular Metrics Group Trigger Manager, distributed=true >[0m[0m2017-10-18 12:17:08,511 INFO [org.hawkular.metrics.alerter.ConditionManager] (ServerService Thread Pool -- 64) Starting Hawkular Metrics External Alerter, distributed=true >[0m[0m2017-10-18 12:17:08,554 INFO [org.hawkular.metrics.alerter.ConditionManager] (ServerService Thread Pool -- 64) Registering Trigger UPDATE/REMOVE listener >[0m[0m2017-10-18 12:17:08,561 INFO [org.hawkular.metrics.alerter.groups.GroupTriggerManager] (ServerService Thread Pool -- 70) Registering Trigger UPDATE/REMOVE listener >[0m[0m2017-10-18 12:17:08,565 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table leases >[0m[0m2017-10-18 12:17:08,606 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 70) Initializing Mojarra 2.2.12.SP4 for context '/hawkular/__alerter-metrics' >[0m[0m2017-10-18 12:17:08,757 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 70) WFLYUT0021: Registered web context: /hawkular/__alerter-metrics >[0m[0m2017-10-18 12:17:08,785 INFO [org.jboss.as.server] (ServerService Thread Pool -- 37) WFLYSRV0010: Deployed "hawkular-metrics.ear" (runtime-name : "hawkular-metrics.ear") >[0m[0m2017-10-18 12:17:08,794 INFO [org.jboss.as.server] (ServerService Thread Pool -- 37) WFLYSRV0010: Deployed "activemq-rar.rar" (runtime-name : "activemq-rar.rar") >[0m[0m2017-10-18 12:17:09,030 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management >[0m[0m2017-10-18 12:17:09,030 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990 >[0m[0m2017-10-18 12:17:09,030 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.0.8.GA (WildFly Core 2.1.18.Final-redhat-1) started in 53761ms - Started 1600 of 2134 services (856 services are lazy, passive or on-demand) >[0m[0m2017-10-18 12:17:09,157 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Creating change log table hawkular_metrics.cassalog >[0m[33m2017-10-18 12:17:09,258 WARN [com.datastax.driver.core.Cluster] (cluster7-worker-2) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='UPDATE hawkular_metrics.cassalog SET applied_at = ? WHERE bucket = ? AND revision = ?' >[0m[0m2017-10-18 12:17:09,278 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: set-keyspace >USE hawkular_metrics >-- >[0m[0m2017-10-18 12:17:09,288 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 1.0 >CREATE TABLE sys_config ( > config_id text, > name text, > value text, > PRIMARY KEY (config_id, name) >) WITH compaction = { 'class': 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,370 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 1.1 >ALTER TABLE data WITH gc_grace_seconds = 86400 >-- >[0m[0m2017-10-18 12:17:09,415 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 1.2 >ALTER TABLE data ADD tags map<text,text> >-- >[0m[0m2017-10-18 12:17:09,466 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 1.3 >ALTER TABLE data ADD s_value text >-- >[0m[0m2017-10-18 12:17:09,542 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 1.4 >INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'string-size', '2048') >-- >[0m[0m2017-10-18 12:17:09,549 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: set-keyspace >USE hawkular_metrics >-- >[0m[0m2017-10-18 12:17:09,553 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 2.0 >CREATE TABLE locks ( > name text PRIMARY KEY, > value text, >) WITH compaction = { 'class': 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,600 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 2.1 >CREATE TABLE jobs ( > id uuid PRIMARY KEY, > type text, > name text, > params map<text, text>, > trigger frozen <trigger_def> >) WITH compaction = { 'class': 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,645 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 2.2 >CREATE TABLE scheduled_jobs_idx ( > time_slice timestamp, > job_id uuid, > PRIMARY KEY (time_slice, job_id) >) WITH compaction = { 'class': 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,731 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 2.3 >CREATE TABLE finished_jobs_idx ( > time_slice timestamp, > job_id uuid, > PRIMARY KEY (time_slice, job_id) >) WITH compaction = { 'class': 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,784 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 2.4 >CREATE TABLE active_time_slices ( > time_slice timestamp PRIMARY KEY >) WITH compaction = { 'class' : 'LeveledCompactionStrategy' } >-- >[0m[0m2017-10-18 12:17:09,847 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.0 >ALTER TABLE hawkular_metrics.data WITH COMPRESSION = {'sstable_compression': 'DeflateCompressor'}; >-- >[0m[0m2017-10-18 12:17:09,916 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.1 >ALTER TABLE data WITH compaction = {'class': 'DateTieredCompactionStrategy'} AND default_time_to_live = 604800 >-- >[0m[0m2017-10-18 12:17:09,965 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.2 >ALTER TABLE data DROP aggregates >-- >[0m[0m2017-10-18 12:17:10,038 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.3 >DROP TYPE aggregate_data >-- >[0m[0m2017-10-18 12:17:10,085 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.4 >ALTER TABLE data DROP data_retention >-- >[0m[0m2017-10-18 12:17:10,175 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.5 >DROP TYPE aggregation_template >-- >[0m[0m2017-10-18 12:17:10,228 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 3.6 >DROP TABLE tenants_by_time >-- >[0m[0m2017-10-18 12:17:10,326 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: set-keyspace >USE hawkular_metrics >-- >[0m[0m2017-10-18 12:17:10,335 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 4.0 >CREATE TABLE data_compressed ( > tenant_id text, > type tinyint, > metric text, > dpart bigint, > time timestamp, > c_value blob, > ts_value blob, > tags blob, > PRIMARY KEY ((tenant_id, type, metric, dpart), time) >) WITH CLUSTERING ORDER BY (time DESC) >-- >[0m[0m2017-10-18 12:17:10,400 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 4.1 >ALTER TABLE data WITH COMPRESSION = {'sstable_compression': 'LZ4Compressor'}; >-- >[0m[0m2017-10-18 12:17:10,453 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.0 >ALTER TABLE scheduled_jobs_idx ADD job_name text >-- >[0m[0m2017-10-18 12:17:10,502 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.1 >ALTER TABLE scheduled_jobs_idx ADD job_params frozen<map<text, text>> >-- >[0m[0m2017-10-18 12:17:10,548 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.2 >ALTER TABLE scheduled_jobs_idx ADD trigger frozen<trigger_def> >-- >[0m[0m2017-10-18 12:17:10,607 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.3 >ALTER TABLE scheduled_jobs_idx ADD job_type text >-- >[0m[0m2017-10-18 12:17:10,666 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.4 >ALTER TABLE scheduled_jobs_idx ADD status tinyint >-- >[0m[0m2017-10-18 12:17:10,710 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.5 >INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'gc_grace_seconds', '604800') >-- >[0m[0m2017-10-18 12:17:10,717 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 5.6 >BEGIN UNLOGGED BATCH > INSERT INTO sys_config (config_id, name, value) > VALUES ('org.hawkular.metrics', 'ingestion.retry.max-retries', '5'); > INSERT INTO sys_config (config_id, name, value) > VALUES ('org.hawkular.metrics', 'ingestion.retry.max-delay', '30000'); >APPLY BATCH; >-- >[0m[0m2017-10-18 12:17:10,743 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 6.0 >ALTER TABLE data_compressed WITH compaction = { > 'class': 'TimeWindowCompactionStrategy', > 'compaction_window_unit': 'DAYS', > 'compaction_window_size': '1' >} >-- >[0m[0m2017-10-18 12:17:10,816 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 6.1 >ALTER TABLE data WITH compaction = {'class': 'SizeTieredCompactionStrategy'} >-- >[0m[0m2017-10-18 12:17:10,891 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet >-- version: 7.0 > CREATE TABLE metrics_expiration_idx ( > tenant_id text, > type tinyint, > metric text, > time timestamp, > PRIMARY KEY ((tenant_id, type), metric) > ) >-- >[0m[0m2017-10-18 12:17:10,953 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The keyspace hawkular_metrics is using a replication_factor of 1 >[0m[0m2017-10-18 12:17:11,166 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100001: Using a key space of 'hawkular_metrics' >[0m[0m2017-10-18 12:17:11,217 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100008: Using max size of 2048 for string metrics >[0m[0m2017-10-18 12:17:11,217 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100009: Using max number of retries 5 and max retry delay of 30000 ms for inserting data points >[0m[0m2017-10-18 12:17:11,219 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100007: Using default data retention of 604800 seconds >[0m[33m2017-10-18 12:17:11,223 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (metricsservice-lifecycle-thread) Did not find meta data for timer openshift-oauth-kubernetes-response-time that has been added to the registry >[0m[33m2017-10-18 12:17:11,223 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (metricsservice-lifecycle-thread) Did not find meta data for timer openshift-oauth-latency that has been added to the registry >[0m[33m2017-10-18 12:17:11,438 WARN [com.datastax.driver.core.Cluster] (cluster7-worker-1) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='SELECT DISTINCT tenant_id, tname FROM metrics_tags_idx' >[0m[0m2017-10-18 12:17:11,448 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Preparing to create and schedule COMPRESS_DATA job >[0m[0m2017-10-18 12:17:11,605 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Created and scheduled JobDetailsImpl{jobId=72ebffdb-6172-4634-b974-ff0fedc9a84a, jobType=COMPRESS_DATA, jobName=COMPRESS_DATA, parameters={}, trigger=RepeatingTrigger{triggerTime=1508331600000, interval=7200000, delay=60000}, status=NONE} >[0m[0m2017-10-18 12:17:11,611 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Preparing to create and schedule DELETE_EXPIRED_METRICS job >[0m[0m2017-10-18 12:17:11,620 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Created and scheduled JobDetailsImpl{jobId=86799cc0-6d99-41cf-8d2c-dabbe459a31b, jobType=DELETE_EXPIRED_METRICS, jobName=DELETE_EXPIRED_METRICS, parameters={}, trigger=RepeatingTrigger{triggerTime=1508371200000, interval=604800000, delay=60000}, status=NONE} >[0m[0m2017-10-18 12:17:11,630 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (metricsservice-lifecycle-thread) Checking tables in hawkular_metrics to see if gc_grace_seconds needs to be updated >[0m[0m2017-10-18 12:17:11,634 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) HAWKMETRICS200005: Metrics service started >[0m[0m2017-10-18 12:17:11,634 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for active_time_slices is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:11,642 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for cassalog is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:11,642 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for data is set to 86400. Resetting to 0 >[0m[0m2017-10-18 12:17:11,745 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-2) gc_grace_seconds for data_compressed is set to 864000. Resetting back to zero. >[0m[0m2017-10-18 12:17:11,752 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-4) gc_grace_seconds for finished_jobs_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:11,875 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-3) gc_grace_seconds for jobs is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:11,985 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-2) gc_grace_seconds for leases is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,055 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-4) gc_grace_seconds for locks is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,160 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-3) gc_grace_seconds for metrics_expiration_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,240 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for metrics_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,287 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-2) gc_grace_seconds for metrics_tags_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,312 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-4) gc_grace_seconds for retentions_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,386 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-3) gc_grace_seconds for scheduled_jobs_idx is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,462 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for sys_config is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,503 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-2) gc_grace_seconds for tasks is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,545 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-4) gc_grace_seconds for tenants is set to 864000. Resetting to 0 >[0m[0m2017-10-18 12:17:12,662 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-2) Finished gc_grace_seconds updates in 1031 ms >[0m[33m2017-10-18 12:17:20,308 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (default task-18) Did not find meta data for timer GET {dual_path} that has been added to the registry >[0m[31m2017-10-18 12:17:37,863 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:17:38,864 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:17:39,865 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:17:40,866 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-3,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[33m2017-10-18 12:18:23,219 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (default task-31) Did not find meta data for timer GET gauges that has been added to the registry >[0m[33m2017-10-18 12:18:26,989 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (default task-38) Did not find meta data for timer GET counters that has been added to the registry >[0m[31m2017-10-18 12:18:32,635 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:18:33,636 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:18:34,637 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:18:35,637 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:19:39,162 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:19:40,163 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:19:41,164 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:19:42,166 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-3,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:20:44,087 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:20:45,087 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:20:46,089 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:20:47,090 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:21:40,390 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:21:41,390 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:21:42,393 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:21:43,394 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[33m2017-10-18 12:22:04,046 WARN [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-62) RESTEASY002142: Multiple resource methods match request "GET /". Selecting one. Matching methods: [public javax.ws.rs.core.Response org.hawkular.metrics.api.jaxrs.handler.BaseHandler.baseJSON(), public void org.hawkular.metrics.api.jaxrs.handler.BaseHandler.baseHTML(javax.servlet.ServletContext) throws java.lang.Exception] >[0m[33m2017-10-18 12:22:15,406 WARN [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-51) RESTEASY002142: Multiple resource methods match request "GET /". Selecting one. Matching methods: [public javax.ws.rs.core.Response org.hawkular.metrics.api.jaxrs.handler.BaseHandler.baseJSON(), public void org.hawkular.metrics.api.jaxrs.handler.BaseHandler.baseHTML(javax.servlet.ServletContext) throws java.lang.Exception] >[0m[33m2017-10-18 12:22:19,779 WARN [org.hawkular.metrics.core.dropwizard.HawkularMetricsRegistryListener] (default task-60) Did not find meta data for timer POST {dual_path}/stats/query that has been added to the registry >[0m[31m2017-10-18 12:22:28,597 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:22:29,597 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:22:30,598 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:22:31,599 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-3,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:23:16,876 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:23:17,876 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:23:18,877 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:23:19,878 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:24:06,078 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:24:07,079 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:24:08,079 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:24:09,080 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:25:12,698 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:25:13,699 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:25:14,699 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:25:15,700 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:26:01,292 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:26:02,293 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:26:03,293 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:26:04,294 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:26:56,797 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:26:57,798 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:26:58,798 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:26:59,799 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-3,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:27:42,250 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:27:43,251 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:27:44,258 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:27:45,259 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:28:25,665 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:28:26,665 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:28:27,666 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:28:28,667 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:29:32,523 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:29:33,523 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:29:34,524 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-2,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:29:35,525 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-2,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]] >[0m[31m2017-10-18 12:30:38,985 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:30:39,986 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[31m2017-10-18 12:30:40,987 SEVERE [org.openshift.ping.common.stream.TokenStreamProvider] (thread-3,ee,hawkular-metrics-n15zd) Could not create trust manager for /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:110) > at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339) > at org.openshift.ping.common.stream.TokenStreamProvider.configureCaCert(TokenStreamProvider.java:73) > at org.openshift.ping.common.stream.TokenStreamProvider.getSSLSocketFactory(TokenStreamProvider.java:106) > at org.openshift.ping.common.stream.TokenStreamProvider.openStream(TokenStreamProvider.java:49) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:25) > at org.openshift.ping.common.stream.OpenStream.call(OpenStream.java:7) > at org.openshift.ping.common.Utils.execute(Utils.java:210) > at org.openshift.ping.common.Utils.openStream(Utils.java:50) > at org.openshift.ping.kube.Client.getNode(Client.java:84) > at org.openshift.ping.kube.Client.getPods(Client.java:90) > at org.openshift.ping.kube.KubePing.doReadAll(KubePing.java:196) > at org.openshift.ping.common.OpenshiftPing.readAll(OpenshiftPing.java:249) > at org.openshift.ping.common.OpenshiftPing.sendMcastDiscoveryRequest(OpenshiftPing.java:201) > at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62) > at org.jgroups.protocols.PING.findMembers(PING.java:32) > at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244) > at org.jgroups.protocols.Discovery.down(Discovery.java:388) > at org.openshift.ping.common.OpenshiftPing.down(OpenshiftPing.java:196) > at org.jgroups.protocols.MERGE3$InfoSender.run(MERGE3.java:381) > at org.jgroups.util.TimeScheduler3$Task.run(TimeScheduler3.java:291) > at org.jgroups.util.TimeScheduler3$RecurringTask.run(TimeScheduler3.java:325) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at org.jboss.as.clustering.jgroups.ClassLoaderThreadFactory.lambda$newThread$0(ClassLoaderThreadFactory.java:52) > at java.lang.Thread.run(Thread.java:748) >Caused by: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- > at sun.security.provider.X509Factory.checkHeaderFooter(X509Factory.java:646) > at sun.security.provider.X509Factory.readOneBlock(X509Factory.java:636) > at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:96) > ... 25 more >[0m[33m2017-10-18 12:30:41,987 WARN [org.jgroups.protocols.openshift.KUBE_PING] (thread-3,ee,hawkular-metrics-n15zd) Problem getting Pod json from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.openshift.ping.common.stream.TokenStreamProvider@5e158d38] for cluster [ee], namespace [openshift-infra], labels [metrics-infra=hawkular-metrics,name=hawkular-metrics]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: java.security.cert.CertificateException: Could not parse certificate: java.io.IOException: Illegal header: -----BEGIN CERTIFICATE----- ]]
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1500471
: 1340164