Attachment 936110 Details for Bug 1108168 – server.log

server.log

server.log (text/plain), 2.91 KB, created by Armine Hovsepyan on 2014-09-10 11:38:16 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Armine Hovsepyan

Created: 2014-09-10 11:38:16 UTC

Size: 2.91 KB

patch

obsolete

>.redhat.com, cqlPort=9142, operationMode=ADD_MAINTENANCE, mtime=1410348365730]
>07:32:40,622 INFO  [org.rhq.enterprise.server.storage.StorageNodeOperationsHandlerBean] (EJB default - 9) Running addNodeMaintenance for storage node StorageNode[id=1001, address=bz1108168.bc.jonqe.lab.eng.bos.redhat.com, cqlPort=9142, operationMode=NORMAL, mtime=1410347746623]
>07:33:00,370 INFO  [org.rhq.enterprise.server.cloud.instance.CacheConsistencyManagerBean] (EJB default - 3) bz1108168.bc.jonqe.lab.eng.bos.redhat.com took [45]ms to reload cache for 1 agents
>07:33:27,996 INFO  [org.rhq.server.metrics.StorageSession] (New I/O worker #9) Changing request throughput from 65000.0 request/sec to 35000.0 requests/sec
>07:33:27,997 INFO  [org.rhq.enterprise.server.storage.StorageClusterMonitor] (New I/O worker #9) Storage node at 10.16.23.202 is down
>07:33:28,025 WARN  [org.rhq.server.metrics.StorageSession] (Cassandra Java Driver worker-8) Encountered NoHostAvailableException due to following error(s): {bz1108168.bc.jonqe.lab.eng.bos.redhat.com/10.16.23.202=[bz1108168.bc.jonqe.lab.eng.bos.redhat.com/10.16.23.202] Channel has been closed}
>07:33:28,025 INFO  [org.rhq.enterprise.server.storage.StorageClusterMonitor] (Cassandra Java Driver worker-8) Storage cluster is down
>07:33:30,288 INFO  [org.rhq.enterprise.server.cloud.instance.ServerManagerBean] (EJB default - 9) Notified communication layer of server operation mode MAINTENANCE
>07:33:39,981 INFO  [org.rhq.server.metrics.StorageSession] (Hashed wheel timer #3) Changing request throughput from 35000.0 request/sec to 5000.0 requests/sec
>07:33:39,982 INFO  [org.rhq.enterprise.server.storage.StorageClusterMonitor] (Hashed wheel timer #3) Storage node at 10.16.23.205 is down
>07:33:39,985 ERROR [com.datastax.driver.core.ControlConnection] (Cassandra Java Driver blocking tasks worker-1) [Control connection] Cannot connect to any host, scheduling retry
>07:33:40,986 ERROR [com.datastax.driver.core.ControlConnection] (Reconnection-1) [Control connection] Cannot connect to any host, scheduling retry in 2000 milliseconds
>07:33:46,136 INFO  [org.rhq.server.metrics.StorageSession] (Cassandra Java Driver worker-7) /10.16.23.205 is up
>07:33:46,136 INFO  [org.rhq.server.metrics.StorageSession] (Cassandra Java Driver worker-7) Changing request throughput from 5000.0 request/sec to 35000.0 requests/sec
>07:33:46,136 INFO  [org.rhq.enterprise.server.storage.StorageClusterMonitor] (Cassandra Java Driver worker-7) Storage cluster is up
>07:33:46,137 INFO  [org.rhq.enterprise.server.storage.StorageClusterMonitor] (Cassandra Java Driver worker-7) Storage node at 10.16.23.205 is up
>07:34:00,285 INFO  [org.rhq.enterprise.server.cloud.instance.ServerManagerBean] (EJB default - 7) Removed 2 obsolete agent reference(s) to server bz1108168.bc.jonqe.lab.eng.bos.redhat.com
>07:34:00,287 INFO  [org.rhq.enterprise.server.cloud.instance.ServerManagerBean] (EJB default - 7) Notified communication layer of server operation mode NORMAL
>

Actions: View

Attachments on bug 1108168: 907679 | 936110 | 936111 | 936113 | 936116 | 936117 | 936118 | 944490