Bug 1411983 - Exception performing operations on MW Deployments in server deployments list
Summary: Exception performing operations on MW Deployments in server deployments list
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Providers
Version: 5.7.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: GA
: 5.7.1
Assignee: Mike Thompson
QA Contact: Sunil Kondkar
URL:
Whiteboard:
Depends On: 1404877
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-10 22:21 UTC by Satoe Imaishi
Modified: 2017-06-12 14:21 UTC (History)
11 users (show)

Fixed In Version: 5.7.1.0
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1404877
Environment:
Last Closed: 2017-06-12 14:21:08 UTC
Category: ---
Cloudforms Team: Middleware
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
screenshot (74.78 KB, image/png)
2017-02-15 14:40 UTC, Sunil Kondkar
no flags Details
MW deployment disabled (100.98 KB, image/png)
2017-06-06 01:07 UTC, Mike Thompson
no flags Details
MiQ Versions (70.76 KB, image/png)
2017-06-06 01:11 UTC, Mike Thompson
no flags Details

Comment 2 CFME Bot 2017-01-10 22:26:14 UTC
New commit detected on ManageIQ/manageiq/euwe:
https://github.com/ManageIQ/manageiq/commit/1d0f05420eab3fbae2f4e7eedd2a418011d88d93

commit 1d0f05420eab3fbae2f4e7eedd2a418011d88d93
Author:     Dan Clarizio <dclarizi>
AuthorDate: Tue Dec 13 15:58:54 2016 -0800
Commit:     Satoe Imaishi <simaishi>
CommitDate: Tue Jan 10 17:21:37 2017 -0500

    Merge pull request #12991 from ammendonca/MIQ-MW-FIX-ACTIONS-DEPLOYMENTS-LIST
    
    Enable deployment actions in server deployments list view
    (cherry picked from commit 4c8f892d6b05b192beca9c4d61ce0b4f2e29eccf)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1411983

 app/controllers/middleware_server_controller.rb | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Comment 5 Sunil Kondkar 2017-02-15 14:39:49 UTC
Tested on CFME Version 5.7.1.1.20170206165110_3c42361

Middlware Manager version-0.23.0.Final-redhat-1 

Tested as below:

1) Navigate to Middleware->Providers->Provider Summary page->Relationships.Middleware Servers->Middleware Server Summary page->Relationships.Middleware Deployments->Deployments list page

Any operation on the deployment like 'Restart' operation shows the message in UI - Button not yet implemented
Please refer the attached screenshot.

2) Navigate to Middleware->Servers->Middleware Server Summary page->Relationships.Middleware Deployments->Deployments list page
Any operation on the deployment like 'Restart' operation shows the message in UI - Button not yet implemented

3) Operations on deployments works successfully when user navigates to Middleware->Deployments->Deployments list page

Comment 6 Sunil Kondkar 2017-02-15 14:40:34 UTC
Created attachment 1250625 [details]
screenshot

Comment 16 Mike Thompson 2017-06-05 23:24:39 UTC
I was able to reproduce this when trying to disable a deployed war file:

16:10:45,361 INFO  [org.jboss.as.server] (OkHttp http://localhost:8080/hawkular/command-gateway/feed/4c3611d8-c0d7-45a3-853e-7503b9f5af2a) WFLYSRV0009: Undeployed "ticket-monster.war" (runtime-name: "ticket-monster.war")
16:12:14,000 ERROR [org.jboss.as.controller.management-operation] (Hawkular-WildFly-Agent-Scheduler-Metrics-2) WFLYCTL0013: Operation ("read-attribute") failed - address: ([
    ("deployment" => "ticket-monster.war"),
    ("subsystem" => "undertow")
]) - failure description: "WFLYCTL0216: Management resource '[
    (\"deployment\" => \"ticket-monster.war\"),
    (\"subsystem\" => \"undertow\")
]' not found"
16:12:14,001 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-2) HAWKMONITOR010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[null]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Unsuccessful fetching DMR attribute [active-sessions]
	at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:158)
	at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:447)
	at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
	at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: org.hawkular.dmr.api.OperationFailureException: Could not perform operation [read-attribute]: WFLYCTL0216: Management resource '[
    ("deployment" => "ticket-monster.war"),
    ("subsystem" => "undertow")
]' not found
	at org.hawkular.dmr.api.OperationBuilder$OperationResult.assertSuccess(OperationBuilder.java:422)
	at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:155)
	... 8 more

This looks more like a Hawkular agent / hawkular-client-ruby error. We to pull in other resources to look at this.

@Mazz can you help diagnose this DMR issue (or recommend someone)?

Comment 18 John Mazzitelli 2017-06-06 00:13:59 UTC
(In reply to Mike Thompson from comment #16)

I think these are "OK" and "to be expected". I'll explain inline below, but in short, you successfully undeployed a deployment, but afterwards the agent was attempting to collect a deployment metric (which now fails, since the deployment is gone).


> 16:10:45,361 INFO  [org.jboss.as.server] (OkHttp
> http://localhost:8080/hawkular/command-gateway/feed/4c3611d8-c0d7-45a3-853e-
> 7503b9f5af2a) WFLYSRV0009: Undeployed "ticket-monster.war" (runtime-name:
> "ticket-monster.war")


Notice INFO level above. This is a success - the deployment "ticket-monster.war" has been successfully undeployed.


> 16:12:14,000 ERROR [org.jboss.as.controller.management-operation]
> (Hawkular-WildFly-Agent-Scheduler-Metrics-2) WFLYCTL0013: Operation
> ("read-attribute") failed - address: ([
>     ("deployment" => "ticket-monster.war"),
>     ("subsystem" => "undertow")
> ]) - failure description: "WFLYCTL0216: Management resource '[
>     (\"deployment\" => \"ticket-monster.war\"),
>     (\"subsystem\" => \"undertow\")
> ]' not found"


Notice now (1.5 minutes later) the ticket-monster.war is missing - which is correct because it has been undeployed. Why is this error spit out, see below...


> 16:12:14,001 ERROR [org.hawkular.agent.monitor.protocol.EndpointService]
> (Hawkular-WildFly-Agent-Scheduler-Metrics-2) HAWKMONITOR010049: Could not
> access resources of endpoint [DMREndpointService[Endpoint[Local]:[null]]]:
> org.hawkular.agent.monitor.protocol.ProtocolException: Unsuccessful fetching
> DMR attribute [active-sessions]
> 	at
> org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.
> java:158)
> 	at
> org.hawkular.agent.monitor.protocol.EndpointService.
> measureMetrics(EndpointService.java:447)


The agent was attempting to collect the "active-sessions" metric for this deployment resource - but it is now gone so that resource can no longer be accessed. The metric collection failed. This is to be expected since you undeployed the resource.

The question is: why is the agent trying to collect the metrics 1.5 minutes after the undeployment happened? This is a problem. But it isn't a major issue - just you'll see these ERROR messages. This could warrant a HWKAGENT JIRA to find out why an undeploy does not unschedule metric collections for the undeployed resource and correct the algorithm.

Comment 19 Mike Thompson 2017-06-06 01:02:47 UTC
And it did complete and disable as shown by the attachement

Comment 20 Mike Thompson 2017-06-06 01:07:13 UTC
Created attachment 1285183 [details]
MW deployment disabled

So it did go through.

Comment 21 Mike Thompson 2017-06-06 01:11:18 UTC
Created attachment 1285184 [details]
MiQ Versions


Note You need to log in before you can comment on or make changes to this bug.