Bug 997583 - a file-scanner deployment that fails on deploy will not be .failed after reboot when it should
a file-scanner deployment that fails on deploy will not be .failed after rebo...
Status: VERIFIED
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Domain Management (Show other bugs)
6.1.0
Unspecified Unspecified
unspecified Severity medium
: DR1
: EAP 6.4.0
Assigned To: Chao Wang
Petr Kremensky
Russell Dickenson
:
: 1119159 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-15 12:46 EDT by Brad Maxwell
Modified: 2016-01-06 20:19 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
JBoss Issue Tracker WFLY-1305 Major Resolved a file-scanner deployment that fails on boot will not be undeployed after failure 2016-10-27 10:20 EDT

  None (edit)
Description Brad Maxwell 2013-08-15 12:46:57 EDT
Description of problem:

Deploy via copying file into deployments directory, where it has an xml needing a resource that does not exist causes it to fail because of missing dependencies.  A .failed marker file is created.  Restart the server and the .failed becomes a .deployed even though it is really still failed and not accessible.

Version-Release number of selected component (if applicable):

JBoss EAP 6.1.0

How reproducible:


Steps to Reproduce:
1. deploy jboss-as-kitchensink-jsp.war which needs java:jboss/datasources/KitchensinkJSPQuickstartDS, which if you don't define will cause this:

JBAS014775: New missing/unsatisfied dependencies:
service jboss.naming.context.java.jboss.datasources.KitchensinkJSPQuickstartDS (missing) dependents: service jboss.persistenceunit."jboss-as-kitchensink-jsp.war#primary"

2. see that it creates a jboss-as-kitchensink-jsp.war.failed

3. Restart the JBoss server, it will then have jboss-as-kitchensink-jsp.war.deployed, however it still logs the missing dependencies and the war is not accessible because it is failed, though the marker file does not indicate this.

4. see that a .deployed marker file is created even though it is actually failed

Actual results:

After restart a .deployed marker file exists even though it is missing dependencies and is not accessible

Expected results:

After restart there is still a .failed marker file 

Additional info:

Looks to be the same issue as described in WFLY-1305

a file-scanner deployment that fails on boot will not be undeployed after failure
https://issues.jboss.org/browse/WFLY-1305
Comment 1 JBoss JIRA Server 2013-08-15 22:52:15 EDT
Brian Stansberry <brian.stansberry@redhat.com> made a comment on jira WFLY-1305

It's not undeployed because the management ops created by the scanner are inserted into the list of boot ops, and boot ops do not roll back on failure.

I'm not sure why the .deployed file is written instead of a .failed file, since even though the ops don't roll back, they still failed. Figuring that out will hopefully point to an overall solution.
Comment 2 JBoss JIRA Server 2013-08-15 23:05:57 EDT
Chao Wang <chaowan@redhat.com> made a comment on jira WFLY-1305

As example in attached file, one of the composite ops "add" in step-1 comes out "success", without roll-back the overall outcome is success.
Comment 3 JBoss JIRA Server 2013-08-16 00:06:18 EDT
Brian Stansberry <brian.stansberry@redhat.com> made a comment on jira WFLY-1305

I assume the "RESULT" section in the attached operation-outcome.xml is what is assigned to var 'result' at FileSystemDeploymentService L423.

So the problem is

"result" => {"step-1" => {
        "outcome" => "success"

The operation is actually a composite of composites. So that "step-1" is itself a composite. And it is being marked as "success" even though one of *it's* steps failed:

           "step-2" => {
                "outcome" => "failed",
                "failure-description" => {"JBAS014671: Failed services" => {"jboss.module.service.\"deployment.test.war\".main" => "org.jboss.msc.service.StartException in service jboss.module.service.\"deployment.test.war\".main: JBAS018759: Failed to load module: deployment.test.war:main
    Caused by: org.jboss.modules.ModuleNotFoundException: aaa.bbb:1.0"}}
            }

This happens because CompositeOperationHandler only marks the composite operation as "failed" if rollback occurs. I have a vague memory that that's by design; I'll need to do some research.

FileSystemDeploymentService can likely deal with this by

1) knowing it's executing during boot, which it could learn from DeploymentOperation (which would have to have a property added to reveal that.)

2) If so digging further into that RESULT to look for problems and not relying on the result of the composite. Perhaps it should do that always.

That will get the .failed marker correctly written. Analysis/testing also needs to be done to ensure that FileSystemDeploymentService will correctly handle the deployment being subsequently fixed; i.e. it needs to recognize that the deployment is still registered with the server and needs to be replaced, not added a second time.
Comment 4 JBoss JIRA Server 2013-08-16 13:45:13 EDT
Brian Stansberry <brian.stansberry@redhat.com> made a comment on jira WFLY-1305

"This happens because CompositeOperationHandler only marks the composite operation as "failed" if rollback occurs. I have a vague memory that that's by design; I'll need to do some research."

My vague memory was correct. The docs on this are quite clear (and I wrote them):

https://docs.jboss.org/author/display/AS72/Admin+Guide#AdminGuide-BasicCompositeOperationResponses

"The high level format of a basic composite operation response is largely the same as that of a simple operation response, although there is an important semantic difference. For a composite operation, the meaning of the outcome flag is controlled by the value of the operation request's rollback-on-runtime-failure header field. If that field was false (default is true), the outcome flag will be success if all steps were successfully applied to the persistent configuration even if none of the composite operation's steps was successfully applied to the runtime."

So, the RESULT in the attachment is per design. We'll have to figure out how to have FileSystemDeploymentService dig further into the result if rollback-on-runtime-failure will be set to 'false'. Or perhaps always. Should be solvable easily enough.
Comment 8 Chao Wang 2014-11-04 03:21:16 EST
*** Bug 1119159 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.