Bug 826235

Summary: Deployment of bundle fails because of corrupted war archive
Product: [JBoss] JBoss Operations Network Reporter: Jochen Cordes <jcordes>
Component: CLIAssignee: John Mazzitelli <mazz>
Status: CLOSED NOTABUG QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: unspecified    
Version: JON 3.1.0CC: mazz
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-05-30 18:41:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
The deployment bundle
none
JON script library
none
Bundle creation script
none
Bundle deployment script
none
Bundle creation shell script
none
Bundle deployment shell script
none
Another bundle example none

Description Jochen Cordes 2012-05-29 20:17:23 UTC
Created attachment 587502 [details]
The deployment bundle

Description of problem:

When trying to deploy a bundle via JON-CLI 8with JON 3.1.0 Beta 1 I get an error on JBoss EAP 5.1.2 (AS5 plugin version is 4.4.0JON310Beta1)

Version-Release number of selected component (if applicable):
RHQ Remoting 4.4.0JON310Beta
JON Server 3.1.0 Beta 1
JBoss AS5 Plugin 4.4.0JON310Beta1
JBoss EAP 5.1.2

How reproducible:
100%

Steps to Reproduce:
1. Create a deployment bundle via JON-CLI

./create-bundle.sh /opt/rhq-remoting-cli-4.4.0.JON310BETA1/scripts/shoutout-bundle.zip "Shoutout_bundle_test" "1.0"

2. Deploy the bundle via JON-CLI 

/deploy-bundle.sh "Shoutout_Application" "shoutout-main" "shoutout-main" "The_Shoutout_Application" "deploy" "1.0" "Deployment_of_Shoutout"
  
Actual results:

An exception is thrown in the server logs:

2012-05-29 21:56:20,756 INFO  [org.jboss.resource.connectionmanager.ConnectionFactoryBindingService] (HDScanner) Bound ConnectionManager 'jboss.jca:service=DataSourceBinding,name=ShoutoutDS' to JNDI name 'java:ShoutoutDS'
2012-05-29 21:56:30,769 WARN  [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Failed to add deployment: vfszip:/opt/jboss-eap-5.1/jboss-as/server/default/deploy/shoutout-0.0.1-SNAPSHOT.war/
org.jboss.deployers.spi.DeploymentException: Error determining structure: shoutout-0.0.1-SNAPSHOT.war
        at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(DeploymentException.java:49)
        at org.jboss.deployers.vfs.plugins.structure.explicit.DeclaredStructure.determineStructure(DeclaredStructure.java:89)
        at org.jboss.deployers.vfs.plugins.structure.StructureDeployerWrapper.determineStructure(StructureDeployerWrapper.java:73)
        at org.jboss.deployers.vfs.plugins.structure.VFSStructuralDeployersImpl.doDetermineStructure(VFSStructuralDeployersImpl.java:196)
        at org.jboss.deployers.vfs.plugins.structure.VFSStructuralDeployersImpl.determineStructure(VFSStructuralDeployersImpl.java:221)
        at org.jboss.deployers.structure.spi.helpers.AbstractStructuralDeployers.determineStructure(AbstractStructuralDeployers.java:77)
        at org.jboss.deployers.plugins.main.MainDeployerImpl.determineStructure(MainDeployerImpl.java:1001)
        at org.jboss.deployers.plugins.main.MainDeployerImpl.determineDeploymentContext(MainDeployerImpl.java:437)
        at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:387)
        at org.jboss.deployers.plugins.main.MainDeployerImpl.addDeployment(MainDeployerImpl.java:297)
        at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.addDeployment(MainDeployerAdapter.java:86)
        at org.jboss.system.server.profileservice.hotdeploy.HDScanner.scan(HDScanner.java:385)
        at org.jboss.system.server.profileservice.hotdeploy.HDScanner.run(HDScanner.java:294)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.RuntimeException: java.util.zip.ZipException: error in opening zip file
        at org.jboss.virtual.plugins.context.AbstractExceptionHandler.handleZipEntriesInitException(AbstractExceptionHandler.java:39)
        at org.jboss.virtual.plugins.context.helpers.NamesExceptionHandler.handleZipEntriesInitException(NamesExceptionHandler.java:63)
        at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:636)
        at org.jboss.virtual.plugins.context.zip.ZipEntryContext.checkIfModified(ZipEntryContext.java:786)
        at org.jboss.virtual.plugins.context.zip.ZipEntryContext.getChild(ZipEntryContext.java:830)
        at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.createChildHandler(ZipEntryHandler.java:195)
        at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.structuredFindChild(AbstractVirtualFileHandler.java:690)
        at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.getChild(ZipEntryHandler.java:169)
        at org.jboss.virtual.plugins.context.DelegatingHandler.getChild(DelegatingHandler.java:107)
        at org.jboss.virtual.VirtualFile.getChild(VirtualFile.java:492)
        at org.jboss.deployers.vfs.plugins.structure.explicit.DeclaredStructure.determineStructure(DeclaredStructure.java:64)
        ... 19 more
Caused by: java.util.zip.ZipException: error in opening zip file
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.<init>(ZipFile.java:131)
        at java.util.jar.JarFile.<init>(JarFile.java:150)
        at java.util.jar.JarFile.<init>(JarFile.java:114)
        at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.ensureZipFile(ZipFileWrapper.java:173)
        at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.acquire(ZipFileWrapper.java:250)
        at org.jboss.virtual.plugins.context.zip.ZipEntryContext.initEntries(ZipEntryContext.java:485)
        at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:629)
        ... 27 more 

Expected results:


Additional info:

Comment 1 Jochen Cordes 2012-05-29 20:18:35 UTC
Created attachment 587503 [details]
JON script library

Comment 2 Jochen Cordes 2012-05-29 20:19:30 UTC
Created attachment 587504 [details]
Bundle creation script

Comment 3 Jochen Cordes 2012-05-29 20:20:06 UTC
Created attachment 587505 [details]
Bundle deployment script

Comment 4 Jochen Cordes 2012-05-29 20:20:50 UTC
Created attachment 587506 [details]
Bundle creation shell script

Comment 5 Jochen Cordes 2012-05-29 20:21:27 UTC
Created attachment 587507 [details]
Bundle deployment shell script

Comment 6 John Mazzitelli 2012-05-29 20:38:00 UTC
could this be related to bug 751506 ?

Comment 7 John Mazzitelli 2012-05-29 20:43:22 UTC
Jochen - have you tried to take your shoutout-0.0.1-SNAPSHOT.war and manually deploy it directly in your EAP (i.e. don't use JON to deploy it).

I just want to make sure the actual .war itself is valid and can be deployed by EAP outside of JON.

I would also try to deploy this using the GUI (as opposed to the CLI) to make sure the CLI upload didn't corrupt the war some how.

Comment 8 Jochen Cordes 2012-05-29 21:23:17 UTC
(In reply to comment #7)
> Jochen - have you tried to take your shoutout-0.0.1-SNAPSHOT.war and
> manually deploy it directly in your EAP (i.e. don't use JON to deploy it).
> 
> I just want to make sure the actual .war itself is valid and can be deployed
> by EAP outside of JON.
Actually deploying the war as such is not successful as it is an EE6 application:

2012-05-29 23:14:04,424 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd
2012-05-29 23:14:11,836 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd
2012-05-29 23:14:13,123 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/javaee/web-common_3_0.xsd
2012-05-29 23:14:14,901 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/javaee/javaee_6.xsd
2012-05-29 23:14:17,740 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/javaee/javaee_web_services_client_1_3.xsd
2012-05-29 23:14:18,701 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file U
RL: http://java.sun.com/xml/ns/javaee/jsp_2_2.xsd
2012-05-29 23:14:19,276 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanne
r) Trying to resolve systemId as a non-file URL: http://java.sun.com/xml/ns/javaee/javaee_6.xsd
2012-05-29 23:14:19,499 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file URL: http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd
2012-05-29 23:14:24,330 WARN  [org.jboss.util.xml.JBossEntityResolver] (HDScanner) Trying to resolve systemId as a non-file URL: http://java.sun.com/xml/ns/javaee
2012-05-29 23:14:54,488 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] (HDScanner) Error installing to Parse: name=vfszip:/opt/jboss-eap-5.1/jboss-as/server/default/deploy/shoutout-0.0.1-SNAPSHOT.war/ state=PreParse mode=Manual requiredState=Parse
org.jboss.deployers.spi.DeploymentException: Error creating managed object for vfszip:/opt/jboss-eap-5.1/jboss-as/server/default/deploy/shoutout-0.0.1-SNAPSHOT.war/

But this is a different error. I will try to find an application that can be deployed without any issues in EAP 5.1.2 and then go for the bundle. Maybe this was a bad example, but nevertheless I should have gotten to this stage via JON.

> 
> I would also try to deploy this using the GUI (as opposed to the CLI) to
> make sure the CLI upload didn't corrupt the war some how.

I have the same issue when deploying via the GUI.

Comment 9 Jochen Cordes 2012-05-29 21:38:05 UTC
(In reply to comment #8)
> (In reply to comment #7)
> > Jochen - have you tried to take your shoutout-0.0.1-SNAPSHOT.war and
> > manually deploy it directly in your EAP (i.e. don't use JON to deploy it).
> > 
> > I just want to make sure the actual .war itself is valid and can be deployed
> > by EAP outside of JON.
> Actually deploying the war as such is not successful as it is an EE6
> application:
> 
> 2012-05-29 23:14:04,424 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd
> 2012-05-29 23:14:11,836 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd
> 2012-05-29 23:14:13,123 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/javaee/web-common_3_0.xsd
> 2012-05-29 23:14:14,901 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/javaee/javaee_6.xsd
> 2012-05-29 23:14:17,740 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/javaee/javaee_web_services_client_1_3.xsd
> 2012-05-29 23:14:18,701 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file U
> RL: http://java.sun.com/xml/ns/javaee/jsp_2_2.xsd
> 2012-05-29 23:14:19,276 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanne
> r) Trying to resolve systemId as a non-file URL:
> http://java.sun.com/xml/ns/javaee/javaee_6.xsd
> 2012-05-29 23:14:19,499 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file URL:
> http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd
> 2012-05-29 23:14:24,330 WARN  [org.jboss.util.xml.JBossEntityResolver]
> (HDScanner) Trying to resolve systemId as a non-file URL:
> http://java.sun.com/xml/ns/javaee
> 2012-05-29 23:14:54,488 ERROR
> [org.jboss.kernel.plugins.dependency.AbstractKernelController] (HDScanner)
> Error installing to Parse:
> name=vfszip:/opt/jboss-eap-5.1/jboss-as/server/default/deploy/shoutout-0.0.1-
> SNAPSHOT.war/ state=PreParse mode=Manual requiredState=Parse
> org.jboss.deployers.spi.DeploymentException: Error creating managed object
> for
> vfszip:/opt/jboss-eap-5.1/jboss-as/server/default/deploy/shoutout-0.0.1-
> SNAPSHOT.war/
> 
> But this is a different error. I will try to find an application that can be
> deployed without any issues in EAP 5.1.2 and then go for the bundle. Maybe
> this was a bad example, but nevertheless I should have gotten to this stage
> via JON.
Now tried with another appliction that deploys fine on it's own. It's still failing when trying to deploy via JON-CLI. I have attached the new application to the bug.
> 
> > 
> > I would also try to deploy this using the GUI (as opposed to the CLI) to
> > make sure the CLI upload didn't corrupt the war some how.
> 
> I have the same issue when deploying via the GUI.

Comment 10 Jochen Cordes 2012-05-29 21:39:41 UTC
Created attachment 587536 [details]
Another bundle example

Another example for a bundle that fails to deploy (but deploys just fine on it's own).

Comment 11 John Mazzitelli 2012-05-30 17:39:34 UTC
I think I know what the problem is. It appears to be due to the incorrect usage of the rhq:file in conjunction with replace="true" when the file is a binary file (as opposed to a simple text file).

Specifically, I believe the culprit is this:

   <rhq:file name="Calendar.war" destinationFile="${rhq.deploy.dir}/Calendar.war" replace="true"/>

Notice what this is asking here. First, this is using the rhq:file tag - so you are saying you want to copy the raw file "Calendar.war" to the destination location byte-for-byte.

But now notice that replace="true" - so you are asking it to go through that raw file and replace all replacement variables (@@ tokens) with their real values.

If, in the war file (which is not a raw text file, its a binary jar file) you have templatized files with @@ in them (or the binary data just happens to have a @@x@@ string), those tokens can be seen by a byte-by-byte search in the war binary itself. So the replace="true" is changing the actual war binary data, and thus causing corruption to that war (it can no longer be unpacked by Java because it has been changed/corrupted and is no longer a valid war file).

So, I think this is just a misuse of rhq:file in conjunction with its replace attribute set to true.

Here is how you should fix this.

First, when you have binary archives (like zip, jar, war, ear, etc), you should use rhq:archive, not rhq:file, in your RHQ bundle recipe. When you use rhq:archive, you can then use the rhq:replace child element if that binary archive has templatized files in them.

Note that You CAN use rhq:file for these binary files, but ONLY if you do NOT also set replace="true" (for the reasons stated above; you will hit this problem you are having if you do so).

Here are some sections of the documentation you should read to understand the rhq:archive, rhq:replace, rhq:file elements and rhq:file's replace attribute:

rhq:archive:
   http://rhq-project.org/display/JOPR2/Ant+Bundles#AntBundles-rhq%3Aarchive

rhq:replace:
   http://rhq-project.org/display/JOPR2/Ant+Bundles#AntBundles-rhq%3Areplace

rhq:file:
   http://rhq-project.org/display/JOPR2/Ant+Bundles#AntBundles-rhq%3Afile

A small section on templatizing files:
   http://rhq-project.org/display/JOPR2/Provisioning#Provisioning-TemplatizingFiles

Jochen, let me know if this solves your problem - I think this is the cause. If so, I'll close this as NOTABUG.

Comment 12 Jochen Cordes 2012-05-30 18:37:16 UTC
Hi John!

I have now played it save and have used rhq:archive instead of rhq:file and it is working now.

Thanks a lot for pointing this out! Please close as NOTABUG (but rather a user error).