Bug 1107336

Summary: AntBundlePluginComponent.deployBundle fails if there are more then 200 files named by rhq:file or rhq:archive elements
Product: [JBoss] JBoss Operations Network Reporter: Larry O'Leary <loleary>
Component: ProvisioningAssignee: RHQ Project Maintainer <rhq-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Sunil Kondkar <skondkar>
Severity: high Docs Contact:
Priority: unspecified    
Version: JON 3.1.2CC: myarboro, skondkar
Target Milestone: ER01Keywords: TestCaseNeeded, Triaged
Target Release: JON 3.2.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-09-05 15:40:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Larry O'Leary 2014-06-09 23:48:47 UTC
Description of problem:
If a bundle contains over 200 files, if fails to deploy and the agent reports file not found exceptions:

    ERROR [ResourceContainer.invoker.nonDaemon-1] (org.rhq.plugins.ant.AntBundlePluginComponent)- Failed to deploy bundle [class org.rhq.core.pluginapi.bundle.BundleDeployRequest: deployment=[BundleResourceDeployment: bdd=[BundleDeployment[id=10001, name=Deployment [1] of Version [1.0] to [Test-01]]], resource=[Resource[id=10001, uuid=c8a07c0c-593d-437c-abc8-f370c739ff77, type={Platforms}Linux, key=Agent 01091768 on jboss-01.example.com, name=Agent 01091768 on jboss-01.example.com, parent=<null>, version=Linux 3.14.4-200.fc20.x86_64]]], full-deploy-directory=[/tmp/Test-01], clean=[false], revert=[false]]
    java.lang.Exception: Failed to execute the bundle Ant script
        at org.rhq.plugins.ant.AntBundlePluginComponent.deployBundle(AntBundlePluginComponent.java:159)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:634)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
    Caused by: java.lang.RuntimeException: Failed to execute bundle deploy file [/opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/ant-bundle-recipe7078114028841300667.xml]. Cause: /opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/ant-bundle-recipe7078114028841300667.xml:9: Failed to deploy bundle [Many Files Test Bundle] version [1.0]: java.io.FileNotFoundException: /opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/files/file-104.txt (No such file or directory)
        at org.rhq.bundle.ant.AntLauncher.executeBundleDeployFile(AntLauncher.java:118)
        at org.rhq.plugins.ant.AntBundlePluginComponent.executeDeploymentPhase(AntBundlePluginComponent.java:306)
        at org.rhq.plugins.ant.AntBundlePluginComponent.deployBundle(AntBundlePluginComponent.java:139)
        ... 9 more
    Caused by: /opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/ant-bundle-recipe7078114028841300667.xml:9: Failed to deploy bundle [Many Files Test Bundle] version [1.0]: java.io.FileNotFoundException: /opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/files/file-104.txt (No such file or directory)
        at org.rhq.bundle.ant.type.DeploymentUnitType.install(DeploymentUnitType.java:218)
        at org.rhq.bundle.ant.task.BundleTask.execute(BundleTask.java:148)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:390)
        at org.apache.tools.ant.helper.ProjectHelper2.parse(ProjectHelper2.java:179)
        at org.apache.tools.ant.ProjectHelper.configureProject(ProjectHelper.java:80)
        at org.rhq.bundle.ant.AntLauncher.executeBundleDeployFile(AntLauncher.java:111)
        ... 11 more
    Caused by: java.io.FileNotFoundException: /opt/jboss/jboss-on//rhq-agent/data/tmp/bundle-versions/10001/files/file-104.txt (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)
        at org.rhq.core.util.updater.Deployer.extractZipAndRawFiles(Deployer.java:838)
        at org.rhq.core.util.updater.Deployer.performInitialDeployment(Deployer.java:340)
        at org.rhq.core.util.updater.Deployer.deploy(Deployer.java:181)
        at org.rhq.bundle.ant.type.DeploymentUnitType.install(DeploymentUnitType.java:202)
        ... 23 more


Version-Release number of selected component (if applicable):
3.1.2

How reproducible:
Always

Steps to Reproduce:
1.  Install and start JBoss ON 3.1.2 system.
2.  Import platform resource.
3.  Create new compatible resource group called _All Platforms_:

    *   *Name*: `All Platforms`
    *   *Select Members*: _add all resources from platform category_
    
4.  Create jon-31-bundle-manyFiles bundle archive containing 300 files:

        mkdir -p jon-31-bundle-manyFiles
        pushd jon-31-bundle-manyFiles
        mkdir -p files

        cat >deploy.xml <<EOF
<?xml version="1.0"?>
<project name="manyFiles"
         default="main"
         xmlns:rhq="antlib:org.rhq.bundle">
  <target name="main" />

  <rhq:bundle name="Many Files Test Bundle"
              version="1.0"
              description="A sample bundle consisting of 300 files">
    <rhq:deployment-unit name="files"
                         manageRootDir="false">
EOF

        for file in {001..300}; do
            echo "This is File #${file}." >"files/file-${file}.txt"
            echo '      <rhq:file name="files/file-'"${file}"'.txt" destinationFile="files/file-'"${file}"'.txt" replace="true" />' >>deploy.xml
        done

        cat >>deploy.xml <<EOF
    </rhq:deployment-unit>
  </rhq:bundle>
</project>
EOF

        zip -qr ../jon-31-bundle-manyFiles.zip .
        popd

5.  From JBoss ON UI, create bundle using jon-31-bundle-manyFiles bundle archive.
6.  Deploy _Many Files Test Bundle_ to destination _Test-01_:

    *Destination Name*: `Test-01`
    *Resource Group*: _All Platforms_
    *Deployment Directory*: `/tmp/Test-01`
    *Deploy Options*: _Latest Version_

Actual results:
Deployment reported failure due to java.io.FileNotFoundException and agent log contains: ERROR [ResourceContainer.invoker.nonDaemon-1] (org.rhq.plugins.ant.AntBundlePluginComponent)- Failed to deploy bundle [class org.rhq.core.pluginapi.bundle.BundleDeployRequest: deployment=[BundleResourceDeployment: bdd=[BundleDeployment[id=10001, name=Deployment [1] of Version [1.0] to [Test-01]]], resource=[Resource[id=10001, uuid=c8a07c0c-593d-437c-abc8-f370c739ff77, type={Platforms}Linux, key=Agent 01091768 on jboss-01.example.com, name=Agent 01091768 on jboss-01.example.com, parent=<null>, version=Linux 3.14.4-200.fc20.x86_64]]], full-deploy-directory=[/tmp/Test-01], clean=[false], revert=[false]]

Expected results:
No error and 300 files contained in /tmp/Test-01/files directory.

Additional info:
This is most likely caused by paging. When the the bundle file list is obtained using a search query, no paging is being used and therefore the default of 200 is applied.

Comment 1 Larry O'Leary 2014-06-10 01:44:50 UTC
This issue was resolved in JBoss ON 3.2 but we may still need a test case and QE verification to validate.

Comment 2 Simeon Pinder 2014-08-15 03:19:12 UTC
Moving to ON_QA as this is available for test in JON 3.2.3 ER01 build:

http://jon01.mw.lab.eng.bos.redhat.com:8042/dist/release/jon/3.2.3.GA/8-14-14/

Comment 3 Sunil Kondkar 2014-08-21 09:07:29 UTC
Verified on version : 3.2.0.GA Update 03 build number :	bca1bc8:e19c43d

Followed the steps and verified that the bundle deployment is successful without errors. The files are deployed to /tmp/Test-01/files directory.

Added the test case.
Test run:
https://tcms.engineering.redhat.com/run/168756/