Bug 738088

Summary: rhq-input boolean propery breaks Bundle system deployment functionality completely
Product: [Other] RHQ Project Reporter: Nabeel Saad <nsaad>
Component: ContentAssignee: Deon Ballard <dlackey>
Status: CLOSED CURRENTRELEASE QA Contact: Mike Foley <mfoley>
Severity: high Docs Contact:
Priority: high    
Version: unspecifiedCC: gcooper, hrupp, mazz
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-09-28 21:26:02 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 625146, 734807    
Attachments:
Description Flags
The deploy.xml that will succeed in deploying
none
The deploy.xml that will fail in deploying none

Description Nabeel Saad 2011-09-13 20:56:02 UTC
Created attachment 523007 [details]
The deploy.xml that will succeed in deploying

Description of problem:
I have found that adding an rhq:input-property of type="boolean" tends to break the bundle system.  You can upload the recipe/bundle and the new version will be created, but after that, any attempt to deploy that bundle or any previously functional bundles, all of them fail with a very un-informative error:
  java.lang.Exception:Failed to execute the bundle Ant script -> java.lang.NullPointerException:null

Version-Release number of selected component (if applicable):
JON 2.4.1 with all recent patches

How reproducible:
Everytime

Steps to Reproduce:
1. Create a directory called bundle-test
2. Download EAP 5.1.1 zip to the bundle-test directory ensuring it's named jboss-eap-5.1.1.zip (you can delete all the server instances leaving default if you want to make the zip smaller)
3. Use the attached deploy_success.xml recipe file renaming it into deploy.xml
4. Zip the jboss-eap-5.1.1.zip and the deploy.xml files into a single ZIP - for example, eap-bundle.zip
5. Log into JON, go to Groups -> New Group
   a. Name it "test-group"
   b. Choose "Contains" -> Compatible Resources with "Linux" as Resource Type
   c. Once the Group definition opens, click "Add to Group" under Group Members
   d. Add one of the Linux resources to the group
6. Go to Administration -> Content -> Bundles
7. Create a New bundle, using the "Upload" bundle option.
8. Click into the bundle, click Deploy.
9. Create a new destination
   a. Name = test-destination
   b. Root deployment directory = /opt/example
   c. Group = test-group
10. Input properties
   a. instance = default
   b. port-set = 03
   c. jmx-admin-password (leave default)

   *. [when dealing with the fail scenario]
      start-server (leave it at option no)

11. Click Next through till the end of the wizard.
12. Click into the Deployment [1] of Version [2.0] of [test-destination]
13. Once the deployment completes, you should see "Success" status
14. Repeat all the steps from (3) using deploy_failure.xml as deploy.xml up till step (12)
15. Once the deployment completes, you should see a "Failure" status
16. If you expand it and click on the error details, you'll see the error:
    	
java.lang.Exception:Failed to execute the bundle Ant script -> java.lang.NullPointerException:null

17. If you attempt to re-deploy the "EAP 5.1.1-vanilla-success" bundle, it will fail with the same error in step (16)
  
Actual results:
EAP 5.1.1-vanilla-success bundle succeeds
EAP 5.1.1-vanilla-failure bundle fails with "Failed to execute the bundle Ant script" due to a NullPointerException
Attempting to redeploy EAP 5.1.1-vanilla-success also fails and will continue to fail even after deleting the "EAP 5.1.1-vanilla-failure" bundle


Expected results:
Successful deployment

Additional info:
The only way I've found to have the bundles work again is to actually completely remove my JON instance (clearly, there must be an easier way to do this), and never uploading the bundle with the boolean type again.

Comment 1 Nabeel Saad 2011-09-13 20:56:44 UTC
Created attachment 523008 [details]
The deploy.xml that will fail in deploying

Comment 2 Charles Crouch 2011-09-26 13:58:40 UTC
Nabeel, is the following suitable content to reproduce the issue with: 
http://download.devel.redhat.com/released/JBEAP-5/5.1.1/zip/jboss-eap-5.1.1.zip

Comment 3 Nabeel Saad 2011-09-27 22:55:25 UTC
Hello Charles, yes that EAP zip you provided should work fine for testing this issue.  For simplicity sake, I would recommend deleting all the profiles other then default (to just decrease the size of the zip).

Also, you would need to put the above zip and the provided deploy.xml into another zip, say bundle.zip and then use that as your bundle.

Let me know if you have any questions.

Comment 4 John Mazzitelli 2011-09-28 18:58:05 UTC
there must be something else going on. I just deployed successfully a simple bundle that had a boolean property.

can you delete the agent logs directory, then restart the agent, put it in debug mode (execute prompt command "debug -e") and then redeploy your bundle. Once you see you have the error, can you attach the new agent log file that produced.

Comment 5 John Mazzitelli 2011-09-28 19:12:25 UTC
I just successfully used the the deploy-failure.xml. What version of RHQ are you using? I will try to do the full replication steps - I will do the successfully one first, then the failure to follow exactly the replication steps listed. But so far, I can't break this.

I am on the latest master build though - perhaps this was an unknown bug that has since been unknowingly fixed?

Comment 6 John Mazzitelli 2011-09-28 21:26:02 UTC
OK, I just tried this on the latest codebase and things work OK. No errors. I suspect this BZ was written against a previous version. The latest codebase seems to work OK.

There is one thing I want to say though, now that I went through this step by step.

The idea with bundles is to not have multiple bundle destinations use the same base directory. Otherwise, you'll be installing different bundles to the same locations and that could cause problems (i.e. you'd be overwriting a previously installed bundle with some other different bundle). This is alluded to here:

http://rhq-project.org/display/JOPR2/Provisioning#Provisioning-Introduction

when it defines "destination" as: "...only ever associated with one bundle (for example, a destination for bundle "Banking App" can only have versions of the Banking App bundle deployed to it)."

For example, the thinking is that you have bundle "App Server" and bundle "My Other Thing". Each bundle can have multiple bundle *versions*, like:

App Server v1.0
App Server v1.5
App Server v2.0
My Other Thing v1.0
My Other Thing v3.5

Bundle destinations are on a PER BUNDLE basis. So, you could have a destination for bundle App Server that is:

"My App Server Destination" (whose base directory is /opt/example/app)

and for My Other Thing bundle you have

"My Other Thing Destination" (whose base directory is /opt/example/other)

Now when I install any version of App Server, it goes in /opt/example/app on my group's machines. When I install any version of My Other Thing, it goes in a DIFFERENT directory called /opt/example/other on potentially the same group of machines.

Notice that your attached deploy.xml files have different bundle names (<bundle name=...) thus are different bundles (analogous to the App Server and My Other Thing examples above). But yet in the replication procedures, your two different bundle destinations used the SAME base directory. So it would be like you are asking to install App Server and My Other Thing in the same location. This is usually not what you want - you'd be overwriting one with the other. So, in this case, I wouldn't expect things to work properly anyway.

But that said, I was still able to overlay the two test bundles successfully, no NullPointerExceptions. Of course, one just overwrote the other, but no errors occurred. I suspect this was a problem that has since been fixed in master and will be in the next release.

I am putting this in CLOSED/CURRENTRELEASE status since it looks like the use-case presented in the replication procedures is not meant to be how bundles should actually be used, but nevertheless, it looks like the actual error condition is fixed already in the master branch.

Comment 7 Nabeel Saad 2011-09-29 08:39:07 UTC
Hello John,

Thanks for all your comments.  So here's a couple answers/comments to your questions:

-- I was using version JON 2.4.1.  This definitely breaks in 2.4
-- I did not try a "simpler" version of my package, but I did remove everything I had in the deploy.xml file, and adding things in one by one, got me to finding that it was the rhq:input type="boolean" that was breaking things.

** You are correct, this is resolved in JON 3.0.0-BETA1.  I just tested it there and it deployed successfully.

-- Regarding the fail/success deploy samples that I provided, even though I named the bundles differently, they were really just meant to be the same bundle, I just split it to be able to have both bundles in the system at the same time.  So yes, realistically, they would only ever go into the one location. 

-- It's good to hear your recommendations regarding installing bundles to the same location, especially given that JON 3 has the "Purge" option for bundles.  I had originally considered having the following setup for my "demo":
  -- eap-5.1.1-base-server: deployed to /opt/jboss
  -- eap-5.1.1-default-instance: deploy to /opt/jboss/jboss-as/server

But given your suggestion, I would think that it should look more like:
  -- eap-5.1.1-base-server: deployed to /opt/jboss with a symbolic link to /opt/server
  -- eap-5.1.1-default-instance: deploy to /opt/server

Otherwise, great to know about the "debug -e" option on the agent, my main issue is that in my testing, I've mostly been using the embedded agent, will try to use a full agent more often then.

However, on a final note with regards to this, I raised the bug originally on JON 2.4.1.  It's great knowing that it's resolved in JON 3.0, but would there be any plan to back port it into JON 2.4.1 given that that version will still be support for another almost 2 years...

Many thanks for your feedback though, much appreciated!

Comment 8 Heiko W. Rupp 2011-09-29 08:47:12 UTC
Deon,
can you make sure the docs contain a blurb about what Mazz wrote in Comment 6:

---
The idea with bundles is to not have multiple bundle destinations use the same
base directory. Otherwise, you'll be installing different bundles to the same
locations and that could cause problems (i.e. you'd be overwriting a previously
installed bundle with some other different bundle). This is alluded to here:
---

Comment 9 John Mazzitelli 2011-09-29 14:33:53 UTC
(In reply to comment #7)
> I had originally considered having the following setup for my "demo":
>   -- eap-5.1.1-base-server: deployed to /opt/jboss
>   -- eap-5.1.1-default-instance: deploy to /opt/jboss/jboss-as/server
> 

You will be able to do something like this in the new release, too. This is what the new "manageRootDir" attribute is for in the deployment-unit tag - see bug #659142 and http://rhq-project.org/display/JOPR2/Ant+Bundles#AntBundles-rhq%3Adeploymentunit for more on that.