Right now you can't deploy a bundle into /deploy of JBAS without having the Bundle subsystem also clean out /deploy. The workaround is to push the bundle out to a subdirectory of deploy, e.g. /deploy/myapp, and then your bundle contents would deploy into the ./myapp folder and keep that in sync. A solution, which mazz outlines below, would be to have a flag saying don't clean out the deploy directory when you lay down the bundle. Edited conversation on the topic: (4:59:51 PM) mazz: well, if you think about what bundles are - they are an "installation" of something (5:00:05 PM) mazz: so if I have a bundle called with files "a" and "b/ccc" (5:00:12 PM) mazz: and I deploy that to directory "foo" (5:00:21 PM) mazz: then foo will match exactly what the bundle has (5:00:29 PM) mazz: so on the target machine, directory foo will look like (5:00:31 PM) mazz: foo/a (5:00:35 PM) mazz: foo/b/ccc (5:00:39 PM) mazz: no more no less (5:00:46 PM) ccrouch: agreed (5:00:52 PM) mazz: replace "foo" with "deploy/" (5:00:53 PM) mazz: :) (5:01:18 PM) mazz: and you will see the problem (5:01:29 PM) mazz: "problem" in quiotes (5:01:43 PM) mazz: since its not really a problem, just a misunderstanding of what the bundle provisioning system is for and what its doing (5:02:03 PM) mazz: you can't have a bundle with say "jmx-console/" in it (5:02:28 PM) mazz: deploy it to the "deploy/" directory (5:02:50 PM) mazz: because the bundle system will expect that you want the deploy/ dierctory to look just like your bundle.zip - which includes JUST this jmx-console/ directory and its subfiles (5:03:04 PM) mazz: thus if we find anything else in deploy/, we think they are rogue files that need to be deleted (5:03:32 PM) ccrouch: ok, i gotya now (5:04:25 PM) mazz: thus there are two solutions (5:04:37 PM) mazz: a) use <ignore> to ignore all the other stuff in deploy/ (5:04:53 PM) mazz: b) bundle the entire JBossAS distro in your bundle, packaged with your deployment files (5:05:02 PM) mazz: b) is how we typically envisioned people using bundle provisioning (5:05:12 PM) mazz: you bundle the entire distribution of an app server installation (5:05:25 PM) mazz: but there is nothing that I see that can prohibit you from doing a) (5:10:06 PM) mazz: ok, based on how bundles work, what they do - all I can say is "that's how it works". You have a zip file, you delpoy it to a remote directory and you want that remote directory to match exactly what is in your zip (5:10:19 PM) mazz: any differences get rectified (5:10:29 PM) mazz: any missing files are adding, any extra files are removed (5:10:47 PM) mazz: thus we match the remote directory with your exactly bundle content (which we then tweek with file replacements) (5:16:33 PM) mazz: we could enhance this of course (this goes back to what I put I think in some etherpad or email somewhere) (5:16:45 PM) mazz: where I said someone should spend a few days enhancing/fixing up bundles (5:16:58 PM) mazz: for example, we could add maybe a new flag, "leave top root directory content alone" (5:17:00 PM) mazz: something to that effect (5:17:17 PM) mazz: where we are saying, "deploy the bundle in the deploy-dir, but any other files found in the top level deploy dir, leave alone" (5:18:04 PM) mazz: so if I had "jmx-console/" as the top dir in my bundle, and I deploy my bundle to "deploy/" (5:18:29 PM) mazz: if I set this flag, I leave all other content found in deploy/ alone and I just put whatever files are in the bundle (i.e. jmx-console/* files" in deploy/ (5:21:01 PM) mazz: in short, the way you look at bundles is - unless you are ignoring files, your deployment directory file lists will look EXACTLY like the file lists in your bundle zip (5:21:05 PM) mazz: no more no less (5:21:40 PM) mazz: if v1 had file "a", but v2 does not have file "a", if you deploy v2, you will NOT have a file "a" (5:21:53 PM) mazz: v1's file "a" will be deleted once you deploy v2 because v2 doesn't have "a" (5:24:50 PM) ccrouch: mazz: right the concept of "overlays" or whatever is not what we're aiming for with the provisioning feature
One issue with the approach outlined in the initial comment is that it can't handle file removals at the base level, e.g. Consider two bundles... bundle v1.0 /my-ds.xml /my-jms-queue.xml /my-webapp/ /WEB-INF/ web.xml bundle v2.0 /my-ds.xml <--- my-jms-queue.xml removed /my-webapp/ /WEB-INF/ <--- web.xml removed Initial position: deploy/a.xml deploy/b.xml Deploy bundle v1.0 into /deploy: deploy/a.xml deploy/b.xml deploy/my-ds.xml deploy/my-jms-queue.xml deploy/my-webapp/ /WEB-INF/ web.xml Deploy bundle v2.0 into /deploy: deploy/a.xml deploy/b.xml deploy/my-ds.xml deploy/my-jms-queue.xml <--- my-jms-queue.xml *NOT* removed deploy/my-webapp/ /WEB-INF/ <-- web.xml is removed my-jms-queue.xml is still there because the flag said not to clear stuff out of the directory we were deploying into.
> One issue with the approach outlined in the initial comment > is that it can't handle file removals at the base level There may not be a way around this. What we can do is perhaps spit out a warning to the user or at least log something if we detect that there are files in the base level of the new bundle version (we might even know if there was files at the base level of the old version, have to check to code to see if that's true). If we see this, we could at least spit out a warning to say, "you told us not to touch the base level, but you put files in the bundle at the base level , we cannot guarantee to keep those files in sync". Other than that one issue, I think this is very possible to implement.
implementing this will make deploying ears/wars easier
*** Bug 653880 has been marked as a duplicate of this bug. ***
question - should this "leave top root directory content alone" setting be in the recipe itself or as part of the deployment wizard? In other words, does the bundle version itself require it be deployed with the root directory content left alone? Or could a bundle version be deployed either way and let the user determine at deploy time if the root directory should be cleaned or not. Its a question of whether the nature of the bundle version itself implies how it should be deployed OR if the user should be knowledgable enough to know how the bundle should be deployed. If the user should and does know how the bundle should be deployed, the setting is in the UI wizard (which of course forces the user to make the decision). If we don't want to make the user have to understand this setting and we don't want him to make the decision; the setting should be in the recipe itself (thus deployment is easier for the user, but the packager of the bundle version distro has to know how we wants the bundle content deployed). I'm inclined to think this is a recipe-setting. 1) I don't want to introduce another setting for the user to have to know about and choose a value in the UI and 2) I'm not sure a bundle distro author would be able to package a bundle properly to support both settings. For example, if I'm bundling an ear, I would think I would bundle it knowing that it would get deployed in a deploy/ directory and thus there isn't a choice, it HAS to be deployed with this "leave top root directory content alone" setting turned on. You can't do it with that setting off (if you did, you get the problem that caused this BZ to be written).
another separate thought - no matter if its a UI setting or recipe setting, this will only be supported by the Ant bundle type since the file template bundle type doesn't do this rpm-like deployment/merging of files.
here's what I propose <rhq:deployment-unit name="the-name-here" manageTopDeploymentDir="false"> where "manageTopDeploymentDir" is this new setting being discussed. The default will be true (which is how it works today). Setting it to false will tell the deployer to not touch any files located at the top deployment directory (i.e. where the bundle is being deployed). It will only manage files in subdirectories under that top deployment dir.
I think there is going to be one major caveat/limitation to this. You will only be able to deploy one bundle to a specific location, even if you specify manageRootDir (manageRootDir is currently the name I'm going with). Here's an example. Suppose I have two war files - a.war and b.war: a.war: /WEB-INF/ web.xml b.war: /WEB-INF/ web.xml I could deploy them both in my JBossAS instance IF I use a different Bundle Destination for both, where the destination directory for destination #1 would be ".../deploy/a.war" and for #2 it would be ".../deploy/b.war". When they are deployed, RHQ needs to store metadata about what was provisioned in its .rhqdeployments directory, which is located in the destination dir: .../deploy/a.war/.rhqdeployments/ -> metadata files are stored in this hidden dir .../deploy/b.war/.rhqdeployments/ -> metadata files are stored in this hidden dir Now look what happens if I deploy directly to the deploy/ directory. In this case, the wars must be packaged with their own top level directory: a.war: /a.war/ /WEB-INF/ web.xml b.war: /b.war/ /WEB-INF/ web.xml Deploying both to the same Bundle Destination whose destination directory is ".../deploy" results in: .../deploy/a.war/WEB-INF/web.xml .../deploy/b.war/WEB-INF/web.xml .../deploy/.rhqdeployments/ -> metadata files are stored in this hidden dir - but WHOSE metadata????? Notice that we are able to store a.war and b.war, however, their metadata collide because they both want to use the same .rhqdeployments directory to store their metadata. Bad things will now happen because the metadata will mix between the two deployments. What we have to do is document this limitation. If someone wants to deploy to a location, only a single bundle deployment can happen in that one location. If you have multiple packages (like a.war and b.war in this example) the user must merge the two into a single bundle distribution file. Note that this limitation is nothing new, its always been there. However, it shows itself more readily because now we have a real use-case where someone would want to provision two bundles in the same dest dir.
(In reply to comment #8) > Note that this limitation is nothing new, its always been there. However, it > shows itself more readily because now we have a real use-case where someone > would want to provision two bundles in the same dest dir. I am going to see if I can implement something that takes out this metadata. But I don't know if I easily can. I may be able to store multiple bundle metadata in the .rhqdeployments directory, perhaps keying on bundle name? (.rhqdeployments/a.war). If I can't easily fix that, I'm going to skip it for now - we can circle back later if we need it.
(In reply to comment #9) > (In reply to comment #8) > > Note that this limitation is nothing new, its always been there. However, it > > shows itself more readily because now we have a real use-case where someone > > would want to provision two bundles in the same dest dir. > > I am going to see if I can implement something that takes out this metadata. > But I don't know if I easily can. I may be able to store multiple bundle > metadata in the .rhqdeployments directory, perhaps keying on bundle name? > (.rhqdeployments/a.war). If I can't easily fix that, I'm going to skip it for > now - we can circle back later if we need it. Another idea is to just have different current-deployment.properties files in the .rhqdeployments directory. IIRC, all metadata .properties are keyed on deployment.id - so it might be possible to deploy multiple things and have metadata stored in the .rhqdeployments/##### folders (since each deployment will have its own unique ID across all bundles - and if we revert a deployment, the unique IDs will enable us to find the metadata we need without worry about subdirectories under .rhqdeployments like .rhqdeployments/a.war). The main problem would be we'd need to support multiple current-deployment.properties files (today, there is assumed to be only one). This would require lots of thought/design discussions. I'm not going to do that right now, there's alot that could go wrong trying to implement that. We'll circle back later.
and yet a third idea would be to create multiple top-level hidden metadata directories. Instead of having .rhqdeployments, we could have one per bundle destination: .rhqdestination12345/.rhqdeployments where 12345 is the bundle destination ID. There could be multiple destination metadata directories in the same location. I don't know if the agent has the destination ID, so this might not be possible. Again, need to think hard about all possibilities. I'm just trying to jot down ideas as they come to me so I don't forget.
git commit 067f66c0c2e4f1e9f544158f9d356f0ca70f2479 this adds the necessary support to the Deployer utility, and adds the Ant attribute "manageRootDir". New unit tests have been added to test this functionality. More testing needs to get done on the Ant side to make sure the recipe can pass this flag properly to Deployer. The limitation exists that you can only deploy a single bundle to any one destination directory. In other words, you can only deploy a single bundle to JBossAS deploy/ directory (thus you must bundle your ears and wars and other deploy artifacts in a single bundle). This is not a new limitation, it has always been that a single directory can only host a single bundle deployment.
commit d2a766a39f4e3cf9b6e823d53dd356f8d42c5b1c adds unit test showing the manageRootDir=false attributing working in an ant recipe.
closing this - ready for QA. Here's how to test : 1) create an ANT bundle whose recipe's rhq:deployment-unit has the attribute "manageRootDir=false". The bundle can have one file or archive - doesn't have to be big - just a file or two that you will deploy 2) Via the GUI, upload the bundle 3) Via the GUI, create bundle destination to some directory (e.g. /tmp/bundle) 4) On your file system, create /tmp/bundle directory 5) In that directory you created in step 4, add files and subdirectories at the top level: * echo "this is unrelated to the bundle" > /tmp/bundle/unrelated-file.txt * mkdir /tmp/bundle/unrelated-dir * echo "another unrelated file" > /tmp/bundle/unrelated-dir/somefile.dat These are files that will not be managed and should not be touched during the bundle deployment. 6) Now deploy your bundle to the destination (this will put your bundle files in /tmp/bundle) 7) Verify that unrelated-file.txt and unrelated-dir still exist 8) Re-deploy your bundle again - this performs an "upgrade" 9) Do the same verification as step 7. If your unrelated files and directories still exist and remain as they were when you created them, then this issue has been fixed and we can close this as verified. --- Here's an example recipe you can try when testing - just call it "deploy.xml" and zip it up along with a file called "bundle-file.txt" and use that as a bundle distro file you can upload to the server via the GUI: <?xml version="1.0"?> <project name="test" default="main" xmlns:rhq="antlib:org.rhq.bundle"> <rhq:bundle name="test" version="1" description="test bundle"> <rhq:deployment-unit name="test" manageRootDir="false"> <rhq:file name="bundle-file.txt" destinationFile="bundle-file.txt"/> </rhq:deployment-unit> </rhq:bundle> <target name="main"/> </project>
Verified on build#1090 (Version: 4.0.0-SNAPSHOT Build Number: f19c23e) Created a bundle whose recipe's rhq:deployment-unit has the attribute "manageRootDir=false". Created dir /tmp/bundle and files and subdirectories at the top level to /tmp/bundle. Deployed the bundle to /tmp/bundle. Observed that the unrelated-file.txt and unrelated-dir still exist after bundle deployment. Also tested this with redeploying the bundle. Marking as verified.
Bookkeeping - closing bug - fixed in recent release.