Bug 807411

+++ This bug was initially created as a clone of Bug #801926 +++

Suppose I have a bundle with a subdirectory in it with a file inside the subdir:

bundle.zip:
   subdir/foo.txt

Now suppose I deploy this to a destination directory that already has that subdirectory in it, BUT with a different file:

dest-dir:
   subdir/bar.txt

What should the proper behavior be with managedRootDir=true? This is easy - the bundle has subdir in it with a file foo.txt. Bundle deployment wants to make the destination directory IDENTICAL to the bundle, and since manageRootDir=true, we don't care about what's in the destination directory. We deploy and the destination directory will have "subdir/foo.txt" and that is it - this matches the bundle.

BUT! Now suppose that my bundle has manageRootDir=FALSE. What should the behavior be?

On the one hand - the bundle specifies a "subdir" subdirectory inside of it. Therefore, we could say "the bundle has declared that it wants to own a subdirectory called "subdir" under the destination directory; therefore, the bundle will want to manage that subdirectory entirely - any files found in the subdir will be managed by the bundle, therefore, bar.txt will be removed and only foo.txt will exist." This is how it should be working today in the current code.

On the other hand, however, the destination directory (aka the "root dir") already has this subdir in it and our recipe told the bundle to not manage the root directory. What should we do with subdir/bar.txt? Should we leave it alone because we aren't supposed to manage the root directory (which, in this case, had a subdirectory with a file in it - and this file is not mentioned in the bundle).

The crux of the issue is therefore:

1) manageRootDir=false 
2) AND both the bundle and the destination directory has a subdirectory in it
3) AND the bundle subdirectory and the destination directory subdirectory do not have the same files in them.

What do we do with the files in the destination directory subdirectory that are NOT specified in the bundle subdirectory?

Which ever way we do it, we need some unit tests to test this out. Because this is not something I think we have unit tests for.

--- Additional comment from loleary on 2012-03-09 19:42:36 EST ---

In either case, we need to make the initial deploy to a destination consistent with the update/redeploy to the same destination.

As of http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=commit;h=ccd5b1f03334064e17f6353bff208aa304a116f8 the behavior is inconsistent. When you deploy the bundle for the first time to a destination and manageRootDir=false, we do not touch any of the files or subdirectories in the destination directory even when the bundle has the same subdirectory but on redeploy we do cleanup those files not in the bundle.

To illustrate:

bundle.zip:
   subdir/foo.txt


dest-dir:
   subdir/bar.txt


On initial deploy I end up with:

dest-dir:
   subdir/bar.txt
   subdir/foo.txt


But on update or redeploy I end up with:

dest-dir:
   subdir/foo.txt


From a package management stand-point, it seems that the initial deploy behavior is the correct one and the update/redeploy is the incorrect one.

--- Additional comment from mfoley on 2012-03-12 11:41:05 EDT ---

per bz triage 3/12/2012 target for 3.1, medium priority, (crouch, loleary, foley)

--- Additional comment from mazz on 2012-03-12 16:07:54 EDT ---

(In reply to comment #1)
> In either case, we need to make the initial deploy to a destination consistent
> with the update/redeploy to the same destination.

I created BZ #802537 to track this.

--- Additional comment from mazz on 2012-03-14 16:33:10 EDT ---

I documented some use-cases on the wiki:

http://rhq-project.org/display/JOPR2/Provisioning#Provisioning-ExampleUseCases

and this use case specifically is here:

http://rhq-project.org/display/JOPR2/Provisioning#Provisioning-DeployingConfigurationFilesToAnAlreadyInstalledApplicationServer

After fixing bug #802537 which forced me to think about this more, I'm of the opinion now that this issue should be closed as NOTABUG. Here's why....

First, recall what bundles are. They are a *complete* set of files that make up a FULL piece of software. When first envisioned, this was supposed to be an entire set of files that make up a fully installed server (such as, for example, a JBoss Application Server installation). People liked bundles so much :) they wanted to use this to deploy applications to already installed app servers so we later enhanced this to allow things like a WAR (or EAR) to be deployed inside an already existing JBossAS server (such that we can deploy it in the deploy/ directory and leave other WARS and EARS (aka xARs) found here alone). This was the manageRootDir=false setting. But notice that these xAR bundles are STILL FULL applications. All the files that make up the xAR are in the bundle - there is nothing missing and nothing is to be added. Thus, when you lay down a xAR bundle, all the files from the bundle make up the full WAR/EAR app and any files found in the deployment directory (and more specifically their subdirectories) are removed to make way for the bundle.

Bundles were not meant to be a partial set of miscellenous or ad-hoc files that could be "added" to existing software in an existing deployment directory to "complete" that software installation. By this I mean, whatever is in a bundle is ALL the files that should be in the destination directory when the bundle is laid down. The bundle distribution file should look exactly how you want the end resultant destination directory to look (including all subdirectories found in the bundle). Any extra files that were found in the destination directory were backed up and removed.  That means any files found in any subdirectories that are in the bundle are cleaned to make way for the bundle's subdirectory content.

This is being stretched beyong its limits via this latest use-case - that being, someone wants to AUGMENT an existing JBoss Application Server's conf/ and conf/props directories. This is because, while it is allowed to deploy a bundle to the conf/ directory with manageRootDir=false, the problem is the bundle is shipped with a "props/jmx-user.properties" files but the app server already has files in this "props/" subdirectory. According to the design and rules of bundle deployment, because the bundle has a "props" subdirectory, this is saying that the bundle wants the "props" subdirectory to look EXACTLY how the bundle has it - in this case, with a single file called jmx-users.properties. Any other files found here during deployment are removed, because other wise, the destination directory's subdirectories won't look EXACTLY how the bundle has defined it (and that is what the RHQ bundle subsystem is told to ensure).

So, how can you support doing this? There are two things you can do. Either:

a) use the bundle subsystem as it was originally designed/intended - that is, to deploy the full JBoss App Server installation itself - this way, you can control the entire content of the JBossAS - including the content of conf/ and conf/props. So create a bundle with your full JBossAS installation, including your custom conf and conf/props files.

or

b) in your bundle, place all the files that you want in the "props" directory - not just the jmx-user.properties, but put in *all* the files that you want to exist in the conf/props (like jmx-roles.properties, etc.). This way, your bundle will manage the full conf/props directory and not just partially manage some of the content (remember, the bundle subsystem manages the entire content of all subdirectories, not just a partial subset of content).

I'm going to close this as NOTABUG - if folks feel this needs to be fixed, we can revisit. But if we are going to support something like this, realize this will require some new design/architecture in the bundle subsystem because it is going against one of the cardinal rules of the bundle system (that being, the bundles define the FULL set of content, not a subset). Implementation wise, I believe alot of changes and tests will be required to allow for this use-case.

--- Additional comment from mazz on 2012-03-14 16:39:28 EDT ---

there is a third alternative that I didn't think of. You could use additional ant code in your recipe to move files around if you want. So have the bundle deploy the files in some central location, then you just use ant in a post-install target to move the files to where you want.

--- Additional comment from loleary on 2012-03-14 17:11:42 EDT ---

I think this needs to be fully documented in the product documentation before we call this done.

Basically, your use-cases and examples above describe using bundles as a means of deploying WARs or EARs to an EAP instance. If this is the goal then we need to create EAR/WAR bundles. For example, the actual EAR/WAR resource type itself needs to extend itself to this concept. Outside of that, we are simply talking about provisioning an application server. Such provisioning requires configuration and library management outside of the deploy directory. So, my suggestion is not to think about *deploy* but to think about the *root* file system. An example of this is an application which requires database configuration. It is possible that a file would need to be added to lib to allow the application to access the database yet the bundle won't know or understand what other bundles and applications and configuration has been deployed in order to explicitly list the files and directories to ignore. The list is essentially impossible to know.

I agree that this is not how it was originally envisioned but what I can tell is that real-world use-cases were not considered when the proof-of-concept was created and instead of fleshing out those real use-cases, we ran with the PoC.

As it stands, this functionality is broken because *manageRootDir* seems to do something different then implied. Perhaps we need to change the name of this property to *cleanDestDir* or *cleanDeployDir*.

--- Additional comment from mazz on 2012-03-14 17:31:18 EDT ---

changing assignment to deon for documentation