Bug 110234 - Multiple problems with error & command line parameter handling in HostInit command
Multiple problems with error & command line parameter handling in HostInit c...
Status: CLOSED RAWHIDE
Product: Red Hat Web Application Framework
Classification: Retired
Component: other (Show other bugs)
nightly
All Linux
medium Severity medium
: ---
: ---
Assigned To: Dennis Gregorovic
Jon Orris
:
Depends On:
Blocks: 109665
  Show dependency treegraph
 
Reported: 2003-11-17 08:28 EST by Daniel Berrange
Modified: 2007-04-18 12:59 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2003-12-04 17:33:18 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Daniel Berrange 2003-11-17 08:28:50 EST
Description of problem:

The following command line transcript demonstrates most problems:

[root@barnet root]# ccm hostinit       
no such parameter: http-port
[root@barnet root]# ccm hostinit -help
Unknown option: help
no such parameter: http-port
[root@barnet root]# 
[root@barnet root]# ccm hostinit --help
Unknown option: help
no such parameter: http-port
[root@barnet root]# ccm
usage: ccm [OPTIONS | COMMAND]

Options:
  --help     Display help
  --usage    Print this message

Commands:
  load       Load a CCM package
  get        Print one or more values from a CCM configuration database
  set        Set one or more values in a CCM configuration database
  status     Report on the status of a CCM instance
  which      Find a resource or class in the CCM classpath
  hostinit   Write the servlet container configuration file and merge
webapp files.
  start      Start the servlet container configured for CCM
  stop       Stop the servlet container configured for CCM
[root@barnet root]# ccm --help hostinit
usage: ccm [OPTIONS | COMMAND]

Options:
  --help     Display help
  --usage    Print this message

Commands:
  load       Load a CCM package
  get        Print one or more values from a CCM configuration database
  set        Set one or more values in a CCM configuration database
  status     Report on the status of a CCM instance
  which      Find a resource or class in the CCM classpath
  hostinit   Write the servlet container configuration file and merge
webapp files.
  start      Start the servlet container configured for CCM
  stop       Stop the servlet container configured for CCM
[root@barnet root]# ccm hostinit http-port 9000
no such parameter: http-port
[root@barnet root]# ccm hostinit http-port=9000
[root@barnet root]# ccm hostinit http-port=9000
[root@barnet root]# ccm hostinit http-port=9000
[root@barnet root]# ccm hostinit http-port=9000
[root@barnet root]# ccm hostinit http-port=2
[root@barnet root]# 


In all the examples above where the command line was incorrect (it
displayed a error message), it still went ahead and did the copying of
files. It must stop immediately, performing no action if any command
line option were incorrect.

It seems that is impossible to get help about the parameters
'hostinit' requires. As should above 'ccm hostinit --help' fails

A further problem is that it doesn't check to see if a webapp already
exists - it just overwrites whatever is there, with [in some of these
examples] incorrect configuration. A much safer mode of operation is
to print an error message & exit if the webapp already exists.
Additional command line arg '--update' or '--overwrite' should be
available to tell hostinit to overwrite an existing app.

Before writing the config file it should also verify that the port
number is sensible (ie > 1024) and at least display a warning message
if something is listening on that port already.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:

Expected results:
1. When errors are encountered no action should be taken
2. It should be possible to get command line help on 'hostinit'
3. It should not silently overwrite existing webapps
4. It should sanity check port numbers.

Additional info:
Comment 1 Dennis Gregorovic 2003-11-17 16:08:17 EST
In changelist 38093 I have improved the error handling so that ccm
hostinit will exit immediately and print usage text if it encounters
invalid command line switches.  Also, there is now real usage text. 
Try "ccm --usage" or "ccm hostinit --usage".

I agree that deleting an existing webapp root as part of "ccm
hostinit" is overly dangerous.  On the otherhand, not copying files to
an existing webapp root is problematic in the expected common use case
where a user adds/upgrades a package in an existing ccm installation
and runs hostinit to deploy the files.  

My suggestion is:
  * The default behavior is to copy all files to the webapp root, 
    overwriting any existing files which are older than the source 
    files (i.e. "cp --update -r from to")
  * Add a --overwrite flag which copies all files to the webapp
    root regardless of timestamp.  (i.e. "cp -r from to").  This would
    likely be used by the user if they downgrade a package or have 
    customized files in the webapp root that they want to revert.
  * Add a --clean flag which first deletes the webapp root before
    copying the files.  (i.e. "rm -rf to; cp -r from to") This is the
    current behavior.  This would likely be use by the user to 'reset'
    a CCM instance.  This may also be used after a package has been 
    removed from a CCM instance.

As for the verification, it's my opinion that hostinit should do
validation, but not policy enforcement.  For example, we should check
that the port number is actually a valid decimal number.  However, it
should not check whether it is above or below 1024 as that it an
installation decision.  
Comment 2 Daniel Berrange 2003-11-18 05:53:26 EST
WRT to the file copying issue, I think any behaviour which either
causes files to be overwritten, or obsolete files not to be removed is
going to have a very high potential for creating inconsistency in the
webapp. I think we are causing unneccessary complication by making
host init deal with too many use cases. I suggest that there are two
primary uses cases for hostinit:

  1. Initial setup of the webapp. Copying all files for all configured
applications to the webapp. 

  2. Re-setting an existing webapp. Remove all existing files & copy
all files for all configured applications to the webapp.

Since case 1. is the common one, it suggests that the default
behaviour should be to exit with an error if the webapp already
contains files. For case 2., the addition of a --clean flag would be
sufficient.

The remaining use cases to be dealt with are:

  3. Adding a new application. Copying all files from new application
to existing webapp. Ensure that no files from a pre-existing app are
overwritten.

  4. Upgrading an existing application. Copying all files from new
application to existing webapp. Ensure that no files from a
pre-existing app are overwritten. Delete all existing files that are
no longer present in upgraded version.

  5. Deleting an existing application. Remove all files that are part
of the application being deleted.

Use case 3, suggests a new command 'add application', since the using
the 'host init' command would do faaar more work that is desired - ie,
re-copying all existing apps, regenerating the servlet configuration
file (with potential errors that entails, such as  accidentally giving
it a different port number).

Use cases 4 & 5, also suggest new commands, but in order to reliably
implement them we'd have to make the system keep a manifest of all
files it copies about on the system. We'd essentially be writing our
own custom 'RPM':-( Of course the best solution is not to have to do
any file copying whatsoever, ie use WARs. Depending on how far off the
 WAR solution is, it is probably a satisfactory short term solution to
deal with uses cases 4 & 5, but saying use 'ccm hostinit --clean'.

The other issue mentioned was wrt to 'user customized files' in the
webapp. If we support this by letting the tool only overwrite based on
timestamps we are going to cause serious pain for deployments. The
golden rule of build process, packaging & deployment is that we should
*not* support any overwriting / customization of files, except at
source (ie in the dev area containing the pristine code from version
control). We should explicitly document that any changes a user makes
to a deployed webapp will be blown away / overwritten upon next
invocation of the ccm tools. 

It is not really us that is defining the policy, we're just checking
that users don't violate the OS policy. Consider:

 a) On UNIX port numbers <= 1024 are only bindable by root
 b) Application servers run as unprivileged user 'servlet'

Taking those two points together, we need to check that the port
number is > 1024 at some point before the first server startup. AFAICT
the 'ccm hostinit' command is the only place before server startup
that this check can be performed. Checking that nothing else is
listening on the specified port is also pretty important - since it is
a surprisingly common problem - many software packages & users simply
pick 8080 or 8000 as the not-port-80 default value for web servers. 
Comment 3 Richard Li 2003-11-20 16:57:19 EST
Not all application servers run as unprivileged user 'servlet', and
there are plenty of CCM deployments out there that run on ports < 1024.

So, I think the correct thing to do here is print a polite INFO
message reminding people, but it is not an error or warning.

Likewise for server port (what if you're generating config files for a
different server? what if you are planning to shut down your server?).

Actually, I think the nice thing to do is write a separate tool to
check your config file on demand. That also addresses the user
customized files use case. Dan, why don't you write this, and you can
do whatever you want :)?

Seriously, I think that while what you propose is a reasonable policy
that is clearly followed by APLAWS, it's not clear that it's followed
by everyone, so the right thing to do is create a policy-checking
mechanism that is independent of product i.e., a separate program that
enforces policy on the generated files.
Comment 4 Daniel Berrange 2003-11-21 05:10:42 EST
> Not all application servers run as unprivileged user 'servlet', and
> there are plenty of CCM deployments out there that run on ports 
> < 1024.

All CCM deployments that need to listen < 1024 have an Apache (or
equiv) front end listening on the low port & use mod_caucho/mod_jk to
forward to the application server. Allowing (let alone encouraging)
people to run an application server as root is just plain stupid.

cf bug 110561 - both Apache & SQUID *refuse* point blank to run as
'root'. We should do the same. Since Java has no mechanism to drop
user privileges once the JVM has started, there is no circumstance in
which setting a port < 1024 is anything but an error.

> Actually, I think the nice thing to do is write a separate tool to
> check your config file on demand. That also addresses the user
> customized files use case. Dan, why don't you write this, and you can
> do whatever you want :)?

Validation should take place at the time you are setting the
parameters. If it is in a separate tool, you can practically guarentee
no-one will ever run it.

Our config/startup processes should enforce good practice.
If a tiny minority of people want to shoot themselves in the head &
run as root, then they can skip our automated resin.conf/server.xml
generation & manually configure their app server.
Comment 5 Dennis Gregorovic 2003-11-24 13:33:24 EST
validation added @38249
Comment 6 Dennis Gregorovic 2003-11-24 15:01:25 EST
I have some comments on Dan's use case #3.

* We do want to regenerate the servlet configuration file when adding
a new application.  In the near-to-medium future, each webapp will
have a separate entry in the config file.  This will be integral to
removing the file copy step.  So, instead of copying files to a common
webapp directory, we'll have entries like:

<web-app id='forums' 
appbase='/usr/share/java/webapps/ccm-forums-1.0.0/forums'/>

* We now have functionality that will 'remember' previous parameter
values for ccm hostinit, so we don't have to worry as much about the
user forgetting to specify http-port after the first run.  See 38250.

In my opinion 'adding a new application' is actually the common use
case and the initial setup is a specific instance of that operation. 
I suggest that the default behavior is to copy all source files but
not overwrite any existing target files.  I agree that we should not
support customized files in the deploy directory and that timestamp
checking isn't needed.
Comment 7 Dennis Gregorovic 2003-11-26 18:13:35 EST
file copying changes checked in @38338
Comment 8 Jon Orris 2003-12-01 11:02:20 EST
Marking QA Ready.
Dan, can you comment on whether the solution is OK, as you opened the
ticket?
Comment 9 Daniel Berrange 2003-12-01 11:18:46 EST
The (UNIX only) check for 'http-port & shutdown-port > 1024' needs to
be an error condition rather than just a warning - since we don't ever
run as root (bug 11056) it is impossible to bind to <= 1024.
Comment 10 Dennis Gregorovic 2003-12-01 12:59:39 EST
warning changed to an error @38382
Comment 11 Jon Orris 2003-12-03 14:44:48 EST
Fails. Issues:

--clean flag does not appear to work
[root@watershed ccm]# ccm hostinit --clean --container=resin
http-port=9000
Unknown option: clean
usage: hostinit [OPTIONS]

Options:
  --help      Display help
  --usage     Print this message
  --container Specify the servlet container to initialize

* Smaller issue: Port validation works with ccm hostinit. Hovever, ccm
load does no validation on the port at all, so if you set it to 80,
you'll have to go back and change this as well.
Comment 12 Dennis Gregorovic 2003-12-03 15:11:08 EST
the problems with --clean have been fixed @38467

Note You need to log in before you can comment on or make changes to this bug.