packaging: setup: re-write using otopi

OUTLINE

We introduce a complete rewrite of ovirt-engine setup, which includes
engine-setup, engine-upgrade, engine-cleanup.

The new implementation is installed side-by-side, user may have a choice
which implementation to use.

COMPATIBILITY

Implementation are incompatible, once setup using legacy, upgrade and
cleanup will not work using the new implementation and vise versa.
Answer file is not compatible between implementations.

Implementation is not feature complete, setup is is almost complete,
cleanup needs more work to clean files that where partially modified,
currently, setup is not able to upgrade non-new implementation setup.

MAJOR FEATURES

Fully support development environment installation.

Up until now, developers had maintained parallel environment, manually
customized, partial featured.
Now, developers may setup product much like production using unprivileged
user, custom location and custom database. For more information refer to
README.developer file.

Platform independent implementation

The new implementation can/should be able to run in multiple
distributions, setting up valid engine environment. So far tested
fedora, rhel, centos, gentoo, ubuntu.

Database access

Split between host provisioning and database setup.

Host provisioning is the part setup can configure the database package on
host to be able to serve the application. The provisioning phase is now
completely optional and is supported only at legacy distributions.

Database setup is the part that a schema is created in existing
database. This enables the dba to provide ovirt administration
unprivileged database user and database for his use.

Implementation

Pluggable, mission centric, consistent, modern, customizable implementation.
Reuse the otopi infrastructure that is also used for host-deploy, and
future reuse of setup for dwh and reports components.

Change-Id: Id587977be496f979f692ee5fd06a4e264d3f9197
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=911191
Signed-off-by: Alon Bar-Lev <alonbl>
Signed-off-by: Sandro Bonazzola <sbonazzo>
Signed-off-by: Alex Lourie <alourie>

During implementation, the pki configuration were moved from vdc_options to /etc/ovirt-engine/engine.conf[.d/*].

These are determine by setup and should not be changed, so there is no reason to store them in database, and handle setup/upgrade that are database related.

Another reason was that the root application (/) should serve some of these resources, and having root access database is undesired.

---

vdc_options (removed)

CACertificatePath
CABaseDirectory
CertificateFileName
keystoreUrl
keystorePass
TruststoreUrl
TruststorePass
SignScriptName
CertAlias

engine.conf (added)

ENIGNE_PKI=/etc/pki/ovirt-engine
ENGINE_PKI_CA=${ENGINE_PKI}/ca.pem
ENGINE_PKI_ENGINE_CERT=${ENGINE_PKI}/certs/engine.cer
ENGINE_PKI_TRUST_STORE=${ENGINE_PKI}/.truststore
ENGINE_PKI_TRUST_STORE_PASSWORD=
ENGINE_PKI_ENGINE_STORE=${ENGINE_PKI}/keys/engine.p12
ENGINE_PKI_ENGINE_STORE_PASSWORD=
ENGINE_PKI_ENGINE_STORE_ALIAS=1

engine-setup, engine-cleanup are now symlink to otopi implementation.

Hi,

I still want us to drop the user/password authentication and use PK it will allow us to:
1. drop the prompt for root password, and the chance we lock it out.
2. drop the python module ssh dependency.

Hi Alon, can that be done in a separate bug?

(In reply to Sandro Bonazzola from comment #5)
> Hi Alon, can that be done in a separate bug?

separate, same... same to me... as long as it is in queue for this release.

Documentation major notes:

ovirt-engine setup/upgrade/cleanup was completely re-written in 3.3.

Major changes includes:

1. engine-upgrade is obsoleted

engine-setup is now used to both setup a fresh product and existing product. In case of existing product it will actually perform an upgrade.

engine-setup will also allow to configure new components, for example if a new package is installed after setup (example: ovirt-engine-websocket-proxy), then it will allow configuration of that package when running engine-setup again.

2. engine-check-upgrade is obsoleted and replaced by engine-upgrade-check, which is simply a wrapper that basically only executes yum transaction similar to:

# yum --disableplugin=versionlock 'ovirt-*'

3. engine-setup/engine-cleanup command-line parameter changed.

# engine-setup --help
Usage: /usr/bin/engine-setup
    --log=file
        write log to this file.
    --config=file
        Load configuration files.
    --config-append=file
        Load extra configuration files.
    --offline
        Offline mode.
    --generate-answer=file
        Generate answer file.
    --jboss-home=dir
        Use this jboss.

# engine-cleanup --help
Usage: /usr/bin/engine-cleanup
    --log=file
        write log to this file.
    --config=file
        Load configuration files.
    --config-append=file
        Load extra configuration files.
    --generate-answer=file
        Generate answer file.

4. Answer file format changed

Answer file is now otopi style configuration file, it means that it is an environment with set of key value pair.

The advantage of using environment is that variables are used only if actually supported, there is no failure if new variables are introduced, and there is a default for every variable so specifying only subset possible.

When automatic generating answer file, only the interactive prompts are written, but in fact there are many more variables that can effect the setup execution, these can also be specified when required.

5. Pluggable infrastructure

More functionality can be easily added and integrated using 3rd party packages, just by dropping python module into pre-defined location. This enables future work to be created without dependency of the ovirt-engine core components.

closing as this should be in 3.3 (doing so in bulk, so may be incorrect)