Bug 618885

The list of features and their metadata was specifically requested by the QE team when they reviewed the first draft of the document.  The intention is to help the reader understand what functionality is shipped in the default database and what their relationships are to one another (deps, includes, conflicts).  This should help the user understand how the metadata works as well as allow them to plan what they need to add for their own functionality (or what features they wish to deploy).  It is likely a high maintenance item, but I feel it should be included.  I'm ok with including it in an appendix though.

As for the processes, I'll take them in order:
1) Isn't this covered with the example?  It shows adding a group and nodes to the store.  The issue I have here is that "configuring the store with ccs" is a rather large and open ended item.  It all depends on what you want to configure in the store, and as such I went with a generic "Here's what the important options are" approach in the description.  What exactly are you looking for?

2) Again, isn't this covered with the example?  In that example I use ccp to configure 5 machines into a working condor pool.  This is again a very open ended item and I attempted to cover with generic "Here's what's important" documentation.  If you're looking for more, please be specific as this alone could take up pages of documentation.

3) There isn't a built-in editor, it is whatever the user sets in their environment variable (or vi if nothing).  The assumption is that the user should know how to use whatever EDITOR they set in their environment, or vi (the default).

3.3: By processes, do you mean example uses here?  I can provide one line examples of the tools if that's what you're looking for.

Also, need to update the list of features w/ metadata:

Add:
BaseJobExecuter (New Feature)
  include: None
  depends: None
  conflits: None

ExecuteNode
  include: BaseJobExecuter

BaseScheduler
  include: BaseJobExecuter

Remove:
Credd

Should also add a note (maybe a * next to the following) stating they are not intended to the installed as a feature by themselves.  They must be installed with a feature that depends upon them:
BaseJobExecuter
BaseScheduler

(In reply to comment #1)
> The list of features and their metadata was specifically requested by the QE
> team when they reviewed the first draft of the document.  The intention is to
> help the reader understand what functionality is shipped in the default
> database and what their relationships are to one another (deps, includes,
> conflicts).  This should help the user understand how the metadata works as
> well as allow them to plan what they need to add for their own functionality
> (or what features they wish to deploy).  It is likely a high maintenance item,
> but I feel it should be included.  I'm ok with including it in an appendix
> though.

OK, let's move it out to an Appendix. Do you have some text we can wrap around it to explain what it's all about? I'll confess, I don't really understand the relevance.


> 
> As for the processes, I'll take them in order:
> 1) Isn't this covered with the example?  

It needs to be described in the text *as well as* put in an example. An example should not introduce new ideas.

It shows adding a group and nodes to
> the store.  The issue I have here is that "configuring the store with ccs" is a
> rather large and open ended item.  It all depends on what you want to configure
> in the store, and as such I went with a generic "Here's what the important
> options are" approach in the description.  What exactly are you looking for?

Perhaps we could run with a process along the lines of:

1. Here's the syntax
2-n. Here are the things you need to make sure you think about (in order). Explain each setting, show an example command for each step.

> 
> 2) Again, isn't this covered with the example?  In that example I use ccp to
> configure 5 machines into a working condor pool.  This is again a very open
> ended item and I attempted to cover with generic "Here's what's important"
> documentation.  If you're looking for more, please be specific as this alone
> could take up pages of documentation.

See previous notes.

> 
> 3) There isn't a built-in editor, it is whatever the user sets in their
> environment variable (or vi if nothing).  The assumption is that the user
> should know how to use whatever EDITOR they set in their environment, or vi
> (the default).

<title>Editing metadata</title>
	<para>
		The metadata is edited using the YAML format. The three types of input are handled differently by YAML:
	</para>


> 
> 3.3: By processes, do you mean example uses here?  I can provide one line
> examples of the tools if that's what you're looking for.

That would be great :)

LKB

(In reply to comment #2)
> Also, need to update the list of features w/ metadata:
> 
> Add:
> BaseJobExecuter (New Feature)
>   include: None
>   depends: None
>   conflits: None
> 
> ExecuteNode
>   include: BaseJobExecuter
> 
> BaseScheduler
>   include: BaseJobExecuter
> 
> Remove:
> Credd
> 
> Should also add a note (maybe a * next to the following) stating they are not
> intended to the installed as a feature by themselves.  They must be installed
> with a feature that depends upon them:
> BaseJobExecuter
> BaseScheduler

Done.

LKB

I think this sentence in 4.1 should refer to appendix C:

"For more information about the metadata associated with various features used in the remote configuration feature, see Appendix B, Codes."

Maybe appendix C should be titled "Feature MetaData", as it's only listing out metadata for predefined features.

(In reply to comment #5)
> I think this sentence in 4.1 should refer to appendix C:
> 
> "For more information about the metadata associated with various features used
> in the remote configuration feature, see Appendix B, Codes."

<para>
	For more information about the metadata associated with various features used in the remote configuration feature, see <xref linkend="appe-Grid_User_Guide-Metadata" />.
</para>

(In reply to comment #6)
> Maybe appendix C should be titled "Feature MetaData", as it's only listing out
> metadata for predefined features.

<appendix id="appe-Grid_User_Guide-Metadata">
	<title>Feature Metadata</title>


LKB

*** Bug 628105 has been marked as a duplicate of this bug. ***

Configuring the store using condor_configure_store

The condor_configure_store tool is used to add, remove, and edit parameters, features, groups, subsystems, and nodes in the configuration store.  Only 1 action (defined as add, remove, or edit) may be performed per tool invocation, but multiple target types (defined as parameters, features, groups, subsystems, or nodes) may be performed for the action.  To action upon multiple target, each target must be separated by a comma.

The condor_configure_store tool does not need to be run on the same machine as the configuration store, nor the same machine as the broker the configuration store is communicating with.  The condor_configure_store tool defaults to looking for the AMQP broker on the machine it is running on, but it is possible to tell the tools to communicate with a broker on a remote machine, even if it is listening on a non-standard port.

Note: When adding or editing parameters on features, if the parameter is given a blank value ("" in the editor) the condor_configure_store tool will ask if the parameter should use the default value defined in the parameter's metadata.

Note: Group entities have no metadata associated with them


Adding entities to the configuration store

To add entities into the configuration store, the add command is used (-a or --add) and provide the target types and targets desired to add into the store.  It is not possible to add an entity into the store that already exists.

Adding entities into the configuration store will invoke an editor (vi by default or whatever is defined in $EDITOR) to allow the user to enter or edit metadata about the objects. After saving the modifications to the objects, the tool may prompt the user for additional instructions.

Example:
Adding a feature named "feature1" and nodes named "node1" and "node2" to the configuration store:

condor_configure_store -a -f feature1 -n node1,node2

The editor defined by $EDITOR (or vi by default) will be launched with metadata for feature1, node1, and node2.


Editing entities in the configuration store

To edit entities in the configuration store, the edit command is used (-e or --edit) and provide the target types and targets desired to add into the store.  The entities to be edited must already exist in the configuration store.

editing entities in the configuration store will invoke an editor (vi by default or whatever is defined in $EDITOR) to allow the user to enter or edit metadata about the objects. After saving the modifications to the objects, the tool may prompt the user for additional instructions.

Example:
Editing subsystems named subsys1 and subsys2, and a parameter named param1

condor_configure_store -e -s subsys1,subsys2 -p param1

The editor defined by $EDITOR (or vi by default) will be launched with metadata for subsys1, subsys2, and param1


Removing entities in the configuration store

To remove entities in the configuration store, the delete command is used (-d or --delete) and provide the target types and targets desired to add into the store.  The entities to be edited must already exist in the configuration store.

Example:
Removing subsystems named subsys1 and subsys2, a parameter named param1, a feature named "feature1", and nodes named "node1" and "node2" from the configuration store:

condor_configure_store -d -s subsys1,subsys2 -p param1 -f feature1 -n node1,node2

Applying configurations to a node with condor_configure_pool

The condor_configure_pool tool is used to apply the pre-defined entities in the configuration store to physical nodes or groups of node, and to manage configurations within the configuration store.  In addition to modifying which parameters and features are configured on nodes or groups of nodes, the condor_configure_pool tool can activate configurations and manage snapshots within the configuration store.  Only one node or group can be acted upon at a time, but multiple features and parameters can by be acted upon with one invocation of the tool. 

The condor_configure_pool tool does not need to be run on the same machine as
the configuration store, nor the same machine as the broker the configuration
store is communicating with.  The condor_configure_pool tool defaults to
looking for the AMQP broker on the machine it is running on, but it is possible
to tell the tools to communicate with a broker on a remote machine, even if it
is listening on a non-standard port.

Note: If adding parameters to a node or group of nodes, the condor_configure_pool tool will prompt for the parameter values before asking to commit the changes to the configuration store.

All invocations of the condor_configure_pool tool ask if the configuration change should be applied. This is the last chance to abort the operation with no changes being made in the configuration store. If 'y' is answered, the tool will begin to make the changes in the configuration store. If needed, the tool will then prompt for any additional parameters that are required.

The condor_configure_pool tool will then prompt to save the configuration. If 'y' is answered then the tool will prompt for a configuration name and generate a named snapshot in the configuration store with the provided name. Snapshots can be loaded into or removed from the store with the --load-snapshot and --remove-snapshot options.

Finally, the condor_configure_pool tool will ask if the changes should be activated. If 'y' is answered, the tool will tell the configuration store to validate the configuration and, if valid, push out the changes to the nodes in the pool. If the configuration is not valid, the tool will notify the user of the problems and then exit. The tool will automatically create a named snapshot when a configuration is successfully activated in this way.

Note: To modify the setup of the configuration store's Default Group, which affects all nodes the configuration store knows about, use the --default-group option as the target instead of specifying a specific node or group name.


Listing configuration store entities:

The condor_configure_pool tool allows displaying entities in the store and their respective metadata by using the --list-all-<type> commands.  It is possible to list more than 1 type with successive commands, but it is not possible to list and perform any other action in the same command invocation.

Example:
Listing all nodes and subsystems in the configuration store:
condor_configure_pool --list-all-nodes --list-all-subsystems


Applying a configuration change to a node

To apply the feature named "feature1" and parameters named "param1" and "param2" to a node named "node1":

condor_configure_pool -a -n node1 -f feature1 -p param1,param2

The condor_configure_pool pool will ask for values for parameters "param1" and "param2".  Once values have been given, the tool will resume with the regular prompts.


Removing a configuration change from a group

To remove features named "feature1" and "feature2" from group "group1":

condor_configure_pool -d -g group1 -f feature1,feature2


Activating configurations

In addition to activating a configuration change at the time the change is made, the condor_configure_pool tool allows the user to activate a configuration at any time.  This allows multiple changes to be made to configurations within the configuration store, and have all the changes validated and applied at once.  This is accomplished by:

condor_configure_pool --activate

As with all activation requests, the store will validate the configuration and, if valid, push it out to all affected nodes.  If the configuration is not valid, the tool will notify the user of the problems and then exit. The tool will NOT create a named snapshot when a configuration is successfully activated in this way.


Managing snapshots

The condor_configure_pool tool is able to create, apply, and remove named snapshots in the store.  In addition to creating snapshots when configuration changes are made, the tool is able to act upon snapshots without requiring configuration changes.  All actions upon a snapshot require the name of the snapshot.

Creating a snapshot:
condor_configure_pool --take-snapshot "A snapshot name"

Applying a snapshot:
condor_configure_pool --load-snapshot "A snapshot name"

Note: Applying a snapshot does not cause the configuration store to activate the configuration or push changes out to affected nodes.  In order apply the configuration in the snapshot to nodes, a configuration activation must be completed (ie condor_configure_pool --activate)

Removing a snapshot:
condor_configure_pool --remove-snapshot "A snapshot name"

Both added and will be available for review on the stage shortly.

LKB