Bug 632024

(In reply to comment #0)
> Description of problem:
> Missing overview - How does configuration work, how do the different
> configuration sources play a role (before diving into the details).  Suggest
> something like:
> 
> MRG Grid's configuration is determined by MRG Grid looking for configuration
> information in multiple files.  The first configuration file that is read is
> the global configuration file, which must exist for MRG Grid to run.  The
> configuration in the global configuration file can be extended or overwritten
> by configuration files located in the local configuration directory or
> specified by the local configuration file parameter.
> 
> The information in these files is layered upon the configuration information
> already known, with any repeated parameters assuming the value read later in
> the configuration pass.
> 
> MRG Grid's configuration parameters are key-value pairs separated by an equal
> sign.  A space must exist on either side of the equal sign.  Valid
> configuration parameters take the form of:
> name = value
> 

<para>
		Configuration in &GRID; is determined by information held in multiple files across the entire system. The global configuration file is read first, and if it does not exist, &GRID; will not run. The settings in the global configuration file are then extended or overwritten by configuration files located in the local configuration directory or other locations specified by the local configuration file parameter.
	</para>
	<para>
		As &GRID; reads each consecutive configuration file, the information is layered upon the details it already knows. If any parameters are repeated, the information read last overwrites the information read earlier.
	</para>
	<para>
		Configuration parameters are specified using key-value pairs separated by an equals (<parameter>=</parameter>) sign. There must be a space character on each side of the equals sign. Valid configuration parameters look like this:
	</para>
<programlisting>
name = value
</programlisting>
	<para>
		The different configuration files are parsed in the following order:
	</para>


> 
> 
> 1. Global configuration file
>   - Do not locate the file on a shared file system
>   - Mention the file is part of the condor.rpm and is managed by the package
>   - We place it in /etc/condor/condor_config
>   - "MRG Grid will look" etc, discussing who reads the configuration files and
> why in an overview will give context to avoid "MRG Grid will", which is vague
> and misleading

<para>
	A global configuration file is required by all machines in the pool and should be the same across all nodes. This file should not be directly edited. The file is managed by the <filename>condor.rpm</filename> package. An example of what the global configuration file looks like is in <xref linkend="exam-Grid_User_Guide-Example_configuration_files-The_default_global_configuration_file" />. 
</para>
<para>
	&GRID; will look in different places for the global configuration file, in the following order:
</para>
 <orderedlist>
	<listitem>
		<para>
			The filename specified in the <command>CONDOR_CONFIG</command> environment variable
		</para>

	</listitem>
	<listitem>
		<para>
			<filename>/etc/condor/condor_config</filename>
		</para>
		<para>
			&GRID; places the global configuration file here by default
		</para>
	</listitem>


> 2. Local configuration directory
>   - Add (probably 1st sentence in the section): The local configuration
> directory allows for easily extending MRG Grid's configuration by placing files
> inside the directory that contain MRG GRID configuration parameters.

<para>
	The local configuration directory is located at <filename>/etc/condor/config.d</filename>. This location  is defined by the default global configuration file. The local configuration directory provides an easy way to extend the configuration of &GRID; by placing files that contain configuration parameters inside the directory.
</para>

>   - "10 - 40 - user configuration files" - Need to mark this as the range for
> the user to extend MRG Grid's configuration

<listitem>
	<para>
		<parameter>10-40</parameter> - user configuration files
	</para>
	<para>
		Use this range to extend the configuration of &GRID;
	</para>
</listitem>

>  3. Local configuration file
>   - Remove "The location of the local configuration file is stored in the
> global configuration file" 
>   - Remove "Settings in this configuration file will override settings in the
> global file for that machine." (covered in new overview section)
>   - Add "The LOCAL_CONFIG_FILE parameter defines the location of the local
> configuration file.
>   - Add Note: The local configuration file is used by the remote configuration
> feature.  Do not set the LOCAL_CONFIG_FILE parameter if remote configuration is
> to be used.

<listitem>
	<para>
		<emphasis>Local configuration file</emphasis>
	</para>
	<para>
		The <command>LOCAL_CONFIG_FILE</command> parameter in the global configuration file can be used to specify the location of a file or directory to be used as the local configuration file.
	</para>
	<note><title>Note</title>
		<para>
			The <command>LOCAL_CONFIG_FILE</command> parameter is used by the remote configuration feature. Do not set this parameter if remote configuration is used.
		</para>
	</note>
</listitem>

> 
> Changing the configuration in the local configuration file
>   - Env settings are case insensitive, not just _CONDOR_ or _condor_
> 

<para>
	Once &GRID; has completed parsing the four configuration file locations, it will check for environment variables. These configuration variables are case-insensitive, and are prefixed by either <command>_CONDOR_</command> or <command>_condor_</command>. &GRID; parses environment variables last, subsequently any settings made this way will override conflicting settings in the configuration files.
</para>

> 
> Adding entries to configuration files
>   - Should add that comments can only appear at the beginning of a line, e.g.
> "FOO = BAR # comment" won't do what you might expect
> 

<listitem>
	<para>
		A line prefixed by a <command>#</command> symbol will be treated as a comment and ignored. The <command>#</command> symbol can only appear at the beginning of a line. It will not create a comment if it is used in the middle of a line.
	</para>

</listitem>

> 
> Initial configuration
>   - 00personal_condor -> 00personal_condor.config

<para>
	Review the configuration files stored at <filename>/etc/condor/condor_config</filename> and <filename>/etc/condor/config.d/00personal_condor.config</filename> before starting &GRID;. 

Rest is coming ...

LKB

(In reply to comment #0)


>   - Extending from a Personal Condor configuration to one with multiple nodes
> should be promoted to its own section.  Suggest removing everything after
> "Personal Condor is a specific style of installation suited for individual
> users who do not have their own pool of machines." 
>   - If we want to show a configuration change, suggest a simpler change be used
> for the example of how to extend the default configuration -- The default START
> policy will give the user of the machine preference over MRG Grid usage, and
> will not advertise the machine as able to run jobs until the user has been on
> the machine for an extended period of time.  To change this behavior:
> 
> echo "START = TRUE" >> /etc/condor/config.d/10START_policy.config
> 

Structural changes will need to occur in the release. Created BZ #633499.

> 
> Executing a program to produce configuration entries
>   - "will only work with CONDOR_CONFIG, or the configuration variable
> LOCAL_CONFIG_FILE" - It only works with LOCAL_CONFIG_FILE.  CONDOR_CONFIG can
> not be set in this way.
>   - make_the_config doesn't populate the file, it emits it on stdout

 <listitem>
	<para>
		&GRID; can run a specialized program to obtain configuration entries. To run a program from the configuration file, insert a <command>|</command> character at the end of the line. This syntax will only work with the configuration variable <command>LOCAL_CONFIG_FILE</command>. For example, to run a program located at <filename>/bin/make_the_config</filename>, use the following entry:
	</para>
			 
<programlisting>LOCAL_CONFIG_FILE = /bin/make_the_config|
</programlisting>
	<para>
		This will produce a configuration file and show it on the standard output.
	</para>
</listitem>

> 
> 2.1. Configuring MRG Grid for Microsoft Windows
> Executing jobs with the user profile loaded
>    - condor_startd is first introduced here, should appear in the overview
> chapter (BZ631560)
> 
> Allowing access to the user's registry
>    - Introduced to "submit description file" for the first time here, should be
> known from the Overview (BZ631560)

I'll address these in the other BZ.

> 
> Starting and stopping jobs under Microsoft Windows
>    - Nice overview, should be part of the general Overview (BZ631560)

Will address in that BZ.

>     shadow is updated every 5 minutes by default (not 20) and is controlled by
> the STARTER_UPDATE_INTERVAL.  STARTER_UPDATE_INTERVAL is in seconds.

<listitem>
	<para>
		While the job is running, the starter monitors the CPU usage and image size of all processes started by the job. This information is sent to the shadow every five minutes, along with the total size of all files contained in the job&#39;s temporary working directory. The shadow then inserts this information into the job&#39;s ClassAd so that policy and scheduling expressions can make use of the information.
	</para>
	<para>
		The frequency of this check can be adjusted by changing the value (in seconds) of the <command>STARTER_UPDATE_INTERVAL</command> configuration parameter.
	</para>
</listitem>

LKB

(In reply to comment #1)
"specify the location of a file or directory to be used as the local configuration file"  - This is not correct.  The local_config_file is for specifying files with configuration to be read.

Previously there was some wording about having multiple files listed for the local configuration file param that should be re-added.

"This will produce a configuration file and show it on the standard output." - This is not correct.  The program can do whatever it wants, but the only way the configuration will get into condor is if the program emits the configuration parameters on stdout.  Suggest something like "The program /bin/make_the_config" must output the configuration parameters on standard output for the configuration parameters to be included in condor's configuration"

(In reply to comment #3)
> (In reply to comment #1)
> "specify the location of a file or directory to be used as the local
> configuration file"  - This is not correct.  The local_config_file is for
> specifying files with configuration to be read.

<para>
	The <command>LOCAL_CONFIG_FILE</command> parameter in the global configuration file can be used to specify the location of a file or directory with configuration to be read.
</para>

> 
> Previously there was some wording about having multiple files listed for the
> local configuration file param that should be re-added.

I'm not sure what this wording was, sorry.

> 
> "This will produce a configuration file and show it on the standard output." -
> This is not correct.  The program can do whatever it wants, but the only way
> the configuration will get into condor is if the program emits the
> configuration parameters on stdout.  Suggest something like "The program
> /bin/make_the_config" must output the configuration parameters on standard
> output for the configuration parameters to be included in condor's
> configuration"

<para>
	The program <filename>/bin/make_the_config</filename> must output the configuration parameters on standard output for the configuration parameters to be included in the configuration.
</para>

LKB