Bug 237356

Summary: Move DS Admin Code into Admin Server
Product: [Community] 389 Reporter: Noriko Hosoi <nhosoi>
Component: AdminAssignee: Noriko Hosoi <nhosoi>
Status: CLOSED CURRENTRELEASE QA Contact: Viktor Ashirov <vashirov>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.0.4CC: amsharma, nkinder, rmeggins
Target Milestone: ---Keywords: VerifiedUpstream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-07 11:31:28 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 427409    
Attachments:
Description Flags
email discussion
none
cvs diff Makefile.am
none
cvs commit message (Comment #2)
none
cvs commit message (Comment #3)
none
cvs commit message (Comment #4, #6)
none
cvs commit message (Comment #5)
none
cvs commit message (Comment #8)
none
cvs commit message (Comment #8, #9)
none
ldif file for o=netscaperoot base data
none
ldif to create backend and suffix for o=netscaperoot
none
parameterized nsroot.ldif.template
none
parameterized servergroup.ldif.template
none
parameterized dstasks.ldif.template
none
parameterized astasks.ldif.template
none
parameterized globalpreferences.ldif.template
none
parameterized ascommands.ldif.template
none
parameterized userpreferences.ldif.template
none
parameter tables for Comment #19 ~ #25
none
Questions that came up while parameterizing...
none
new Dialog perl module
none
new DialogManager perl module
none
new Inf file perl module
none
new Resource perl module
none
new Setup perl module
none
new SetupDialogs perl module
none
new SetupLog perl module
none
new setup script for ds and as combined
none
cvs diff Makefile.am and new files
none
cvs diff files
none
add setup perl modules to ds
none
cvs diff adminutil.h form_post.c
none
cvs diffs
none
cvs commit log
none
cvs commit log for fixes to new ds setup code
none
diffs to add setup perl support to adminserver
none
diffs to add setup perl support to adminserver
none
cvs commit log for new admin server setup perl code
none
cvs commit message (adminutil -- comment #42)
none
cvs diff ldap/admin/src/cfg_sspt.c
none
cvs commit message
none
cvs commit log
none
cvs diff Makefile.am and new files (including server registration script)
none
cvs commit message (comment #54)
none
cvs commit message (comment #54 -- the adminserver side)
none
cvs commit message (comment #58)
none
cvs diff configure.ac Makefile.am + new slapd.inf.in
none
cvs diff Makefile.am + new setup.inf.in (adminserver)
none
cvs commit message (comment #62, #63)
none
move mapfile and template processing code into DS
none
config ds support in adminserver
none
cvs commit log (comment #65)
none
cvs commit log (comment #66)
none
cvs commit message (comment #70)
none
cvs commit message (comment #72)
none
add ldif templates; pwdhash with no config files
none
file list for Comment #74
none
setup - admserv, config ds, server reg.
none
files for Comment #76
none
template-dse.ldif.in
none
template-suffix-db.ldif.in
none
cvs commit log (Comment #74)
none
cvs commit log (Comment #76)
none
cvs diff and commit messages for small bug fixes
none
fix for section.param=value command line args
none
cvs commit log
none
diffs - support for ca cert for configds
none
cvs commit log
none
cvs commit log (Comment #87)
none
cvs commit log - dsktune none

Description Noriko Hosoi 2007-04-20 21:16:50 EDT
Description of problem:
Attaching email discussion for myself...
Comment 1 Noriko Hosoi 2007-04-20 21:16:50 EDT
Created attachment 153230 [details]
email discussion
Comment 2 Noriko Hosoi 2007-04-25 21:17:45 EDT
Code Cleaning-up (not used)
  config 
  include/nsapi20  [include/nsapi20/base, include/nsapi20/frame]
  include/public/oneacl
  ldap/admin/src/java
  ldap/libraries/libldif       
  ldap/libraries/liblitekey       
  ldap/servers/slapd/libsh_stub 

cvs remove from HEAD
Comment 3 Noriko Hosoi 2007-04-25 21:22:16 EDT
2. NT
  httpd [httpd/src]
  lib/libnt
  include/nt
  ldap/servers/ntds 
  ldap/libraries/libutil       # for nt
  ldap/servers/slapd/ntmsgdll  # for nt
  ldap/servers/slapd/ntperfdll # for nt
  ldap/servers/slapd/ntwdog    # for nt 

cvs remove from fedora cvs HEAD
cvs add to the internal cvs (cvs.devel) ldapserver/...
Comment 4 Noriko Hosoi 2007-04-25 21:24:31 EDT
DSGW
  ldap/clients/dsgw
  ldap/clients/dsgw/admhtml
  ldap/clients/dsgw/config
  ldap/clients/dsgw/config/de
  ldap/clients/dsgw/config/en
  ldap/clients/dsgw/config/en-us
  ldap/clients/dsgw/config/es
  ldap/clients/dsgw/config/fr
  ldap/clients/dsgw/config/ja
  ldap/clients/dsgw/config/ko
  ldap/clients/dsgw/config/zh
  ldap/clients/dsgw/html
  ldap/clients/dsgw/html/de
  ldap/clients/dsgw/html/es
  ldap/clients/dsgw/html/fr
  ldap/clients/dsgw/html/info
  ldap/clients/dsgw/html/ja
  ldap/clients/dsgw/html/manual
  ldap/clients/dsgw/html/manual/ja
  ldap/clients/dsgw/pbconfig
  ldap/clients/dsgw/pbhtml
  ldap/clients/dsgw/userhtml 

> dsgw has already been moved out, so please remove that.
cvs remove from fedora cvs HEAD
Comment 5 Noriko Hosoi 2007-04-25 21:29:34 EDT
DSMLGW
  ldap/clients/dsmlgw
  ldap/clients/dsmlgw/misc
  ldap/clients/dsmlgw/src
  ldap/clients/dsmlgw/src/com
  ldap/clients/dsmlgw/src/com/netscape
  ldap/clients/dsmlgw/src/com/netscape/dsml
  ldap/clients/dsmlgw/src/com/netscape/dsml/gateway
  ldap/clients/dsmlgw/src/com/netscape/dsml/test
  ldap/dsml 

ORGCHART
  ldap/clients/orgchart

> We'll need to create cvs modules for orgchart and dsmlgw.
cvs remove from fedora cvs HEAD
cvs add dsmlgw/*
cvs add dsorgchart/*

Since dsgw is in the top level of cvs tree, it'd be reasonable to have dsmlgw
and orgchart in the level.  I'd like to name the module name of orgchart
"dsorgchart".
Comment 6 Noriko Hosoi 2007-04-25 21:35:15 EDT
To remove dsgw from the ldapserver tree, needs to eliminate the dependency from
this file.  (that implies to localize dsgw, we may need to move out the libsi18n
and rebuild it for dsgw and install the lib with dsgw???)

Index: ldapserver/lib/libsi18n/gsslapd.h
===================================================================
RCS file: /cvs/dirsec/ldapserver/lib/libsi18n/gsslapd.h,v
retrieving revision 1.6
diff -t -w -U4 -r1.6 gsslapd.h
--- ldapserver/lib/libsi18n/gsslapd.h   10 Nov 2006 23:46:05 -0000      1.6
+++ ldapserver/lib/libsi18n/gsslapd.h   26 Apr 2007 01:30:51 -0000
@@ -51,15 +51,13 @@
 #include "libaccess/dbtlibaccess.h"
 #undef LIBRARY_NAME
 #include "libadmin/dbtlibadmin.h"
 #undef LIBRARY_NAME
-#include "../ldap/clients/dsgw/dbtdsgw.h"

 static RESOURCE_GLOBAL allxpstr[] = {
   base,
   libaccess,
   libadmin,
-  dsgw,
   0
 };
Comment 7 Noriko Hosoi 2007-04-26 18:53:15 EDT
Imported dsmlgw, orgchart to cvs.fedora
 cvs -d <fedora_cvs> co dsmlgw
 cvs -d <fedora_cvs> co dsorgchart

Imported nt modules to cvs.devel with the release tag LDAPSERVER_NT
 cvs -d <devel_cvs> co -r LDAPSERVER_NT ldapserver

Note: Nothing has been done to build the components yet...
Comment 8 Noriko Hosoi 2007-04-26 19:23:42 EDT
Created attachment 153562 [details]
cvs diff Makefile.am

Removing ldapserver/lib/libsi18n/acclanglist.c from ldapserver.

Background info:
Richard Megginson wrote:
> Noriko Hosoi wrote:
>> Richard Megginson wrote:
>>> I think I already moved the parts of libsi18n into adminutil - see the
latest adminutil source code, especially resource.c and acclanglist.c.	So we
can either use adminutil or just copy the code into dsgw (if we don't want to
have a dependence on adminutil).
>>
>> Great!  Let me double-check...  I searched acclanglist.c in the global
source root, then I found it in 3 places:
>> ./adminutil/lib/libadminutil
>> ./adminserver/lib/libsi18n
>> ./ldapserver/lib/libsi18n
>> They all look alike.  (Some has a newer fix, some has an older copyright,
but the main codes are almost identical.)  Do we keep them in each component?
>
> No.  We should only be using the one in adminutil.  I will clean up
adminserver/libsi18n as part of my FHS work, and we should get rid of the
ldapserver/libsi18n one when we move the dsadmin code out of ds into
adminserver.

All right.  I'll add ldapserver/lib/libsi18n/acclanglist.c to the to-be-removed

list.
Comment 9 Noriko Hosoi 2007-04-26 19:45:54 EDT
More files not being used:
  modules.sh
  modules.awk

Need to be moved out, as well?
  ldapserver/ldap/admin/src/migratedsgw
  ldapserver/ldap/admin/src/updatedsgw
Comment 10 Rich Megginson 2007-04-27 10:20:32 EDT
(In reply to comment #9)
> More files not being used:
>   modules.sh
>   modules.awk
> 
> Need to be moved out, as well?
>   ldapserver/ldap/admin/src/migratedsgw
>   ldapserver/ldap/admin/src/updatedsgw

Yes, I think these would need to move into the dsgw - we'll need some sort of
migration script to migrate from the bundled dsgw to the "standalone" dsgw.
Comment 11 Noriko Hosoi 2007-04-27 17:55:10 EDT
Created attachment 153667 [details]
cvs commit message (Comment #2)
Comment 12 Noriko Hosoi 2007-04-27 18:19:28 EDT
Created attachment 153668 [details]
cvs commit message (Comment #3)
Comment 13 Noriko Hosoi 2007-04-27 19:01:45 EDT
Created attachment 153674 [details]
cvs commit message (Comment #4, #6)
Comment 14 Noriko Hosoi 2007-04-27 19:19:38 EDT
Created attachment 153675 [details]
cvs commit message (Comment #5)
Comment 15 Noriko Hosoi 2007-04-27 19:41:40 EDT
Created attachment 153677 [details]
cvs commit message (Comment #8)
Comment 16 Noriko Hosoi 2007-04-27 19:49:17 EDT
Created attachment 153678 [details]
cvs commit message (Comment #8, #9)
Comment 17 Rich Megginson 2007-05-21 12:02:13 EDT
Created attachment 155103 [details]
ldif file for o=netscaperoot base data
Comment 18 Rich Megginson 2007-05-21 12:02:49 EDT
Created attachment 155105 [details]
ldif to create backend and suffix for o=netscaperoot
Comment 19 Noriko Hosoi 2007-05-24 20:40:00 EDT
Created attachment 155406 [details]
parameterized nsroot.ldif.template

Parameterized the NetscapeRoot entries.
Comment 20 Noriko Hosoi 2007-05-24 20:55:00 EDT
Created attachment 155407 [details]
parameterized servergroup.ldif.template
Comment 21 Noriko Hosoi 2007-05-24 21:00:58 EDT
Created attachment 155408 [details]
parameterized dstasks.ldif.template
Comment 22 Noriko Hosoi 2007-05-24 21:01:43 EDT
Created attachment 155409 [details]
parameterized astasks.ldif.template
Comment 23 Noriko Hosoi 2007-05-24 21:04:27 EDT
Created attachment 155410 [details]
parameterized globalpreferences.ldif.template
Comment 24 Noriko Hosoi 2007-05-24 21:05:30 EDT
Created attachment 155411 [details]
parameterized ascommands.ldif.template
Comment 25 Noriko Hosoi 2007-05-24 21:06:58 EDT
Created attachment 155412 [details]
parameterized userpreferences.ldif.template
Comment 26 Noriko Hosoi 2007-05-24 21:10:08 EDT
Created attachment 155413 [details]
parameter tables for Comment #19 ~ #25
Comment 27 Noriko Hosoi 2007-05-24 21:15:45 EDT
Note: the template files should be rearranged to have one set for a DS instance
and another for an AS instance...  (e.g., all the entries having %dsid% is one
file; %asid% in another file)
Comment 28 Noriko Hosoi 2007-05-24 21:20:24 EDT
Created attachment 155414 [details]
Questions that came up while parameterizing...

The idea is writing a simple perl script to replacing the parameters in the
ldif template files with the given values via inf files and add the ldif
entries to the Configuration DS.

* If the entries in o=netscaperoot already exist, it'd do modify instead of
add.
Comment 29 Rich Megginson 2007-05-25 18:45:31 EDT
Created attachment 155483 [details]
new Dialog perl module
Comment 30 Rich Megginson 2007-05-25 18:45:51 EDT
Created attachment 155484 [details]
new DialogManager perl module
Comment 31 Rich Megginson 2007-05-25 18:46:42 EDT
Created attachment 155485 [details]
new Inf file perl module
Comment 32 Rich Megginson 2007-05-25 18:47:42 EDT
Created attachment 155486 [details]
new Resource perl module
Comment 33 Rich Megginson 2007-05-25 18:48:00 EDT
Created attachment 155487 [details]
new Setup perl module
Comment 34 Rich Megginson 2007-05-25 18:49:23 EDT
Created attachment 155488 [details]
new SetupDialogs perl module
Comment 35 Rich Megginson 2007-05-25 18:49:45 EDT
Created attachment 155489 [details]
new SetupLog perl module
Comment 36 Rich Megginson 2007-05-25 18:50:16 EDT
Created attachment 155490 [details]
new setup script for ds and as combined
Comment 37 Noriko Hosoi 2007-06-06 16:11:28 EDT
Created attachment 156389 [details]
cvs diff Makefile.am and new files

Modifiled File:
  Makefile.am
Change description:
  - Added following new files to install
  - Added PACKAGE_BASE_NAME and helpdir to the fixupcmd to substitute
    in the build

New Files:
  admserv/newinst/src/register_param.map.in
    --- parameter map file used by register_server.pl to resolve the %...%
	format parameters in the template ldif files.
  admserv/newinst/src/register_server.pl.in
    --- script to resolve the parameters in the template ldif files and add
	the server info entries to the Configuration Directory Server.
	This script is supposed to be called after the server instance
	creation.
  admserv/schema/ldif/00nsroot_backend.ldif
  admserv/schema/ldif/01nsroot.ldif.tmpl
  admserv/schema/ldif/02globalpreferences.ldif.tmpl
  admserv/schema/ldif/10dsdata.ldif.tmpl
  admserv/schema/ldif/11dstasks.ldif.tmpl
  admserv/schema/ldif/20asdata.ldif.tmpl
  admserv/schema/ldif/21astasks.ldif.tmpl
  admserv/schema/ldif/22ascommands.ldif.tmpl
    --- (template) ldif files

[Issues]
1. We need to decide whether we give the appropriate values for these
parameters  or eliminate them.
configroot =	"CONFIG ROOT -- replace me"
as_installedlocation =	"AS INSTALLED LOCATION -- replace me"
as_serverroot = "AS SERVER ROOT-- replace me"
ds_installedlocation =	"DS INSTALLED LOCATION -- replace me"
ds_serverroot = "DS SERVER ROOT-- replace me"

2. The register_server.pl script depends upon the Perl SetupUtil module.  How
the module is going to be installed?  Is it a part of the Admin Server or an
independent package?
perl -I/path/to/perl/setuputil /usr/bin/register_server.pl -m
/usr/share/fedora-ds/inf/register_param.map -w password
/usr/share/fedora-ds/data/[0-9][0-9]*ldif*
Comment 38 Noriko Hosoi 2007-06-07 17:08:58 EDT
Created attachment 156510 [details]
cvs diff files

Modified Files:
Makefile.am
ldap/admin/src/
	       addindex.c
	       create_instance.c
	       create_instance.h
	       ds_bak2db.c
	       ds_db2bak.c
	       ds_db2ldif.c
	       ds_ldif2db.c
	       ds_remove.c
	       ds_rmdb.c
	       ds_snmpctrl.c
	       init_ds_env.c
	       init_ds_env.h
	       instindex.cpp
	       restart.c
	       shutdown.c
	       start.c
	       vlvindex.c

New Files:
wrappers/ds_addindex.in  wrappers/ds_ldif2db.in  wrappers/ds_rmdb.in
wrappers/ds_bak2db.in	 wrappers/ds_listdb.in	 wrappers/ds_shutdown.in
wrappers/ds_snmpctrl.in
wrappers/ds_db2bak.in	 wrappers/ds_remove.in	 wrappers/ds_start.in
wrappers/ds_db2ldif.in*   wrappers/ds_restart.in*  wrappers/ds_vlvindex.in*

Obsolete Files:
cfg_sspt.c  cfg_sspt.h	configure_instance.cpp	configure_instance.h


DS task entries require CGIs to launch the tasks:
$ egrep nsExecRef 11dstasks.ldif.tmpl | egrep -v perl
nsExecRef: ds_start
nsExecRef: ds_shutdown
nsExecRef: ds_restart
nsExecRef: ds_db2bak
nsExecRef: ds_bak2db
nsExecRef: ds_db2ldif
nsExecRef: ds_ldif2db
nsExecRef: ds_listdb
nsExecRef: ds_remove
nsExecRef: ds_vlvindex
nsExecRef: ds_addindex
nsExecRef: ds_snmpctrl
nsExecRef: ds_newinst

The attached diffs enable the build for the CGIs.
Some changes are being made.
1) removing configuration code from create_instance.c
2) removing the dependency on the AdminUtil.  It looks these CGIs do not use of
AdminUtil functionality, but just calling ADMUTIL_Init.  I replaced it with
PR_Init, which is done in ADMUTIL_Init.

A question about ds_create.  The instance creation task entry: "cn=Create,
cn=Operation, cn=Tasks, cn=%brand% Directory Server, cn=Server Group,
cn=%fqdn%, ou=%domain%, o=NetscapeRoot" used to refer ds_create.  I'm replacing
ds_create with ds_newinst in the dstask ldif template and I did't enabled
ds_create build in ldap/admin/src.  I hope the CGI mode of ds_newinst works in
the same way as ds_create does...
Comment 39 Rich Megginson 2007-06-07 18:23:54 EDT
Although the DS code does not use adminutil, it does use almost identical
functions in ldap/admin/lib e.g. ds_get_cgi_var() instead of get_cgi_var(), etc.
 Perhaps we should just use adminutil instead, to simplify code maintenance,
potential security problems, etc.

Just add the pass through auth plugin, but disabled, with a "specify suffix" or
something like that for the suffix.  When this directory server instance is
configured to be managed by admin server/console, that registration code will
need to enable the pta plugin and add the o=NetscapeRoot suffix.

cfg_sspt.c does some stuff in addition to creating the o=NetscapeRoot tree.  It
creates the entry for the default suffix specified during instance creation.  It
adds some default ACIs to the new instance to allow it to be managed by the
console/admin server.  If we remove cfg_sspt.c, we'll have to add this
functionality elsewhere.

ds_create does some stuff in addition to what ds_newinst does.  ds_create will
register the new instance with the config DS, and set some values and ACIs in
the new instance which allows it to be managed by the console/admin server.  So
we will need to add that code elsewhere (server register script?).
Comment 40 Noriko Hosoi 2007-06-07 18:32:50 EDT
Comment on attachment 156510 [details]
cvs diff files

Thank you for the comments, Rich.  I'll go through the changes following your
comments.
Comment 41 Rich Megginson 2007-06-07 18:57:55 EDT
Created attachment 156525 [details]
add setup perl modules to ds
Comment 42 Noriko Hosoi 2007-06-07 21:23:04 EDT
Created attachment 156539 [details]
cvs diff adminutil.h form_post.c

Files:
include/libadminutil/admutil.h
lib/libadminutil/form_post.c

Fix description:
In the effort of using adminutil instead of the each server's local library
(e.g., libds_admin), post_begin equivalent used in the DS code checks the
return value from the function.  Changing the return type from void to int to
adjust.
Comment 43 Noriko Hosoi 2007-06-07 21:33:10 EDT
Created attachment 156540 [details]
cvs diffs

Modified Files:
  configure.ac
  Makefile.am
  ldap/admin/src/addindex.c
  ldap/admin/src/cfg_sspt.c
  ldap/admin/src/create_instance.c
  ldap/admin/src/ds_bak2db.c
  ldap/admin/src/ds_db2bak.c
  ldap/admin/src/ds_db2ldif.c
  ldap/admin/src/ds_ldif2db.c
  ldap/admin/src/ds_remove.c
  ldap/admin/src/ds_rmdb.c
  ldap/admin/src/ds_snmpctrl.c
  ldap/admin/src/init_ds_env.c
  ldap/admin/src/vlvindex.c

New Files:
  m4/adminutil.m4 <== borrowed from the Admin Server

  wrappers/ds_addindex.in  wrappers/ds_listdb.in   wrappers/ds_shutdown.in
  wrappers/ds_bak2db.in     wrappers/ds_snmpctrl.in
  wrappers/ds_db2bak.in    wrappers/ds_remove.in   wrappers/ds_start.in
  wrappers/ds_db2ldif.in   wrappers/ds_restart.in  wrappers/ds_vlvindex.in
  wrappers/ds_ldif2db.in   wrappers/ds_rmdb.in

Deleted Files:
  ldap/admin/lib/dsalib_html.c
  ldap/admin/src/configure_instance.cpp
  ldap/admin/src/configure_instance.h

Description:
1. added a dependency to adminutil
2. replaced functions from dsalib_html.c with the corresponding one in
adminutil
3. eliminated dsalib_html.c
4. enabled building cgi tasks
5. removed adding o=netscaperoot code from cfg_sspt.c
Comment 44 Rich Megginson 2007-06-07 21:42:30 EDT
Created attachment 156541 [details]
cvs commit log

Reviewed by: nhosoi (Thanks!)
Fix Description: This adds the setup related perl modules, scripts, and
resource files to the DS base code.  This will allow a user to interactively
setup (create an instance of) a directory server.  This will also form the base
of the work to add the console and admin server related setup code.
New files/directories:
$libdir/fedora-ds/perl - this is where the perl modules (Setup.pm, etc.) will
be installed.
$bindir/setup-ds.pl - the script to use to interactively create an instance of
directory server.  This has use lib '$libdir/fedora-ds/perl' hard coded into it
at build time, in order to find the "private" setup perl modules.  If you
invoke this script in silent mode (setup-ds.pl -s) then it is exactly the same
as just using ds_newinst.pl.
$sysconfdir/fedora-ds/property/setup-ds.res - Resources for setup-ds.pl and the
associated modules.
I also fixed a problem with the libns-dshttpd linkage.
Platforms tested: RHEL4
Flag Day: no
Doc impact: Yes.  All of these new items will need to be documented.
Comment 45 Rich Megginson 2007-06-07 21:57:42 EDT
(In reply to comment #43)
> Description:
> 1. added a dependency to adminutil

I thought the plan was to move ds admin code into the adminserver?

> 2. replaced functions from dsalib_html.c with the corresponding one in
> adminutil

This is fine except for create_instance.c - this introduces a dependency on
adminutil to the base ds.  It is rather unfortunate that this program
(ds_newinst) in the core ds package is so dependent on CGI functionality.  It's
just begging for a rewrite with parameterized LDIF files and perl scripts . . .

ds_create could still be a CGI wrapper around ds_newinst.

> 3. eliminated dsalib_html.c
> 4. enabled building cgi tasks

See above.

> 5. removed adding o=netscaperoot code from cfg_sspt.c

Looks good.
Comment 46 Noriko Hosoi 2007-06-08 12:41:00 EDT
Thank you, Rich!

(In reply to comment #45)
> (In reply to comment #43)
> > Description:
> > 1. added a dependency to adminutil
>
> I thought the plan was to move ds admin code into the adminserver?
Oops, that's right!  Since these CGIs are DS specific (ds_ldif2db, ds_bak2db,
etc.), I was somehow thinking they belong to the DS tree.  But they are strictly
for the administration, so it makes sense to move them to the adminserver.  I'll
move them to the adminserver.

> > 2. replaced functions from dsalib_html.c with the corresponding one in
> > adminutil
>
> This is fine except for create_instance.c - this introduces a dependency on
> adminutil to the base ds.  It is rather unfortunate that this program
> (ds_newinst) in the core ds package is so dependent on CGI functionality.  It's
> just begging for a rewrite with parameterized LDIF files and perl scripts . . .

Right...  I'll back off the changes for now (resurrecting dsalib_html.c and
letting create_intance.c call the APIs...).  I also think maintaining
create_instance.c is getting a big pain.  Shall we open a bug for that, at least?
Comment 47 Rich Megginson 2007-06-08 12:47:37 EDT
(In reply to comment #46)
> Oops, that's right!  Since these CGIs are DS specific (ds_ldif2db, ds_bak2db,
> etc.), I was somehow thinking they belong to the DS tree.  But they are strictly
> for the administration, so it makes sense to move them to the adminserver.  I'll
> move them to the adminserver.

Yes.  The new "adminserver" package is going to be a "DS Admin" package, so it
is ok to put DS CGIs and other CGI related code in it.

> > This is fine except for create_instance.c - this introduces a dependency on
> > adminutil to the base ds.  It is rather unfortunate that this program
> > (ds_newinst) in the core ds package is so dependent on CGI functionality.  It's
> > just begging for a rewrite with parameterized LDIF files and perl scripts . . .
> 
> Right...  I'll back off the changes for now (resurrecting dsalib_html.c and
> letting create_intance.c call the APIs...).  I also think maintaining
> create_instance.c is getting a big pain.  Shall we open a bug for that, at least?

Sure.  I don't know if we will be able to do it.  But I've already got a
template-config.ldif and template-userroot.ldif in case we do.
Comment 48 Rich Megginson 2007-06-08 16:37:31 EDT
Created attachment 156612 [details]
cvs commit log for fixes to new ds setup code

Fix Description: The Resource class needs to support more than 1 resource file
e.g. for ds-base and ds-admin.
The property dir should be under $datadir.  Property files are data files, not
really config files.
Added a shared_lib_suffix token
Fixed some wording errors in the resource file.
Platforms tested: RHEL4
Flag Day: no
Doc impact: No new doc impact from previous commits for this bug.
Comment 49 Rich Megginson 2007-06-08 16:45:39 EDT
Created attachment 156613 [details]
diffs to add setup perl support to adminserver
Comment 50 Rich Megginson 2007-06-08 16:51:53 EDT
Created attachment 156614 [details]
diffs to add setup perl support to adminserver
Comment 51 Nathan Kinder 2007-06-08 18:21:35 EDT
Looks good.
Comment 52 Rich Megginson 2007-06-08 18:57:32 EDT
Created attachment 156626 [details]
cvs commit log for new admin server setup perl code

Reviewed by: nkinder (Thanks!)
Files: see diff + new files:
adminserver/admserv/newinst/src/ASDialogs.pm.in
adminserver/admserv/newinst/src/setup-ds-admin.pl.in
adminserver/admserv/newinst/src/setup-ds-admin.res.in
Branch: HEAD
Fix Description: This adds the admin server specific setup "ui" and main script
driver.  Some of the values are currently missing because they don't yet have
ui support.  These are commented out in setup-ds-admin.pl.  But at least this
gives us the framework to add support for config DS creation, server
registration, and other post config stuff.
Platforms tested: RHEL4
Flag Day: no
Doc impact: Yes, along with the rest of the new setup stuff.
Comment 53 Noriko Hosoi 2007-06-08 20:29:00 EDT
Created attachment 156633 [details]
cvs commit message (adminutil -- comment #42)

Reviewed by Rich (Thank you!).

Checked in into HEAD.
Comment 54 Noriko Hosoi 2007-06-08 20:37:01 EDT
Created attachment 156634 [details]
cvs diff ldap/admin/src/cfg_sspt.c

Modified File: ldapserver/ldap/admin/src/cfg_sspt.c

Files moved to the adminserver:
  ldap/admin/src/ds_bak2db.c
  ldap/admin/src/ds_db2bak.c
  ldap/admin/src/ds_db2ldif.c
  ldap/admin/src/ds_ldif2db.c
  ldap/admin/src/ds_listdb.c
  ldap/admin/src/ds_remove.c
  ldap/admin/src/ds_rmdb.c
  ldap/admin/src/ds_snmpctrl.c
  ldap/admin/src/vlvindex.c
  ldap/admin/src/addindex.c
  ldap/admin/src/start.c
  ldap/admin/src/restart.c
  ldap/admin/src/shutdown.c

Changes:
1) eliminated the code adding o=netscaperoot related entries from cfg_sspt.c.
2) moving DS task CGIs to the adminserver.
Comment 55 Noriko Hosoi 2007-06-11 13:24:05 EDT
Created attachment 156741 [details]
cvs commit message

Reviewed by Rich (Thank you!)

Checked in into HEAD.
Comment 56 Noriko Hosoi 2007-06-11 13:25:16 EDT
Comment on attachment 156741 [details]
cvs commit message

Oops!  Sorry.  Wrong bug... :p
Comment 57 Rich Megginson 2007-06-12 17:09:28 EDT
Created attachment 156829 [details]
cvs commit log

Fix Description: 1) Need to have $SILENT be greater than $CUSTOM so that dialog
hiding works properly.
2) Need to have the ability to hide or show individual prompts in a dialog e.g.
if using TLS/SSL, need to ask for the CA certificate filename, otherwise, not.
3) Need the ability to call a function to get the default yes or no answer for
DialogYesNo
4) DialogYesNo should match answer case insensitively
Comment 58 Noriko Hosoi 2007-06-12 19:40:10 EDT
Created attachment 156834 [details]
cvs diff Makefile.am and new files (including server registration script)

Enhanced the proposal in the Comment #38.

[New functionalities]
1. support registering multiple DSes
   -i "setup0000.inf setup0001.inf setup0002.inf ..."
2. added a fresh registeration option "-F"
   by default, addition mode
3. tighter error checking and better error reporting

# Usage: register_server.pl [ -h <host> ] [ -p <port> ] [ -D <rootdn> ] \
#			    -w <rootdnpw> [ -d <default_infdir> ] \
#			    -i <inffile(s)> -m <mapfile> <ldiffile> ...
#
# Description: Store server info stored in the ldiffiles to the Configuration
#	       Directory Server replacing the macros with the defined values
#	       in the map file.
#
# -h <host>: configuration server host (localhost, by default)
# -p <port>: configuration server port (389)
# -D <rootdn>: configuration server's rootdn ("cn=Directory Manager")
# -w <rootdnpw>: configuration server's rootdn password
# -d <default_infdir>: the directory where static .inf files are located
#		       ("/usr/share/fedora-ds/inf")
# -i <inffile(s)>: dynamic .inf file(s)
# -m <mapfile>: map file name
# <ldiffile> ...: ldif file(s) or template ldif file(s) to be stored in
#		  the Configuration Directory Server
Comment 59 Noriko Hosoi 2007-06-12 21:28:24 EDT
Created attachment 156839 [details]
cvs commit message (comment #54)

Reviewed by Nathan (Thank you!!)

Checked in into HEAD.
Comment 60 Noriko Hosoi 2007-06-12 21:40:23 EDT
Created attachment 156840 [details]
cvs commit message (comment #54 -- the adminserver side)

Note: libdsa has been copied from ldapserver/ldap/admin/{include,lib} to build
DS CGI task programs.
Comment 61 Noriko Hosoi 2007-06-13 13:50:27 EDT
Created attachment 156895 [details]
cvs commit message (comment #58)

Reviewed by Rich (Thank you!!)

Checked in into HEAD.
Comment 62 Noriko Hosoi 2007-06-14 16:54:43 EDT
Created attachment 157043 [details]
cvs diff configure.ac Makefile.am + new slapd.inf.in

Files:
  configure.ac, Makefile.am
  ldap/cm/newinst/slapd.inf.in

Description: register_server.pl needs slapd.inf to gather the server info.  To
pass some build specific info such as BuildNumber, I'd like to let the autoconf
process it in the ldapserver build.
Comment 63 Noriko Hosoi 2007-06-14 16:59:31 EDT
Created attachment 157044 [details]
cvs diff Makefile.am + new setup.inf.in (adminserver)

Files:
  Makefile.am
  admserv/newinst/common/setup.inf.in

Description: register_server.pl expects to have setup.inf in $infdir.
Comment 64 Noriko Hosoi 2007-06-15 13:07:59 EDT
Created attachment 157133 [details]
cvs commit message (comment #62, #63)

Reviewed by Rich (Thank you!!)

Checked in into HEAD.
Comment 65 Rich Megginson 2007-06-15 17:01:01 EDT
Created attachment 157159 [details]
move mapfile and template processing code into DS
Comment 66 Rich Megginson 2007-06-15 17:08:11 EDT
Created attachment 157160 [details]
config ds support in adminserver
Comment 67 Noriko Hosoi 2007-06-15 18:00:39 EDT
Both diffs in the comment #65 and #66 look good.

Please check them in.  And if you could give us the quick instructions how to
run the new tools, I'd be more than happy to test them!
Comment 68 Rich Megginson 2007-06-15 18:18:30 EDT
Created attachment 157164 [details]
cvs commit log (comment #65)

Commit for comment #65
Reviewed by: nhosoi (Thanks!)
Fix Description: 1) Since we moved the o=NetscapeRoot code out of cfg_sspt.c,
we no longer need to create the suffix and backend in create_instance.c
2) Added code to enable/disable dialogs e.g. for dialogs that can change the
flow conditionally
3) Added code to allow the user to backup to the first prompt on a dialog, for
dialogs with many prompts
4) Allow continuation lines in Resource files, instead of having to have
embedded \n chars.  This allows easier editing and layout.
5) Added an addSuffix function
6) Moved the register_servers.pl code from admin server into DS Util.pm and
made it a little more general purpose.
Platforms tested: RHEL4
Comment 69 Rich Megginson 2007-06-15 18:19:45 EDT
Created attachment 157165 [details]
cvs commit log (comment #66)

commit for comment #66
Reviewed by: nhosoi (Thanks!)
Fix Description: Move the code out of register_servers.pl in to the DS Util.pm
module.
Added the ConfigDSDialogs.
Added code to create the Config DS based on the register_servers code.
Platforms tested: RHEL4
Comment 70 Noriko Hosoi 2007-06-15 18:27:04 EDT
[Problem Description]
Noriko Hosoi wrote:
> I remember once you taught me the Admin CGIs do not have to set
LD_LIBRARY_PATH since it's set in an admin server configuration file.  Could you
tell me again which file and which directive they are?  I think my config is
missing it and I cannot find the email you gave me before... :(
>
>     [Fri Jun 15 13:14:34 2007] [error] [client 172.16.25.156]
>     /export/servers/ds72/lib/fedora-ds/cgi-bin/htmladmin:
>     /usr/lib/libnss3.so: version `NSS_3.11.1' not found (required by
>     /usr/lib/libssldap60.so), referer:
>     http://laputa.sfbay.redhat.com:11110/bin/admin/admin/bin/download 

Richard Megginson wrote:
I think we will have to add it.  SetEnv LD_LIBRARY_PATH @nsprdir@:@nssdir@:....
etc. in admserv.conf - adminserver/admserv/cfgstuff/admserv.conf.in 

[Proposed Fix]
Index: admserv/cfgstuff/admserv.conf.in
===================================================================
RCS file: /cvs/dirsec/adminserver/admserv/cfgstuff/admserv.conf.in,v
retrieving revision 1.4
diff -t -w -U4 -r1.4 admserv.conf.in
--- admserv/cfgstuff/admserv.conf.in    15 May 2007 16:45:44 -0000      1.4
+++ admserv/cfgstuff/admserv.conf.in    15 Jun 2007 22:16:07 -0000
@@ -28,8 +28,10 @@
 ScriptAlias /bin/admin/admin/bin/ "@cgibindir@/"
 ScriptAlias /dist/ "@cgibindir@/"
 ScriptAlias /manual/help/ "@cgibindir@/"

+SetEnv LD_LIBRARY_PATH @nss_libdir@:@nspr_libdir@
+
 # all access is explicitly denied by default in httpd.conf
 # the following Directory directives turn on access for specific
 # directories
 <Directory "@htmldir@/">

[Generated Diff]
+SetEnv LD_LIBRARY_PATH /usr/lib/dirsec:/usr/lib/dirsec
+
Comment 71 Noriko Hosoi 2007-06-15 18:37:49 EDT
Created attachment 157167 [details]
cvs commit message (comment #70)

Reviewed by Rich (Thanks!!)

Checked in into HEAD.
Comment 72 Noriko Hosoi 2007-06-18 13:12:41 EDT
> I tested standalone register_server.pl and had this error:
>
>     $ cd /export/servers/ds72/share/fedora-ds/data
>     $ perl /export/servers/ds72/bin/register_server.pl -v -p 11111 -w
>     Secret123 -d /export/servers/ds72/share/fedora-ds/ldif -m
>     /export/servers/ds72/share/fedora-ds/inf/register_param.map -i
>     ~/tmp/laputa.inf *.tmpl
>     Undefined subroutine &main::hostfqdn called at
>     /export/servers/ds72/bin/register_server.pl line 102.
>
> These changes fix the problem.  Could you please review them?  Regarding
"cn=NetscapeRoot, cn=ldbm database, cn=plugins, cn=config", it happened to be
added in the create_instance.  Now, it's not (, which is correct).  Thus, this
file needs to include it (properly).
>
>     Index: admserv/newinst/src/register_server.pl.in
>     ===================================================================
>     RCS file:
>     /cvs/dirsec/adminserver/admserv/newinst/src/register_server.pl.in,v
>     retrieving revision 1.2
>     diff -t -w -U4 -r1.2 register_server.pl.in
>     --- admserv/newinst/src/register_server.pl.in   15 Jun 2007
>     22:16:28 -0000
>     1.2
>     +++ admserv/newinst/src/register_server.pl.in   16 Jun 2007
>     01:17:33 -0000
>     @@ -57,9 +57,9 @@
>
>      use lib "@perldir@";
>
>      use Getopt::Std;
>     -use Net::Domain qw(hostname);
>     +use Net::Domain qw(hostname hostfqdn);
>      # PERLDAP modules
>      use Mozilla::LDAP::Conn;
>      # Setup Inf module
>      use Inf;
>     Index: admserv/schema/ldif/00nsroot_backend.ldif.tmpl
>     ===================================================================
>     RCS file:
>     /cvs/dirsec/adminserver/admserv/schema/ldif/00nsroot_backend.ldif.tmpl,v
>     retrieving revision 1.1
>     diff -t -w -U4 -r1.1 00nsroot_backend.ldif.tmpl
>     --- admserv/schema/ldif/00nsroot_backend.ldif.tmpl      13 Jun
>     2007 17:48:35 -0000      1.1
>     +++ admserv/schema/ldif/00nsroot_backend.ldif.tmpl      16 Jun
>     2007 01:17:33 -0000
>     @@ -1,4 +1,14 @@
>     +dn: cn=NetscapeRoot, cn=ldbm database, cn=plugins, cn=config
>     +objectClass: top
>     +objectClass: extensibleObject
>     +objectClass: nsBackendInstance
>     +cn: NetscapeRoot
>     +nsslapd-suffix: o=NetscapeRoot
>     +nsslapd-cachesize: -1
>     +nsslapd-cachememsize: 10485760
>     +nsslapd-directory: NetscapeRoot
>     +
>      dn: cn="o=NetscapeRoot",cn=mapping tree,cn=config
>      objectClass: top
>      objectClass: extensibleObject
>      objectClass: nsMappingTree 
Comment 73 Noriko Hosoi 2007-06-18 13:15:48 EDT
Created attachment 157298 [details]
cvs commit message (comment #72)

Reviewed by Rich (Thank you!)

Checked in into HEAD.
Comment 74 Rich Megginson 2007-06-19 11:29:24 EDT
Created attachment 157378 [details]
add ldif templates; pwdhash with no config files
Comment 75 Rich Megginson 2007-06-19 11:30:51 EDT
Created attachment 157381 [details]
file list for Comment #74
Comment 76 Rich Megginson 2007-06-19 11:58:18 EDT
Created attachment 157388 [details]
setup - admserv, config ds, server reg.
Comment 77 Rich Megginson 2007-06-19 12:02:48 EDT
Created attachment 157390 [details]
files for Comment #76
Comment 78 Noriko Hosoi 2007-06-19 13:46:15 EDT
(In reply to comment #74)
> Created an attachment (id=157378) [edit]
> add ldif templates; pwdhash with no config files
> 
Could you attach these new files, too?
ldapserver/ldap/ldif/template-dse.ldif.in
ldapserver/ldap/ldif/template-suffix-db.ldif.in
Comment 79 Rich Megginson 2007-06-19 13:48:56 EDT
Created attachment 157400 [details]
template-dse.ldif.in
Comment 80 Rich Megginson 2007-06-19 13:49:27 EDT
Created attachment 157401 [details]
template-suffix-db.ldif.in
Comment 81 Noriko Hosoi 2007-06-19 14:13:03 EDT
Thank you, Rich.  Attachments in the comment #74, #79 and #80 look good.
Comment 82 Rich Megginson 2007-06-19 14:26:31 EDT
Created attachment 157407 [details]
cvs commit log (Comment #74)

Reviewed by: nhosoi (Thanks!)
Fix Description: These changes are primarily to allow the admin server setup to
run completely in perl with no more setuputil code.
1) Added LDIF templates for DS config.	template-dse.ldif is the core minimal
directory server configuration.  Values can be replaced with parameters in the
same style as used with register_server.pl - %token%.  For the plugin entries,
the plugin shared library name is now just a name.  There is no more full path.
 The code in dynalib.c handles this case by using the compiled in PLUGINDIR. 
The NSPR function PR_GetLibraryName knows the correct shared lib suffix for the
platform.   All of this allows us to do 2).
2) Added ability to run pwdhash with no server configuration.  If no
configuration is given, it uses the template-dse.ldif above.  And instead of
having to worry about where the plugins are installed and the shared lib
suffix, it just depends on the above changes.  This allows us to generate
password hashes during setup before the directory server instance is created,
and also to keep clear text password usage to a minimum.
3) Added defaultuser and defaultgroup.
4) Added support for continuation lines in Inf files.
5) All user visible messages during setup should be localizable
Platforms tested: RHEL4
Flag Day: Yes, autotool file changes.
Doc impact: Yes, along with the previous fixes for this bug.
Comment 83 Rich Megginson 2007-06-19 14:33:09 EDT
Created attachment 157408 [details]
cvs commit log (Comment #76)

Fix Description: This implements support for admin server configuration and
reconfiguration, so that we can remove setuputil related code from
admserv/newinst/src.  The basic flow is this:
Ask user basic information.
Ask user if they want to either use an existing config ds or setup a new one. 
If the user selects No, the code will gather information, create the directory
server, and set it up to be the configuration DS.  Otherwise, it will just
create the directory server and register it with the existing config DS.  The
code will also create and setup and start the admin server, or reconfigure it
and restart it as needed.  Note that setup-ds-admin.pl assumes that a config DS
is to be used.	If you want to setup a directory server instance without using
a config ds, use setup-ds.pl instead.
Platforms tested: RHEL4
Flag Day: Yes - autotool changes
Doc impact: Yes.
Comment 84 Noriko Hosoi 2007-06-19 19:07:46 EDT
Created attachment 157429 [details]
cvs diff and commit messages for small bug fixes

Reviewed by Rich (Thank you!!)

Checked in into HEAD.
Comment 85 Rich Megginson 2007-06-19 19:27:21 EDT
Created attachment 157431 [details]
fix for section.param=value command line args
Comment 86 Rich Megginson 2007-06-20 10:42:21 EDT
Created attachment 157469 [details]
cvs commit log
Comment 87 Rich Megginson 2007-06-20 10:51:19 EDT
Created attachment 157471 [details]
diffs - support for ca cert for configds
Comment 88 Rich Megginson 2007-06-20 12:29:34 EDT
Created attachment 157479 [details]
cvs commit log
Comment 89 Rich Megginson 2007-06-20 12:35:34 EDT
Created attachment 157480 [details]
cvs commit log (Comment #87)

Reviewed by: nkinder (Thanks!)
Fix Description: If the Config DS is set up to use TLS/SSL, we should allow the
admin to setup a new admin server to use TLS/SSL with the Config DS.  The user
may supply either a cacert file in ascii/pem format, or just set the
CACertificate param in the .inf file to the actual ascii value.  This latter
option allows you to have a single .inf file that you can carry around to all
of your servers that you want to set up, instead of having to have an
additional file for the cacert.
However, it only works for the initial setup.  It should probably detect if the
cacert already exists and just use it if so.
File permissions need to be set correctly.  The code that deals with file and
directory creation should ensure that permissions are set properly.  This
mostly applies to the configdir, so that the config files needed to be read and
written by the admin server have the correct permissions and ownership.
Also fixed a minor bug about changing the admin server port, and with detecting
if there is an existing config ds to use or not.
Platforms tested: RHEL4
Flag Day: no
Doc impact: no
Comment 90 Rich Megginson 2007-08-06 14:42:27 EDT
Created attachment 160762 [details]
cvs commit log - dsktune

Fix Description: This adds the dsktune dialog to the initial list of setup
dialogs.
Platforms tested: RHEL4