Bug 436274 - Add Fedora DS support
Summary: Add Fedora DS support
Alias: None
Product: Fedora
Classification: Fedora
Component: migrationtools
Version: 9
Hardware: All
OS: Linux
Target Milestone: ---
Assignee: Jan Safranek
QA Contact: Fedora Extras Quality Assurance
Depends On: 437104
TreeView+ depends on / blocked
Reported: 2008-03-06 10:03 UTC by Jan Safranek
Modified: 2009-07-14 12:23 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2009-07-14 12:23:37 UTC

Attachments (Terms of Use)

Description Jan Safranek 2008-03-06 10:03:56 UTC
Since we have separate package for migrationtools, we may think about making it
working with Fedora Directory Server, see the discussion below.

+++ This bug was initially created as a clone of Bug #236697 +++

-- Additional comment from mastahnke@gmail.com on 2008-03-03 22:49 EST --
I was hoping to not require the openldap server
when using Fedora DS.  I guess that's still not going to be the case?  I
normally do use openldap-clients, but not so much the server.  It just seems odd
that the flagship directory product (and education classes from RH) require you
to install openldap-* (and padl tools) to use Fedora/Red Hat DS.  

-- Additional comment from jsafrane@redhat.com on 2008-03-05 08:42 EST --
The migrationtools require ldapadd and ldif2dbm utilities. In OpenLDAP world,
this functionality is provided by openldap-clients (ldapadd) and
openldap-servers (slapadd). If I am looking right, in Fedora DS this is provided
by mozldap (ldapmodify) and fedora-ds-base (ns-slapd with some parameters).

I have no experience with Fedora DS and I need your help - could you please
confirm, that PADL migrationtools work with ldapmodify and ns-slapd? Try:

export LDAPADD="/usr/lib/mozldap/ldapmodify -a"
export LDIF2DBM="/usr/sbin/ns-slapd ldif2db -f $NSHOME/slapd-$serverID" (if it
is the right command, I really do not know)

and run migrate_all_online/offline

If it works, we can play with dependencies:
- mozldap Provides: ldapadd
- fedora-ds-base Provides: ldif2dbm
- openldap-clients Provides: ldapadd
- openldap-servers Provides: ldif2dbm
- migrationtools Requires: ldapadd, ldif2dbm

-> migrationtools would work whith any combination providing both ldif2dbm and
ldapadd. I could also fix the migrationtools to look in the right directories,
so it could work out of the box without magic env. variables.

-- Additional comment from mastahnke@gmail.com on 2008-03-05 18:13 EST --
I can run these test, but it might take me a day or two.  I also posted this bug
notice in IRC #fedora-ds to ask for any other testers, etc. 

-- Additional comment from rmeggins@redhat.com on 2008-03-05 18:21 EST --
A couple of problems:
export LDAPADD="/usr/lib/mozldap/ldapmodify -a"
will not work on a 64-bit system - /usr/lib64/mozldap

export LDIF2DBM="/usr/sbin/ns-slapd ldif2db -f $NSHOME/slapd-$serverID"

Is not quite right either.  There is usually only one "instance" of OpenLDAP
running on a machine.  Fedora DS by default supports more than one.  So you have
to provide the instance name.  Also, the fedora ds ldif2db tool supports
multiple database backend instances, you have to specify one either by name
(e.g.-n userRoot) or by suffix (e.g. -s "dc=example,dc=com").  I suppose with
openldap it either assumes only one database, or it is smart enough to parse the
ldif into multiple databases if necessary.

So something like this:
/usr/lib[64]/dirsrv/slapd-$serverid/ldif2db -n userRoot
should work in most cases.

Finally, the OpenLDAP libraries must usually be present on the system for things
like nss_ldap, pam_ldap, and other ldap aware apps.  It is a small matter to
also install the openldap-clients package to get ldapsearch, etc.  You can use
the openldap ldapsearch et. al. with Fedora DS - you do not have to use the
mozldap ldapsearch (although Fedora DS scripts do use mozldap ldapsearch, so
mozldap-tools is a requirement for Fedora DS - but users can use openldap

-- Additional comment from hyc@symas.com on 2008-03-05 19:42 EST --
slapadd tries the first database instance by default. It will return an error if
that database's suffix doesn't match the LDIF. You can select a specific one by
suffix (-b dc=example,dc=com) or by number (-n 1 = first database in
configuration)  If the configuration uses multiple glued databases, then by
default the LDIF is parsed into the individual databases automatically.

Comment 1 Jan Safranek 2008-03-07 10:44:48 UTC
I installed fedora-ds and with little migrationtools hacking I am able to import
data to it with LDAPADD="/usr/lib64/mozldap/ldapmodify -c -a"
(migrate_all_online) and ldapsearch returns the data.

I have trouble with LDIF2LDBM="/usr/lib64/dirsrv/slapd-test/ldif2db -n userRoot
" and migrate_all_offline - it does something, but I can't find the inserted
data. Here is ldif2db output:

[07/Mar/2008:10:52:23 +0100] - dblayer_instance_start: pagesize: 4096, pages:
515414, procpages: 43020
[07/Mar/2008:10:52:23 +0100] - cache autosizing: import cache: 204800k
[07/Mar/2008:10:52:23 +0100] - li_import_cache_autosize: 50, import_pages:
51200, pagesize: 4096
[07/Mar/2008:10:52:23 +0100] - WARNING: Import is running with
nsslapd-db-private-import-mem on; No other process is allowed to access the database
[07/Mar/2008:10:52:23 +0100] - dblayer_instance_start: pagesize: 4096, pages:
515414, procpages: 43020
[07/Mar/2008:10:52:23 +0100] - cache autosizing: import cache: 204800k
[07/Mar/2008:10:52:23 +0100] - li_import_cache_autosize: 50, import_pages:
51200, pagesize: 4096
[07/Mar/2008:10:52:23 +0100] - import userRoot: Beginning import job...
[07/Mar/2008:10:52:23 +0100] - import userRoot: Index buffering enabled with
bucket size 100
[07/Mar/2008:10:52:24 +0100] - import userRoot: Processing file
[07/Mar/2008:10:52:24 +0100] - 2 duplicate values for attribute type cn detected
in entry cn=nextstep,ou=Services,dc=test. Extra values ignored.
[07/Mar/2008:10:52:24 +0100] - Duplicate value for attribute type cn detected in
entry cn=cvsup,ou=Services,dc=test. Extra value ignored.
[07/Mar/2008:10:52:24 +0100] - import userRoot: Finished scanning file
"/tmp/nis.ldif.D15020" (4985 entries)
[07/Mar/2008:10:52:24 +0100] - import userRoot: WARNING: Skipping duplicate
entry "cn=raid-am,ou=Services,dc=test" found at line 16269 of file
[07/Mar/2008:10:52:24 +0100] - import userRoot: WARNING: Skipping duplicate
entry "cn=terminaldb,ou=Services,dc=test" found at line 16326 of file
[07/Mar/2008:10:52:24 +0100] - import userRoot: WARNING: Skipping duplicate
entry "cn=whosockami,ou=Services,dc=test" found at line 16340 of file
[07/Mar/2008:10:52:24 +0100] - import userRoot: Workers finished; cleaning up...
[07/Mar/2008:10:52:25 +0100] - import userRoot: Workers cleaned up.
[07/Mar/2008:10:52:25 +0100] - import userRoot: Cleaning up producer thread...
[07/Mar/2008:10:52:25 +0100] - import userRoot: Indexing complete. 
[07/Mar/2008:10:52:25 +0100] - import userRoot: Flushing caches...
[07/Mar/2008:10:52:25 +0100] - import userRoot: Closing files...
[07/Mar/2008:10:52:25 +0100] - All database threads now stopped
[07/Mar/2008:10:52:25 +0100] - import userRoot: Import complete.  Processed 4985
entries (3 were skipped) in 2 seconds. (2492.50 entries/sec)

After service dirsrv start, ldapsearch -x -b "dc=test" does not return anything
(but it does in the migrate_all_online case, so the database is configured
correctly). And if I try to add some entry, ldapadd prints "Already exists" ->
the data is somewhere, I am just not able to find it...

Comment 2 Rich Megginson 2008-03-07 22:28:01 UTC
When you ran setup-ds-admin.pl, did you choose dc=test as your default suffix?

Does /usr/lib64/dirsrv/slapd-test/db2ldif -n userRoot -a /tmp/data.ldif give
your the data you expect to see in /tmp/data.ldif?

Comment 3 Jan Safranek 2008-03-10 08:36:37 UTC
(In reply to comment #2)
> When you ran setup-ds-admin.pl, did you choose dc=test as your default suffix?


> Does /usr/lib64/dirsrv/slapd-test/db2ldif -n userRoot -a /tmp/data.ldif give
> your the data you expect to see in /tmp/data.ldif?


Still, ldapsearch -x -b "dc=test" does not return anything and I don't see
dc=test in idm-console (fedora-ds-1.1.0-3.fc8.x86_64)

Comment 4 Rich Megginson 2008-03-10 15:48:03 UTC
This is really weird.  Can you post your dse.ldif file?  Does the access log
show any connection, bind, or search attempts?  Any other errors in the error log?

Comment 5 Jan Safranek 2008-03-12 14:00:06 UTC
Finally solved it: ldapsearch -x -b "dc=test" -D "cn=Directory Manager" -x -w
my_password works as expected. ldif2db seems not to set anonymous read
permission, while ldapadd does. I am not able to judge whether it's bug or
feature (and I am too lazy to dig into Fedora DS to get the answer... I am
OpenLDAP guy).

Anyway, I filled bug #437104 and I start implementing proper migrationtools
hacks to get it working. I expect that the admin sets $serverId as suggested by
migration-tools.txt (I'll add a note that it applies also to Fedora DS).

Comment 6 Rich Megginson 2008-03-12 14:06:22 UTC
Ah, ok.  In Fedora DS, access control is off by default - unless you
specifically allow something, everything is denied (except for directory
manager).  Access control is specified by adding the operational attribute "aci"
to entries that you want to control access to.  aci has subtree scope so the
access applies to the entry with the attribute and all child entries.  In
OpenLDAP, access control is usually outside of the data, in slapd.conf or in
cn=config (unless subtree access control has been enabled).  So probably the
OpenLDAP config had allowed anonymous search to the suffix, but since the LDIF
file contained no aci attributes, Fedora DS disallowed access.  You can see some
common acis in the file /usr/share/dirsrv/data/template-baseacis.ldif - just
replace %ds_suffix% with your suffix and pass this to ldapmodify.

Comment 7 Bug Zapper 2008-05-14 05:49:01 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:

Comment 8 Bug Zapper 2009-06-09 23:41:26 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 

Comment 9 Jan Safranek 2009-07-14 12:23:37 UTC
Requires: ldif2ldbm is in migrationtools since Fedora 10, I just forgot to close the bug, sorry.

Note You need to log in before you can comment on or make changes to this bug.