Bug 243227

Nathan, another candidate for winsync cleanup?

When you delete an entry that is sync'd with AD from DS, then add that entry
back elsewhere in the tree, the entry ends up getting deleted from both AD and
DS.  Here's why this behavior is occuring:

    - Entry is deleted from DS (this is what a "cut" in Console does).
    - Deletion operation is immediately sync'd to AD.
    - Entry is added back to DS with all of it's old attributes (this is what a   
      "paste" in Console does).
    - The sync code notices that it has a "ntuniqueid" attribute present for 
      this entry, which means it was sync'd at one time.
    - An add is attempted to be performed against AD, but it's using a GUID 
      based DN, which fails with a naming violation.
    - The next dirsync operation sync's the deletion of the entry back to DS.  
      This deletion finds the entry using the ntuniqueid.

Created attachment 193691 [details]
CVS Diffs

Here's how the proposed fix deals with this problem.

The problem is caused by adding an entry to DS with an "ntuniqueid" attribute
present.  You could add the entry back without the "ntuniqueid", but this
breaks permissions and other things in AD that are tied to that entry's GUID. 
Depending on the exact environment and situation, there are a few things that
this fix does.

When you delete an entry in AD, a tombstone entry is created.  In the AD
version that's included in Windows 2003 Server, tombstone reanimation is
supported.  Unfortunately, the AD version in Windows 2000 does not support
tombstone reanimation.	Because of this difference in capabilities between the
version of AD we are dealing with, we have to behave differently for each
version.

When we are dealing with a Windows 2000 Server, we will rewrite a GUID style DN
to a regular mapped DN when we have an "ntuniqueid" attribute present on the DS
side, but the entry doesn't exist on the AD side.  This means that  doing a
"cut & paste" of an sync'd entry in Console will end up deleting the old entry
in AD, then creating a new entry elsewhere in the tree in AD.  AD will end up
generating a new GUID for this new entry.  This is the best that we can on with
Windows 2000 Server since there is no way of restoring a deleted entry with
it's GUID aside form bringing the Domain Controller offline to do a restore. 
The ideal solution would be to support MODDN with new superior on the DS side
so that an entry can be moved using that style of operation, which we could
then sync to AD.  An explicit delete and add operation would still have the
behavior described above.

When we are dealing with a Windows 2003 Server, we will take advantage of the
tombstone reanimation capabilities of AD.  In the "cut & paste" scenario, we
will sync the delete during the "cut" operation.  When the "add" occurs, we
will see that we have a "ntuniqueid" in the new entry being added.  We then
check for the existence of a tombstone using this GUID.  If the tombstone
exists, we will resurrect it where the new entry is being added in the tree. 
We then change the "add" operation into a "modify" operation to sync over any
attributes lost when the tombstone was generated (AD strips an entry of all
non-essential attributes when a tombstone is created).	The next dirsync
operation that happens does not send any delete over to DS since the tombstone
was resurrected at that point, so everything ends up in sync.  In the case
where the tombstone entry does not exist for the missing entry, we fallback to
doing an add operation of a new entry, which means a new GUID will be generated
by AD that ends up getting sync'd back to DS.  This seems like the proper thing
to do since there is no trace of that GUID in AD.  This would deal with any odd
cases such as adding a new entry to DS with a fake "ntuniqueid" attribute that
AD did not generate.

To implement the solutions described above, there was quite a bit of new code
needed.  I added functions to detect if the peer AD has the capabilities of the
Windows 2003 Server version.  This is done by checking for a special OID in the
"supportedCapabilities" attribute in AD's root DSE.  I also had to extend all
of the functions that send LDAP operations to AD to allow controls to be passed
to AD.	This was required since you need to send a "Return Deleted Objects"
control to be able to do any operations on tombstone entries in AD.  I also
added helper functions for things such as converting GUIDs between formats
since we store them in a different format than they exist in AD, mapping an
entry in DS to it's tombstone DN in AD, checking for the existence of a
particular tombstone in AD, and reanimating a tombstone entry.

Are all entries that match is_guid_dn(remote_dn) a tombstone?  Is it possible
that there could be non-tombstone entries that match is_guid_dn(remote_dn)?  Is
there another way to determine if an entry is a tombstone?

Please use slapi_ch_free_string() instead of slapi_ch_free() for char* values.

Windows allows you to rename/move an entry with a MOD instead of a MODRDN?  Weird.

(In reply to comment #15)
> Are all entries that match is_guid_dn(remote_dn) a tombstone?  Is it possible
> that there could be non-tombstone entries that match is_guid_dn(remote_dn)?  Is
> there another way to determine if an entry is a tombstone?

The is_guid_dn() function is not trying to determine if an entry is a tombstone,
and there are definitely cases where we would have a GUID dn for a non-tombstone
entry.  The only place that we call is_guid_dn() is when we are about to replay
an add operation against AD, but the remote entry does not exist in AD.  For any
newly added entries, we will not have a GUID DN.  If we have a GUID DN at this
point, it means that the entry being added contains a "ntuniqueid" attribute, so
a GUID DN is being used.  We then check for the existence of a tombstone for
this entry and reanimate it if it exists.  If it does not exist, we rewrite the
DN and create a new remote entry.  

> Please use slapi_ch_free_string() instead of slapi_ch_free() for char* values.

Will do.

> Windows allows you to rename/move an entry with a MOD instead of a MODRDN? Weird.

Reanimating a tombstone is a special case.  If you send a modify that replaces
the "distinguishedName" attribute value as well as removes the "isDeleted"
attribute of a tombstone entry, AD picks up on this and does a MODRDN
internally.  I do agree that it's a bit odd, but that's the procedure that must
be done to resurrect a tombstone.

Created attachment 193941 [details]
Revised Diffs

This new set of diffs uses slapi_ch_free_string() for char * values.  I cleaned
this usage up throughout the entire windows_protocol_util.c source file.

Ok, looks good.

Checked into ldapserver (HEAD).  Thanks to Rich for his review!

Checking in repl5.h;
/cvs/dirsec/ldapserver/ldap/servers/plugins/replication/repl5.h,v  <--  repl5.h
new revision: 1.10; previous revision: 1.9
done
Checking in windows_connection.c;
/cvs/dirsec/ldapserver/ldap/servers/plugins/replication/windows_connection.c,v 
<--  windows_connection.c
new revision: 1.16; previous revision: 1.15
done
Checking in windows_private.c;
/cvs/dirsec/ldapserver/ldap/servers/plugins/replication/windows_private.c,v  <--
 windows_private.c
new revision: 1.15; previous revision: 1.14
done
Checking in windows_protocol_util.c;
/cvs/dirsec/ldapserver/ldap/servers/plugins/replication/windows_protocol_util.c,v
 <--  windows_protocol_util.c
new revision: 1.32; previous revision: 1.31
done
Checking in windowsrepl.h;
/cvs/dirsec/ldapserver/ldap/servers/plugins/replication/windowsrepl.h,v  <-- 
windowsrepl.h
new revision: 1.12; previous revision: 1.11
done

Test done as below:

1. create user on RHDS
2. sync with AD
3. "Cut" user from RHDS console
4. Verify: user is being delete from AD
5. "Paste" user back to same OU
6. Verify: user appear on both RHDS and AD.

Notes: 
1) When this test account gets sync'd to AD side, AD shows the default state of
this user as "disabled". Once enabled, this account performs same as other account

2). And "User logon name" is empty. But I think this would be fine