Bug 451116 - make sure master, replica uses different sequence of numbers.
Summary: make sure master, replica uses different sequence of numbers.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: freeIPA
Classification: Retired
Component: ipa-server
Version: 2.0
Hardware: All
OS: Linux
high
low
Target Milestone: ---
Assignee: Rob Crittenden
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On:
Blocks: 431020
TreeView+ depends on / blocked
 
Reported: 2008-06-12 21:36 UTC by Chandrasekar Kannan
Modified: 2015-01-04 23:32 UTC (History)
6 users (show)

Fixed In Version: freeipa-2.0.0-1.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-03-28 09:28:30 UTC
Embargoed:


Attachments (Terms of Use)

Description Chandrasekar Kannan 2008-06-12 21:36:53 UTC
on a master-replica setup, the configuration is not by RANGE yet. looks like
they are both starting off with the same number. 

On the master, I did ipa-adduser s1 (got 16724). did ipa-moduser to 16728. did
ipa-adduser s2 (got 16732).

on the replica, I did ipa-adduser t1 (got 16724). 16724 is not used. so good.
But looks like they both are not returning any type of gaps in between them.

Comment 1 Chandrasekar Kannan 2008-06-13 20:08:36 UTC
david - to note this in release notes

Comment 2 David O'Brien 2008-06-17 05:19:46 UTC
My take on this:
In an MR setup, if you create a user on the master and then another on the
replica, there's every chance that they will end up with the same userid if no
replication has occurred inbetween times. Is this what's happening? If not could
somebody explain it a bit better?

I don't know what "the configuration is not by RANGE yet" means.

From a customer's perspective, what's their "workaround"? Do they just need to
be aware that this can occur and to double-check userids, or to force a
replication before they start creating new users? How can they "make sure the
master and replica use a different sequence of numbers"?

thanks

Comment 4 David O'Brien 2008-06-17 13:53:22 UTC
Added to release notes.

Comment 5 Chandrasekar Kannan 2008-06-17 19:53:59 UTC
davido - ignore comment#3. 
the following is from simo's email.

Writing on the train so I can't update 451116 but here is some explanation.

The way dna solves the problem of creating unique uids across multiple masters
is by making sure each master choose from a different pool of IDs.

The initial solution Pete coded was fixed into a scheme that let you define the
number of masters (default 4) and then simply skip N masters IDs every new
allocation. This solution left huge holes in case all allcoations were happening
against a single master (3/4th of IDs were wasted). It also required to change
the starting ID so that (baseID + N masters) resulted in a different number on
each master.
We never ended up making the code to automatically change the base ID when
creating a replica so all masters were basically allocating from the same pool.
Even worst the original plugin didn't check if the ID was already in use at
least locally so duplicates were the norm.

I modified the plugin to work both this way or to simply increase monotonically
and always check for duplicates before assigning any ID.

What we still miss although is to change the configuration on replicas so that
each replica assigns IDs from a different range.

Right now all servers try to assign IDs starting from 1101 up to 1,000,000,000.
What customers need to do until we fix this in the replica installation code is
to change the dna plugin configuration on replicas to use a different set of
ranges before starting using it.

What users need to do is to do an ldapmodify to change the plugin configuration
so that on the second replica 1,000,000,001-2,000,000,000 is used and on the 3rd
2,000,000,001-3,000,000,000 is used, and so on.

(Note if customer is still using some incredibly old operating system that
accept only 16bit uids, customer will have to change uids manually to fit it
after user creation, good chances are these system are so old they do not
support ldap anyway, so we do not care in the default case)

The entries to change are these so far:
dn: cn=Accounts,cn=Posix,cn=ipa-dna,cn=plugins,cn=config
dn: cn=Groups,cn=Posix,cn=ipa-dna,cn=plugins,cn=config

the attributes to change are:
dnaNextValue
dnaMaxValue

This is an example ldif to use to change the first replica installed:

dn: cn=Accounts,cn=Posix,cn=ipa-dna,cn=plugins,cn=config
changetype: modify
replace: dnaNextValue
dnaNextValue: 1000000001

dn: cn=Accounts,cn=Posix,cn=ipa-dna,cn=plugins,cn=config
changetype: modify
replace: dnaMaxValue
dnaMaxValue: 2000000000

dn: cn=Groups,cn=Posix,cn=ipa-dna,cn=plugins,cn=config
changetype: modify
replace: dnaNextValue
dnaNextValue: 1000000001

dn: cn=Groups,cn=Posix,cn=ipa-dna,cn=plugins,cn=config
changetype: modify
replace: dnaMaxValue
dnaMaxValue: 2000000000



These can only be change ad Directory Manager, so the following command might be
used provided the above ldif is save in a file called dna.ldif

ldapmodify -x -D "cn=Directory Manager" -W -f dna.ldif

This command must be executed ONLY on the first replica, we do not want to
change all replicas (or even the master) to the same ranges, each replica needs
it's own.

We are also not required to jump million by million, admins can choose how to
best distribute ranges of IDs across replicas. The only thing admins must know,
although, is that once one master's dnaNextValue reaches the dnaMaxValue value,
that master will stop servicing new IDs and user/group creation may fail. To fix
this all is needed is to assign a new free range to the server that depleted the
ID space. Although given that it is highly unlikely any installation will really
create millions of users, depletion of IDs MUST make an admin check if something
is wrong in their procedures and is using too many IDs, or some such.

HTH,
Simo.



Comment 6 David O'Brien 2008-06-18 01:51:46 UTC
updated release notes with background and update procedure.

Comment 9 Rob Crittenden 2010-02-16 19:00:15 UTC
What is the DS team recommendation on configuring the DNA Plugin to handle this gracefully?

Comment 10 Nathan Kinder 2010-02-16 19:25:27 UTC
If I'm understanding the issue correctly, you need to configure DNA to automatically transfer ranges between masters.  This capability was added to Directory Server in 8.1 I believe.  There is some high level info in this on the DNA design page:

http://directory.fedoraproject.org/wiki/DNA_Plugin#Shared_Config_Method

The configuration is also documented in the RHDS 8.1 Administrator's Guide:

http://www.redhat.com/docs/manuals/dir-server/8.1/admin/dna.html

The actual ranges you choose don't really matter from a Directory Server perspective.  The key thing is to ensure that they don't overlap.

Comment 11 Rob Crittenden 2010-09-09 17:53:35 UTC
https://fedorahosted.org/freeipa/ticket/198

Comment 12 Rob Crittenden 2011-03-11 03:13:11 UTC
master: 6a5c4763afad6fec2b49ffadbca9628a7ed162d5


Note You need to log in before you can comment on or make changes to this bug.