A little more detail on what I believe is happening:

  1) Client sends SOLICIT.  
  2) Server sends ADVERTISE
  3) Client receives ADVERTISE and adds the server DUID to it's list
     of servers in the ifp structure.
  4) Client sends REQUEST.
  5) Server sends REPLY with NotOnLink error.
  6) Client responds to the NotOnLink error by restarting in the
     SOLICIT state.  The options are cleared, but the ifp structure is
     not.  A new SOLICIT message is sent.
  7) Server sends ADVERTISE message.
  8) Client receives ADVERTISE message, but ignores the message
     options because we "ignore the server if it is known".

It seems like dhcp6c assumes that if it sees two ADVERTISE messages from the same server, only the first ADVERTISE message has anything of value.  This may be a valid assumption, but if that's the case, then I think free_servers(ifp) should be called when the client detects to the NotOnLink error.  Something like:

--- dhcpv6-1.0.10.old/src/dhcp6c.c	2009-06-25 15:29:12.000000000 -0700
+++ dhcpv6-1.0.10.new/src/dhcp6c.c	2009-07-14 09:54:55.793792000 -0700
@@ -1620,6 +1633,7 @@
 			dprintf(LOG_DEBUG, "%s" 
 			    "got a NotOnLink reply for request/rapid commit,"
 			    " sending solicit.", FNAME);
+			free_servers(ifp);
 			newstate = DHCP6S_SOLICIT;
 			break;
 		case DH6OPT_STCODE_NOADDRAVAIL:

Created attachment 351643 [details]
Log file

The log files shows the sequence of events.

Line 135 shows the NotOnLink error:

Jul/01/2009 16:18:53 got a NotOnLink reply for request/rapid commit, sending solicit.

Line 156 shows the receipt of the ADVERTISE message:

Jul/01/2009 16:18:53 receive advertise from fe80::211:43ff:fee8:54af%eth0 scope id 2 eth0

Line 172 shows the ADVERTISE message being ignored:

Jul/01/2009 16:18:53 duplicated server (ID: 00:01:00:01:11:de:8e:f9:00:11:43:e8:54:af)

Created attachment 351645 [details]
tcpdump capture of traffic between client and server

Hi Bryan,

The patch looks reasonable.
Thanks for your investigation and complex description of the problem.

I know e.g. how to configure server (and client) to send NotOnLink error in response to CONFIRM (bug #466251), but I have problem to mis-configure the server (and client) so it it will return a NotOnLink error to REQUESTs.

Do you remember what was wrong with the server configuration that it sent NotOnLink error to REQUESTs ?

Hi Jiri,

Sorry for taking so long to reply.

The server configuration that generated the NotOnLink error to REQUESTs seemed to be that it was configured to ADVERTISE two sets of addresses:

    link MYLINK {
        pool{
            range fc07:1::1:3 to fc07:1::1:ff/112;
            range 2007:1::1:3 to 2007:1::1:ff/112;
        };
    };

I thought that since the NIC was configured with a link-local address:

    3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
       state UP  qlen 1000
       link/ether 00:14:22:1e:64:cc brd ff:ff:ff:ff:ff:ff
       inet 172.16.64.5/18 brd 172.16.127.255 scope global eth1
       inet6 fe80::214:22ff:fe1e:64cc/64 scope link 
          valid_lft forever preferred_lft forever

when the client REQUESTed the the addresses in fc07:1::1:0/122 and 2007:1::1:0/112 ranges, the server was complaining because the REQUESTed addresses didn't have the same scope as the link-local address assigned to the NIC.

However, when the server configuration was changed to:

    link MYLINK {
        pool{
            range fc07:1::1:3 to fc07:1::1:ff/112;
        };
    };

then the problem went away (the server stopped sending NotOnLink errors).

So although I'm now not sure exactly why the server was sending the NotOnLink error, I believe that configuring the server with range options like the one above should make it generate the errors.

(In reply to comment #1)
>   [...]
>   8) Client receives ADVERTISE message, but ignores the message
>      options because we "ignore the server if it is known".
> 
Hi,

with the configuration from comment #6 server really sends NotOnLink status in REPLY.
I'm not sure now why it is and whether it's a bug or configuration problem.

But with your patch the client works as expected, i.e. it doesn't ignore the
second ADVERTISE message and sends the REQUEST message with the IAADR options to server.
Problem is that the server then again sends NotOnLink in REPLY,
client again goes to SOLICIT and so on.
So the client never gets his addresses.

So I think we can fix this "ignoring ADVERTISE" problem with your patch and decide whether the sending of NotOnLink status in REPLY (when using the above mentioned configuration) is bug (worth filling) or not.

I'm lowering the Priority because there's no impact on any customer reported.

(In reply to comment #7)
> with the configuration from comment #6 server really sends NotOnLink status in
> REPLY.
> I'm not sure now why it is and whether it's a bug or configuration problem.
Same situation with the latest (and most likely also the last) dhcpv6 released, i.e. dhcpv6-1.2.0 (Fedora-11).

(In reply to comment #7)
> Problem is that the server then again sends NotOnLink in REPLY,
> client again goes to SOLICIT and so on.
> So the client never gets his addresses.

Yeah, I remember thinking that the bug was preventing the client from going into an endless loop, but in this case, that may actually be the "correct" behavior.  :)

Bryan,

I filled another bug #598439 report for the "sending of NotOnLink status in REPLY" problem.
But I'm not sure whether we want to fix these two bugs now.
- It teases no customer
- risk of regressions
- client eventually gets his addresses (but in non-correct way)

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0034.html