RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1773798 - dhclient -6 -S (statelss) does not wait for interface up
Summary: dhclient -6 -S (statelss) does not wait for interface up
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: initscripts
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Jan Macku
QA Contact: Daniel Rusek
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-18 23:42 UTC by Harald Jensås
Modified: 2020-09-29 19:42 UTC (History)
4 users (show)

Fixed In Version: initscripts-9.49.51-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-29 19:42:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
journal with rc.debug on kernel commandline. (182.78 KB, text/plain)
2019-11-27 09:35 UTC, Harald Jensås
no flags Details
additional debug - 'ip a' commands added in "/etc/sysconfig/network-sycipts/network-functions-ipv6" (185.24 KB, text/plain)
2020-01-09 09:32 UTC, Harald Jensås
no flags Details
journal with rc.debug - and "scope global" removed in ipv6_wait_tentative() (221.19 KB, text/plain)
2020-01-28 13:58 UTC, Harald Jensås
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:3889 0 None None None 2020-09-29 19:42:38 UTC

Description Harald Jensås 2019-11-18 23:42:11 UTC
Description of problem:
When an interface is configured for stateless DHCPv6 the client fail.

It looks like the /usr/sbin/dhclient-script does not execute, thus the PREINIT6 does not do the wait for link up. Since the link is no up the client fails.


Version-Release number of selected component (if applicable):
dhclient-4.2.5-77.el7.

How reproducible:
100%

Steps to Reproduce:
1. Boot system with interface config:
  BOOTPROTO=none
  DEVICE=eth0
  DHCPV6C=yes
  DHCPV6C_OPTIONS=-S
  HWADDR=fa:16:3e:ce:5a:c9
  IPV6INIT=yes
  IPV6_AUTOCONF=yes
  MTU=1450
  NM_CONTROLLED=no
  ONBOOT=yes
  STARTMODE=auto
  TYPE=Ethernet
  USERCTL=no

Actual results:

dhclient[1307]: Can't bind to dhcp address: Cannot assign requested address
dhclient[1307]: Please make sure there is no other dhcp server
dhclient[1307]: running and that there's no entry for dhcp or
dhclient[1307]: bootp in /etc/inetd.conf.   Also make sure you
dhclient[1307]: are not running HP JetAdmin software, which
dhclient[1307]: includes a bootp server.
dhclient[1307]: 
dhclient[1307]: This version of ISC DHCP is based on the release available
dhclient[1307]: on ftp.isc.org.  Features have been added and other changes
dhclient[1307]: have been made to the base software release in order to make
dhclient[1307]: it work better with this distribution.
dhclient[1307]: 
dhclient[1307]: Please report for this software via the CentOS Bugs Database:
dhclient[1307]:     http://bugs.centos.org/
dhclient[1307]: 
dhclient[1307]: exiting.


Expected results:
dhclient[1995]: XMT: Info-Request on eth0, interval 1050ms.
dhclient[1995]: RCV: Reply message on eth0 from fe80::f816:3eff:feac:cefe.
NET[2019]: /usr/sbin/dhclient-script : updated /etc/resolv.conf

Comment 2 Pavel Zhukov 2019-11-19 08:01:31 UTC
(In reply to Harald Jensås from comment #0)
> Description of problem:
> When an interface is configured for stateless DHCPv6 the client fail.
> 
> It looks like the /usr/sbin/dhclient-script does not execute, thus the
> PREINIT6 does not do the wait for link up. Since the link is no up the
> client fails.
dhclient is not supposed to wait for link to be up It's caller responsibility to call dhclient once link is up

Comment 3 Harald Jensås 2019-11-19 15:35:59 UTC
(In reply to Pavel Zhukov from comment #2)
> (In reply to Harald Jensås from comment #0)
> > Description of problem:
> > When an interface is configured for stateless DHCPv6 the client fail.
> > 
> > It looks like the /usr/sbin/dhclient-script does not execute, thus the
> > PREINIT6 does not do the wait for link up. Since the link is no up the
> > client fails.
> dhclient is not supposed to wait for link to be up It's caller
> responsibility to call dhclient once link is up

Then why is there code in dhclient doing just that when dhcpclient is used without the "-S" (stateless) option?
https://source.isc.org/cgi-bin/gitweb.cgi?p=dhcp.git;a=blob;f=client/scripts/linux;h=0c42969744e4fd0ea513f9d5516183cef207c3b4;hb=HEAD#l410
 This code wait for link and duplicate address detection with a configurable timeout upstream. (Downstream this is static ...)


There is this KCS: https://kb.isc.org/docs/aa-01212 that explain that the interface must be up and the "link-local" address available. But at the same time afaict the default dhclient-script does implement the code to wait for exactly this?

This is an upstream bug https://bugs.isc.org/Public/Bug/Display.html?id=36169 that is marked as resolved, but it seem not to have solved the problem for "stateless" operation.

Since dhclient's default script does this for non stateless modes, it makes sense for it to do so for statelss as well?


If you still think this is'nt a dhclient issue, I guess we should change component to initscripts.

Comment 4 Pavel Zhukov 2019-11-19 16:50:25 UTC
Thank you for posting links.

However most of them are related for statefull mode. As you mentioned dhclient does wait for link up in statefull mode but doesn't in stateless one. There is reason behind this. In stateless mode interface must be brought up and configured prior to dhclient asks for configuration otherwise it's possible to have race condition between RA and DHCP which may lead in wrong configuration (missed DNS/routers). It's out of scope of dhclient to do this kind of syncronization between RA and DHCP. That's why caller (initscripts, user (in case of manual calling of dhclient command, systemd, dracut or NetworkManager) must take care of it. 

You can check following links for more information (it's long story with applying and reverting patches):
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1101149#c3
[2] https://bugs.launchpad.net/ubuntu/+source/isc-dhcp/+bug/1764478

I hope this helps.

Comment 5 Harald Jensås 2019-11-19 17:25:37 UTC
Thanks Pavel! I see in the launcpad link you posted why PREINIT6 cannot be used for stateless, this makes sense.

I can workaround the problem by adding "sleep 10" in /etc/sysconfig/network-scripts/ifup-eth

    357     # Initialize the dhclient args for IPv6 and obtain the hostname options if needed:
    358     DHCLIENTARGS="-6 -1 ${DHCPV6C_OPTIONS} ${DHCLIENTCONF} -lf ${LEASEFILE} -pf /var/run/dhclient6-${DEVICE}.pid ${DEVICE}"
    359     set_hostname_options DHCLIENTARGS
    360     sleep 10                                    # <-- Wait for IPv6 required when using '-S' option.
    361     if /sbin/dhclient $DHCLIENTARGS; then
    362         echo $" done."
    363     else


Component changed to initscripts. Version: initscripts-9.49.47-1.el7.x86_64

Comment 6 Pavel Zhukov 2019-11-19 17:29:52 UTC
(In reply to Harald Jensås from comment #5)
> Thanks Pavel! I see in the launcpad link you posted why PREINIT6 cannot be
> used for stateless, this makes sense.
You're welcome!
P.S. There was PREINIT6S back in RHEL 6 but it was dropped due to reasons I mentioned.

Comment 7 Jan Macku 2019-11-26 09:43:32 UTC
Hi Harald,

I wasn't able to fully reproduce this issue in VM.

Could you please add option "rc.debug" on kernel cmd and send me output from journalctl?

Comment 8 Harald Jensås 2019-11-27 09:35:13 UTC
Created attachment 1640046 [details]
journal with rc.debug on kernel commandline.

Hi Jan,

I reproduced the issue and attached the journalctl log with rc.debug enabled.


Regards
Harald

Comment 9 Jan Macku 2019-12-12 12:36:49 UTC
Hi Harald,

Thank you for log. It looks like it should work. There is function "ipv6_wait_tentative()" in initscripts which should avoid this issue. The problem might be, that device isn't available at the time of calling this function.

Could you please open file "/etc/sysconfig/network-sycipts/network-functions-ipv6". At the end of the file there is function "ipv6_wait_tentative()". Please add command "ip a" right before "while [ ${countdown} -gt 0 ]; do" and also inside while loop before "if [ -z $"ip_output" ]; then".

Then could you please send me log from journalctl again with "rc.debug" set on kernel cmd.

Thank you.
Jan

Comment 10 Harald Jensås 2020-01-09 09:32:06 UTC
Created attachment 1650932 [details]
additional debug - 'ip a' commands added in "/etc/sysconfig/network-sycipts/network-functions-ipv6"

Hi Jan,

I reproduced the issue again with the 'ip a' commands added in "/etc/sysconfig/network-sycipts/network-functions-ipv6" as requested. See attached file.


Regards
Harald

Comment 11 Jan Macku 2020-01-24 09:06:24 UTC
Hi Harald,

The function "ipv6_wait_tentative()" which should to be waiting for DAD is waiting only for scope global addresses. Since this issue is related to scope link addresses, waiting for both scopes (link and global) should fix this issue.

Could you please make small modification inside "ipv6_wait_tentative()" function in "/etc/sysconfig/network-sycipts/network-functions-ipv6".

Please change ip command inside while loop to: 

   ip_output="$(ip -6 addr show dev ${device} tentative)"

and try to reproduce it with debug options set on kernel cmd. Then could you please send me log from journalctl.


Thank you.

Regards
Jan

Comment 12 Harald Jensås 2020-01-28 13:58:20 UTC
Created attachment 1656020 [details]
journal with rc.debug - and "scope global" removed in ipv6_wait_tentative()

Hi Jan,

Looks like removing the scope global in ipv6_wait_tentative() fixes the issue.
See attached journal log.


Best Regards
Harald

Comment 13 Jan Macku 2020-01-29 12:32:23 UTC
Hi Harald,

Following patch should fix the issue. 

PR: https://github.com/fedora-sysv/initscripts/pull/309


Fix will be available in RHEL 7.9

Thank you very much for your cooperation.

Best Regards
Jan

Comment 18 errata-xmlrpc 2020-09-29 19:42:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (initscripts bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3889


Note You need to log in before you can comment on or make changes to this bug.