Bug 428785

Summary: dhclient 4.0.0-2 causes segv
Product: [Fedora] Fedora Reporter: Mamoru TASAKA <mtasaka>
Component: dhcpAssignee: David Cantrell <dcantrell>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: high    
Version: rawhideCC: jim.cornette, mcepl, mcepl, meyering, michal, pertusus, petersen, sangu.fedora
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-01-17 02:24:57 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mamoru TASAKA 2008-01-15 08:13:01 UTC
Description of problem:
When I upgraded dhclient to 4.0.0-2.fc9 and tried to
activate network, it failed with dhclient segv.

Version-Release number of selected component (if applicable):
dhclient-4.0.0-2.fc9

How reproducible:
100%

Steps to Reproduce:
1. ifup eth1
2.
3.
  
Actual results:
[root@localhost i386]# LANG=C ifup eth1

Determining IP information for eth1.../etc/sysconfig/network-scripts/ifup-eth:
line 297: 20375 Segmentation fault      (core dumped) /sbin/dhclient
${DHCLIENTARGS} ${DEVICE}
 failed.
[root@localhost i386]# cd /etc/sysconfig/network-scripts/

[root@localhost network-scripts]# gdb dhclient core.20375 
GNU gdb Red Hat Linux (6.7.1-10.fc9rh)
Copyright (C) 2007 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...
Using host libthread_db library "/lib/libthread_db.so.1".

warning: Can't read pathname for load map: Input/output error.

warning: Missing the matching library file:
/usr/lib/debug/.build-id/fb/601234f43e181ec146c30341cfedde4486e4e9

warning: Missing the matching library file:
/usr/lib/debug/.build-id/7f/8d2ae53f877fdc20b83e7cc4f11ace35da0565

warning: Missing the matching library file:
/usr/lib/debug/.build-id/10/d12e928f0dfd302e8c6922e8743ad6786d464d
Reading symbols from /lib/libc.so.6...
warning: Missing the separate debug info file:
/usr/lib/debug/.build-id/fb/601234f43e181ec146c30341cfedde4486e4e9.debug
done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/ld-linux.so.2...
warning: Missing the separate debug info file:
/usr/lib/debug/.build-id/7f/8d2ae53f877fdc20b83e7cc4f11ace35da0565.debug
done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...
warning: Missing the separate debug info file:
/usr/lib/debug/.build-id/10/d12e928f0dfd302e8c6922e8743ad6786d464d.debug
done.
Loaded symbols for /lib/libnss_files.so.2

warning: Missing the matching library file:
/usr/lib/debug/.build-id/fb/601234f43e181ec146c30341cfedde4486e4e9

warning: Missing the matching library file:
/usr/lib/debug/.build-id/7f/8d2ae53f877fdc20b83e7cc4f11ace35da0565

warning: Missing the matching library file:
/usr/lib/debug/.build-id/10/d12e928f0dfd302e8c6922e8743ad6786d464d
Core was generated by `/sbin/dhclient -1 -q -lf
/var/lib/dhclient/dhclient-eth1.leases -pf /var/run/dh'.
Program terminated with signal 11, Segmentation fault.
#0  0x08063331 in get_char (cfile=0x8fcec68) at conflex.c:185
185                     c = cfile->inbuf [cfile->bufix];
(gdb) thread apply all bt

Thread 1 (process 20375):
#0  0x08063331 in get_char (cfile=0x8fcec68) at conflex.c:185
#1  0x080666cf in get_raw_token (cfile=0x8fcec68) at conflex.c:258
#2  0x08066eb8 in get_next_token (rval=0xbfc8bba8, rlen=0x0, cfile=0x8fcec68,
raw=isc_boolean_false) at conflex.c:323
#3  0x080770d2 in skip_to_rbrace (cfile=0x8fcec68, brace_count=0) at parse.c:131
#4  0x08077165 in skip_to_semi (cfile=0x8fcec68) at parse.c:97
#5  0x0804ff76 in read_client_leases () at clparse.c:286
#6  0x0805942f in main (argc=8, argv=0xbfc8beb4, envp=0x8fcec84) at dhclient.c:752
#7  0x001274a0 in __libc_start_main () from /lib/libc.so.6
#8  0x0804df01 in _start ()
(gdb) qui

Expected results:
ifup succeeds normally

Additional info:
dhclient-3.1.0-12.fc9 seems no problem.

Comment 1 David Cantrell 2008-01-15 09:26:07 UTC
The dhclient.leases file formats changed in version 4.0.0.  I will fix up the
postinstall script in the RPM to clean these up on upgrade, but for now you can
do this:

rm /var/lib/dhclient/*

And then run dhclient again.  Should work fine after that.

Comment 2 Mamoru TASAKA 2008-01-15 12:59:25 UTC
Thank you for information.

(In reply to comment #1)
> I will fix up the
> postinstall script in the RPM to clean these up on upgrade, 

I want to try this so I will wait for new rpms.

Comment 3 Michal Jaegermann 2008-01-16 03:05:28 UTC
> but for now you can do this:

> rm /var/lib/dhclient/*

> And then run dhclient again.  Should work fine after that.

I am afraid that it does not work for me. First of all I saved
old leases file for dhclient, restarted an interfarce with
a new dhclient-4.0.0-2.fc9 and I do not see any format differences.
Only timestamps on renew, rebind and expire fields changed - which
is to be fully expected.

The second problem is that if there is no
/var/lib/dhclient/dhclient-eth1.leases file (I have eth1 on DHCP)
then eth1 starts just fine.  After that 'ifdown eth1; ifup eth1'
causes an instant segfault.

Here is what I see in a dumped core after dhcp-debuginfo package
was installed:

Core was generated by `/sbin/dhclient -1 -q -lf
/var/lib/dhclient/dhclient-eth1.leases -pf /var/run/dh'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000041b6cf in get_char (cfile=0x8829e0) at conflex.c:185
185                     c = cfile->inbuf [cfile->bufix];
(gdb) where
#0  0x000000000041b6cf in get_char (cfile=0x8829e0) at conflex.c:185
#1  0x000000000041e5e7 in get_raw_token (cfile=0x8829e0) at conflex.c:258
#2  0x000000000041ec4c in do_peek_token (rval=0x7fffb2410898, rlen=0x0,
    cfile=0x8829e0, raw=isc_boolean_false) at conflex.c:391
#3  0x000000000042dd5f in skip_to_rbrace (cfile=0x8829e0, brace_count=0)
    at parse.c:111
#4  0x000000000040a72e in read_client_leases () at clparse.c:286
#5  0x0000000000412112 in main (argc=8, argv=<value optimized out>,
    envp=<value optimized out>) at dhclient.c:752
#6  0x00000038f701e2b4 in __libc_start_main () from /lib64/libc.so.6
#7  0x0000000000408989 in _start ()
(gdb) l
180                             c = cfile -> read_function (cfile);
181                     } else {
182                             c = EOF;
183                     }
184             } else {
185                     c = cfile->inbuf [cfile->bufix];
186                     cfile->bufix++;
187             }
188
189             if (!cfile->ugflag) {
(gdb) p cfile
$1 = (struct parse *) 0x8829e0
(gdb) p *cfile
$2 = {lexline = 29, lexchar = 3647, token_line = 0x882a0c "",
  prev_line = 0x882a5d "}", cur_line = 0x882a0c "",
  tlname = 0x7fffb2412aac "/var/lib/dhclient/dhclient-eth1.leases",
  eol_token = 0, line1 = '\0' <repeats 80 times>,
  line2 = "}\000rebind 6 2008/01/19
14:40:20;\000om\";\00023.254,199.185.130.12,199.185.130.13;\000\000\000\000\000\000",
lpos = 3648, line = 29,
  tlpos = 3647, tline = 29, token = 0, ugflag = 0, tval = 0x677a78 "",
  tlen = 1,
  tokbuf = "\n\000\0008\000e\000e.com\000dentifier", '\0' <repeats 1477 times>,
warnings_occurred = 0, file = 4,
  inbuf = 0x2aaaaab0f000 <Address 0x2aaaaab0f000 out of bounds>, bufix = 4096,
  buflen = 18446744073709551615, bufsiz = 449, saved_state = 0x0,
  read_function = 0x41f000 <read_function>}
(gdb)

Well, cfile->inbuf is clearly a garbage so a segfault is quite
appriopriate.

An obvious workaround for now is "rm -f" on a leases file before
an attempt to bring an interface up.

    rm -f /var/lib/dhclient/dhclient-${DEVICE}.leases

on line 171 in /etc/sysconfig/network-scripts/ifup-eth does work
for ethX devices but clearly is not a fix.

Comment 4 Darwin H. Webb 2008-01-16 03:08:41 UTC
rm -f /var/lib/dhclient/dhclient-${DEVICE}.leases

This did not work for me.
And .3 from koji did not fix the problem?


Darwin

Comment 5 Michal Jaegermann 2008-01-16 03:27:11 UTC
> rm -f /var/lib/dhclient/dhclient-${DEVICE}.leases
> This did not work for me.

Not on a command line! In a file /etc/sysconfig/network-scripts/ifup-eth
and then ${DEVICE} is set to an interface you are using.  Otherwise
you are trying 'rm -f /var/lib/dhclient/dhclient-.leases' and that
will silently return doing nothing.

> And .3 from koji did not fix the problem?
I do not know; but comment #1 misidentifies the issue so chances are
quite poor.

Comment 6 Jim Cornette 2008-01-16 04:17:17 UTC
So if -3 does not fix the issue, what will work?
I am currently running with a known static address after seeing the failure on
boot. Should I stick to static vs. dynamic? The router seems to be forgiving
without addressing changes to the router set up as a gateway.

Comment 7 Michal Jaegermann 2008-01-16 05:04:17 UTC
> So if -3 does not fix the issue
I looked at sources and I do not see anything for dhclient-4.0.0-3.fc9
which would address the issue.  Just to confirm I run it and got myself
immediately a shiny new core - as expected.

> what will work?
- applying, properly, a hack from comment #3 will temporary paper over
- backing off to dhclient-3.1.0-12.fc9 will work
- using a static address, if you can, will avoid the problem; it should
  NOT be from a range covered by your dynamic pool or you may end up
  with two different interfaces having the same address.


Comment 8 David Cantrell 2008-01-16 05:14:08 UTC
Hey guys, thanks for all the info.  This is a _very_ strange problem that I've
never run in to before.  I am working on it now and will have a real fix soon. 
If you have any other info on dhclient-4.0.0-2.fc9, post here.

Comment 9 Michal Jaegermann 2008-01-16 06:36:35 UTC
Well, 'buflen = 18446744073709551615' in a gdb trace I posted translates
to 0xffffffffffffffff in hex or -1 if signed.  Does that ring a bell?
This does not look a healthy length of a buffer unless it was used
somewhere as a flag.

Comment 10 David Cantrell 2008-01-17 02:24:57 UTC
Fixed in dhcp-4.0.0-4.fc9.  dhclient will now read dhclient.lease files
correctly.  Homemade parsers irritate me.

Thanks again guys for all the troubleshooting help.  Helped me track down the
problem.

Comment 11 Michal Jaegermann 2008-01-17 05:35:45 UTC
Thanks.  dhclient-4.0.0-4.fc9 works for me.  Indeed it does not attempt
to read past the end of leases file. :-)

I did not look what dhcp server from 4.0.0 is doing but if the same parser
is used to process old leases then the bug may show in that place too.
Or this is just the same parser, period?  In that case you fixed both.

Comment 12 David Cantrell 2008-01-17 06:03:06 UTC
The same [bad] parser code is used in both the client and server.

Comment 13 David Cantrell 2008-01-17 19:10:38 UTC
*** Bug 429115 has been marked as a duplicate of this bug. ***

Comment 14 David Cantrell 2008-01-17 19:11:30 UTC
*** Bug 429156 has been marked as a duplicate of this bug. ***