Bug 954007 - httpd segfaults if hostname cannot be resolved
Summary: httpd segfaults if hostname cannot be resolved
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: apr
Version: 19
Hardware: All
OS: Unspecified
unspecified
high
Target Milestone: ---
Assignee: Jan Kaluža
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On: 958652
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-04-19 20:40 UTC by Zbigniew Jędrzejewski-Szmek
Modified: 2014-08-29 15:41 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-08-29 15:41:15 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
proposed patch (519 bytes, patch)
2013-04-24 07:01 UTC, Jan Kaluža
no flags Details | Diff

Description Zbigniew Jędrzejewski-Szmek 2013-04-19 20:40:02 UTC
Description of problem:
$ hostname
fedora-19
$ ping fedora-19
ping: unknown host fedora-19
$ /usr/sbin/httpd -k start -DFOREGROUND
Segmentation fault (core dumped)
$ cat >> /etc/hosts
127.0.0.1 fedora-19
$ /usr/sbin/httpd -k start -DFOREGROUND
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0

How reproducible?
100%

gethostbyname() returns NULL on error.

Comment 1 Jan Kaluža 2013-04-23 06:40:45 UTC
Do you have some extra modules loaded? Could you please get a backtrace from this crash? I'm not able to reproduce it with clean httpd installation in Fedora 19.

Comment 2 Jan Kaluža 2013-04-23 06:42:08 UTC
Also, what version of httpd/apr is it?

Comment 3 Zbigniew Jędrzejewski-Szmek 2013-04-23 14:09:44 UTC
(gdb) bt
#0  apr_getnameinfo (hostname=hostname@entry=0x7fffcbec2468, sockaddr=0x0, flags=flags@entry=0)
    at network_io/unix/sockaddr.c:621
#1  0x00007f5e9a536667 in ap_get_local_host (a=a@entry=0x7f5e9bf8e138) at util.c:2093
#2  0x00007f5e9a533583 in ap_fini_vhost_config (p=p@entry=0x7f5e9bf8e138, main_s=0x7f5e9bfb9e78)
    at vhost.c:565
#3  0x00007f5e9a531ac2 in main (argc=4, argv=0x7fffcbec3038) at main.c:643

> Also, what version of httpd/apr is it?
apr-1.4.6-6.fc19.x86_64
httpd-2.4.4-4.fc19.x86_64

> Do you have some extra modules loaded?
This is in a very small container installation, I just did 'yum install httpd' to reproduce a bug reported by a user.
I have:
# rpm -qa|grep apr
apr-1.4.6-6.fc19.x86_64
apr-util-1.4.1-8.fc19.x86_64
apr-util-debuginfo-1.4.1-8.fc19.x86_64
apr-debuginfo-1.4.6-6.fc19.x86_64
# rpm -qa|grep http
libmicrohttpd-0.9.24-2.fc19.x86_64
httpd-2.4.4-4.fc19.x86_64
httpd-debuginfo-2.4.4-4.fc19.x86_64
httpd-tools-2.4.4-4.fc19.x86_64

and I didn't actually configure apache at all.

Comment 4 Jan Kaluža 2013-04-24 07:00:13 UTC
Can you please try it with following apr scratch build? It only adds attached patch.

http://koji.fedoraproject.org/koji/taskinfo?taskID=5295698

Comment 5 Jan Kaluža 2013-04-24 07:01:07 UTC
Created attachment 739294 [details]
proposed patch

Comment 6 Zbigniew Jędrzejewski-Szmek 2013-04-24 13:57:35 UTC
Scratch build does not fix things.

# rpm -qa|grep apr
apr-1.4.6-7.fc19.x86_64
apr-util-1.4.1-8.fc19.x86_64
apr-util-debuginfo-1.4.1-8.fc19.x86_64
apr-debuginfo-1.4.6-7.fc19.x86_64

Core was generated by `/usr/sbin/httpd -k start -DFOREGROUND'.
Program terminated with signal 11, Segmentation fault.
#0  apr_getnameinfo (hostname=hostname@entry=0x7fff4229e598, sockaddr=0x0, flags=flags@entry=0)
    at network_io/unix/sockaddr.c:628
628         if (sockaddr->family == AF_INET6 &&
(gdb) bt
#0  apr_getnameinfo (hostname=hostname@entry=0x7fff4229e598, sockaddr=0x0, flags=flags@entry=0)
    at network_io/unix/sockaddr.c:628
#1  0x00007f123c6ea667 in ap_get_local_host (a=a@entry=0x7f123e138138) at util.c:2093
#2  0x00007f123c6e7583 in ap_fini_vhost_config (p=p@entry=0x7f123e138138, main_s=0x7f123e163e78)
    at vhost.c:565
#3  0x00007f123c6e5ac2 in main (argc=4, argv=0x7fff4229f168) at main.c:643

Comment 7 Jan Kaluža 2013-04-29 08:33:19 UTC
Could you please tell me what network interfaces are configured on that system? Is there any with IPv6 address?

Comment 8 Zbigniew Jędrzejewski-Szmek 2013-04-29 13:30:46 UTC
(In reply to comment #7)
> Could you please tell me what network interfaces are configured on that
> system? Is there any with IPv6 address?

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp6s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 08:60:6e:00:62:41 brd ff:ff:ff:ff:ff:ff
    inet 129.174.150.197/24 brd 129.174.150.255 scope global enp6s0f0
       valid_lft forever preferred_lft forever
    inet6 fe80::a60:6eff:fe00:6241/64 scope link 
       valid_lft forever preferred_lft forever
3: enp6s0f1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 08:60:6e:00:62:42 brd ff:ff:ff:ff:ff:ff
4: enp6s0f2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 08:60:6e:00:62:43 brd ff:ff:ff:ff:ff:ff
5: enp6s0f3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether 08:60:6e:00:62:44 brd ff:ff:ff:ff:ff:ff
6: enp0s29u1u2u2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
    link/ether 00:50:b6:4a:90:d8 brd ff:ff:ff:ff:ff:ff

Maybe it's not atr.rpm that needs to be upgraded, but also something different (e.g. apr-util)?
# ldd /usr/sbin/httpd
        linux-vdso.so.1 =>  (0x00007fffb167e000)
        libpcre.so.1 => /lib64/libpcre.so.1 (0x00007fcdcbc73000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007fcdcba50000)
        libaprutil-1.so.0 => /lib64/libaprutil-1.so.0 (0x00007fcdcb82a000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fcdcb5f3000)
        libexpat.so.1 => /lib64/libexpat.so.1 (0x00007fcdcb3c9000)
        libdb-5.3.so => /lib64/libdb-5.3.so (0x00007fcdcb011000)
        libapr-1.so.0 => /lib64/libapr-1.so.0 (0x00007fcdcade3000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcdcabc7000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fcdca9c2000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fcdca602000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcdcc155000)
        libuuid.so.1 => /lib64/libuuid.so.1 (0x00007fcdca3fd000)
        libfreebl3.so => /lib64/libfreebl3.so (0x00007fcdca190000)

# rpm -qf /lib64/libfreebl3.so /lib64/libuuid.so.1 /lib64/ld-linux-x86-64.so.2 /lib64/libc.so.6 /lib64/libdl.so.2 /lib64/libpthread.so.0 /lib64/libapr-1.so.0 /lib64/libdb-5.3.so /lib64/libexpat.so.1 /lib64/libcrypt.so.1 /lib64/libaprutil-1.so.0 /lib64/libselinux.so.1 /lib64/libpcre.so.1|sort -u
apr-1.4.6-7.fc19.x86_64
apr-util-1.4.1-8.fc19.x86_64
expat-2.1.0-5.fc19.x86_64
glibc-2.17-4.fc19.x86_64
libdb-5.3.21-9.fc19.x86_64
libselinux-2.1.13-12.fc19.x86_64
libuuid-2.23-0.6.fc19.x86_64
nss-softokn-freebl-3.14.3-1.fc19.x86_64
pcre-8.32-4.fc19.x86_64

Comment 9 Jan Kaluža 2013-04-30 06:45:20 UTC
No, the problem is that apr_sockaddr_info_get(...) returns APR_SUCCESS but does not set sockaddr variable for you (sockaddr=0x0 in the backtrace). I'm not able to reproduce it here and it returns error code instead of APR_SUCCESS here for me in F19 with the same versions of packages...

I have prepared another two testing builds which should tell us more why it returns APR_SUCCESS even when something went wrong. It would be greate if you could try them:

apr-1.4.6-8 - http://koji.fedoraproject.org/koji/taskinfo?taskID=5316769
apr-1.4.6-9 - http://koji.fedoraproject.org/koji/taskinfo?taskID=5316784

Please try first one first and if it will still crash, try the second one.

Comment 10 Zbigniew Jędrzejewski-Szmek 2013-04-30 13:56:46 UTC
Both don't crash:
# /usr/sbin/httpd -k start -DFOREGROUND
AH00557: httpd: apr_sockaddr_info_get() failed for fedora-19
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1. Set the 'ServerName' directive globally to suppress this message

AFAICT, the second behaves identically as the first. Both immediately quit. The error message implies that it'll continue running, I'm not sure what is expected. If I 'echo 127.0.0.1 fedora-19 >> /etc/hosts', httpd continues to run.

Comment 11 Jan Kaluža 2013-04-30 19:38:15 UTC
Great, I will send patch upstream to APR developers, but this looks like bug in getaddrinfo to me. I will investigate it more and discuss it more with glibc devs.

Comment 12 Zbigniew Jędrzejewski-Szmek 2013-04-30 19:44:16 UTC
(In reply to comment #11)
> Great, I will send patch upstream to APR developers

What about the fact that httpd immediately stops? Not segfaulting is of course important, but behaviour still doesn't seem correct.

Comment 13 Jan Kaluža 2013-05-02 07:18:29 UTC
I have submitted getaddrinfo bug as Bug 958652, maybe they will have some questions on you, since you are the only one who can reproduce it so far.

> What about the fact that httpd immediately stops?

Try to not load "mod_unique_id" module (comment out the line mentioning it in /etc/httpd/conf.modules.d/00-base.conf).

Comment 14 Zbigniew Jędrzejewski-Szmek 2013-05-02 17:12:32 UTC
(In reply to comment #13)
> I have submitted getaddrinfo bug as Bug 958652, maybe they will have some
> questions on you, since you are the only one who can reproduce it so far.
I'd be happy to help.

> > What about the fact that httpd immediately stops?
> 
> Try to not load "mod_unique_id" module (comment out the line mentioning it
> in /etc/httpd/conf.modules.d/00-base.conf).
Yes, with this line commented out, it continues to run.

Comment 15 Zbigniew Jędrzejewski-Szmek 2013-05-02 17:27:43 UTC
Hm, I tried to reproduce it with a second (similar) installation, and I don't get the same error! But I get an error like 'Error in `/usr/sbin/httpd': free(): invalid pointer: 0x00007fa4b1a617d8' in both my installations. I'll report the bad free as a separate bug, but if there are memory errors, than this bug could depend on random variations in local setup.

For reference, here are my step to reproduce the installation:

% sudo yum -y --releasever=19 --nogpg --installroot ~/image/fedora-19-b --disablerepo='*' --enablerepo=fedora install systemd passwd yum fedora-release vim-minimal httpd
% sudo systemd-nspawn -bD ~/image/fedora-19-b
Spawning namespace container on /home/zbyszek/image/fedora-19-b (console is /dev/pts/14)
...

and in a second window I do:

% sudo nsenter -m -u -i -n -p -t $(head -n1 /sys/fs/cgroup/systemd/machine/fedora-19-b.nspawn/system/tasks) /bin/bash
# /usr/sbin/httpd -k start -DFOREGROUND
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using fe80::a60:6eff:fe00:6241. Set the 'ServerName' directive globally to suppress this message

^C*** Error in `/usr/sbin/httpd': free(): invalid pointer: 0x00007fa4b1a617d8 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7cef8)[0x7fa4b1724ef8]
/lib64/libapr-1.so.0(apr_pool_destroy+0x1a7)[0x7fa4b1ea0de7]
/etc/httpd/modules/mod_mpm_prefork.so(+0x31ae)[0x7fa4a8d931ae]
/etc/httpd/modules/mod_mpm_prefork.so(+0x31eb)[0x7fa4a8d931eb]
/lib64/libpthread.so.0(+0xefa0)[0x7fa4b1c7afa0]
/lib64/libapr-1.so.0(apr_pool_cleanup_kill+0x32)[0x7fa4b1ea1d42]
/lib64/libapr-1.so.0(apr_pool_cleanup_run+0x11)[0x7fa4b1ea1df1]
/usr/sbin/httpd(ap_mpm_pod_close+0xd)[0x7fa4b31e599d]
/etc/httpd/modules/mod_mpm_prefork.so(+0x31c4)[0x7fa4a8d931c4]
/etc/httpd/modules/mod_mpm_prefork.so(+0x363f)[0x7fa4a8d9363f]
/etc/httpd/modules/mod_mpm_prefork.so(+0x39a6)[0x7fa4a8d939a6]
/etc/httpd/modules/mod_mpm_prefork.so(+0x3a06)[0x7fa4a8d93a06]
/etc/httpd/modules/mod_mpm_prefork.so(+0x46f0)[0x7fa4a8d946f0]
/usr/sbin/httpd(ap_run_mpm+0x4e)[0x7fa4b31c130e]
/usr/sbin/httpd(main+0xa86)[0x7fa4b31badd6]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa4b16c9b75]
/usr/sbin/httpd(+0x1df11)[0x7fa4b31baf11]
...

(The error happens after I press ^C).

Comment 16 Zbigniew Jędrzejewski-Szmek 2013-05-02 17:45:07 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=958934(In reply to comment #15)
> I'll report the bad free as a separate bug, but if there are
> memory errors, than this bug could depend on random variations in local
> setup.
https://bugzilla.redhat.com/show_bug.cgi?id=958934

(Note: I didn't see this error before, because it happens when I try to kill apache, and before it never run long enough for me to kill it.)

Comment 17 Fedora Admin XMLRPC Client 2014-07-16 12:04:06 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 18 Joe Orton 2014-08-29 15:41:15 UTC
I think we can safely close this out: the glibc bug is fixed, the apr workaround for the glibc bug is in and we even patched mod_unique_id to avoid this code path at all.  Three fixes for the price of one...


Note You need to log in before you can comment on or make changes to this bug.