Created attachment 765229 [details]
File: backtrace

Created attachment 765231 [details]
File: cgroup

Created attachment 765233 [details]
File: core_backtrace

Created attachment 765235 [details]
File: dso_list

Created attachment 765237 [details]
File: environ

Created attachment 765239 [details]
File: limits

Created attachment 765241 [details]
File: maps

Created attachment 765244 [details]
File: open_fds

Created attachment 765245 [details]
File: proc_pid_status

Created attachment 765246 [details]
File: var_log_messages

Created attachment 765247 [details]
broken hosts.deny file

This bug is triggered by a broken hosts.deny file.
I have attached my hosts.deny file that was present at the time of the crash.
It looks very strange. I have no idea how that happened, though.

There's a bug in hosts_access.c - host_match() function introduced in cf076b1867b6310182b72007111dfb89612eedcf commit.

*** Bug 986427 has been marked as a duplicate of this bug. ***

See duplicate bug 986427.  This problem caused rpcbind and sshd to segfault on one of my systems.  However, my hosts.deny contained a single line of the form:

ALL:  machine.domain.com

which should be legit as far as I can tell, thus even a non-broken hosts.deny (or hosts.allow) seems to cause problems.

I have experienced the same problem with sshd giving a segfault.

Program terminated with signal 11, Segmentation fault.
#0  0x00007f478907fe29 in _IO_vfprintf_internal (s=s@entry=0x7f478d916e50, format=<optimized out>, format@entry=0x7fff3eef4750 "warning: /etc/hosts.allow, line 11: host name/address mismatch: %s != %.*s", ap=<optimized out>)
    at vfprintf.c:1635
#1  0x00007f47891418a6 in ___vfprintf_chk (fp=fp@entry=0x7f478d916e50, flag=flag@entry=1, format=format@entry=0x7fff3eef4750 "warning: /etc/hosts.allow, line 11: host name/address mismatch: %s != %.*s", ap=ap@entry=0x7fff3eef6788)
    at vfprintf_chk.c:34
#2  0x00007f4789126245 in __GI___vsyslog_chk (pri=<optimized out>, pri@entry=3, flag=flag@entry=1, fmt=fmt@entry=0x7fff3eef4750 "warning: /etc/hosts.allow, line 11: host name/address mismatch: %s != %.*s", ap=ap@entry=0x7fff3eef6788)
    at ../misc/syslog.c:222
#3  0x00007f478b85db61 in vsyslog (__ap=0x7fff3eef6788, __fmt=0x7fff3eef4750 "warning: /etc/hosts.allow, line 11: host name/address mismatch: %s != %.*s", __pri=3) at /usr/include/bits/syslog.h:47
#4  tcpd_diag (tag=tag@entry=0x7f478b85e0e9 "warning", format=format@entry=0x7f478b85e418 "host name/address mismatch: %s != %.*s", ap=ap@entry=0x7fff3eef6788, severity=3) at diag.c:45
#5  0x00007f478b85dc48 in tcpd_warn (format=format@entry=0x7f478b85e418 "host name/address mismatch: %s != %.*s") at diag.c:55
#6  0x00007f478b85cfeb in sock_hostname (host=0x7fff3eef73d0) at socket.c:277
#7  0x00007f478b85c539 in eval_hostname (host=host@entry=0x7fff3eef73d0) at eval.c:77
#8  0x00007f478b85ad88 in host_match (tok=0x7fff3eef69b5 "LOCAL", host=0x7fff3eef73d0) at hosts_access.c:305
#9  0x00007f478b85a54b in list_match (list=<optimized out>, request=0x7fff3eef72c0, match_fn=0x7f478b85b080 <client_match>) at hosts_access.c:211
#10 0x00007f478b85a729 in table_match (table=<optimized out>, request=request@entry=0x7fff3eef72c0) at hosts_access.c:168
#11 0x00007f478b85a84e in hosts_access (request=request@entry=0x7fff3eef72c0) at hosts_access.c:125
#12 0x00007f478be947dd in main (ac=<optimized out>, av=<optimized out>) at sshd.c:2066

I tried looking at a few values using gdb and I found it was triggered by a hostile attack.

In socket.c in the function sock_hostname, if I look at the contents of host, I see host->name gives indoautotech.com. There is absolutely no legitimate reason why this address would connect to my system, so this is clearly hostile. The message passed to tcp_warn is "host name/address mismatch: %s != %.*s" and in the code there is the comment "The host name does not map to the initial address. Perhaps someone has messed up. Perhaps someone compromised a name server." If I look up this name I get:

Name:	indoautotech.com
Address: 67.205.2.42

However, if I look at the contents of sin I get:
{sg_addr = {_sg_sa = {sa_family = 2, 
      sa_data = "\345\324z\264~K\000\000\000\000\000\000\000"}, _sg_sin = {
      sin_family = 2, sin_port = 54501, sin_addr = {s_addr = 1266594938}, 
      sin_zero = "\000\000\000\000\000\000\000"}, _sg_sin6 = {sin6_family = 2, 
      sin6_port = 54501, sin6_flowinfo = 1266594938, sin6_addr = {__in6_u = {
      __u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 
      0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}}
and s_addr corresponds to 122.180.126.75

Looking up this address I get:
Non-authoritative answer:
75.126.180.122.in-addr.arpa	name = indoautotech.com.

So indeed, it looks like it is a spoof of some sort. So this is a serious security issue.

I also note that if I look in tcpd_diag, I see that tcpd_context.file has the value "/etc/hosts.allow" and tcpd_context.line is 11. If I look in my hosts.allow file, that line is simply:
ALL: LOCAL
which should be legitimate, right? I suspect this is spurious. It is simply the first non-comment line in the file which is given in such cases, where the name and address don't match, right?

tcpd_warn is called with arguments:
  "host name/address mismatch: %s != %.*s", inet_ntop(SGFAM(sin), SGADDRP(sin), buf, sizeof(buf)), STRING_LENGTH, hp->h_name

The contents of buf at this point are "122.180.126.75", so I presume that inet_ntop has correctly returned a pointer to buf, but actually this is not guaranteed. If an error occurs, it will return NULL and set errno, so this is a possible cause of the error. I guess this should be trapped, just in case.
Unfortunately hp has been optimised out at this point, so I don't know what it is passing as hp. However, the code path should not be executed if it is NULL, so presumably there is something valid in hp->h_name and since the length is explicitly restricted to STRING_LENGTH (128) I don't see how this could cause a problem.

So unfortunately, I can't see what exactly causes this crash, but it is a security issue, so it is rather serious. I hope this information helps. Unfortunately, it is virtually impossible to reproduce controllably, because it depends on some evil cracker in India trying to break into my system:)

Which version of tcp_wrappers do you use? Will update to the latest tcp_wrappers-7.6-74.fc19 update help?

Sorry, I forgot to mention which version I was using. I was already using the latest tcp_wrappers-7.6-74.fc19. So, no that doesn't fix it.

There was a missing #include <arpa/inet.h> in socket.c so a compiler guessed return value of inet_ntop() to be int instead of const char * 

rawhide build http://koji.fedoraproject.org/koji/taskinfo?taskID=5819690

For what it's worth, that exact IP dropped my openssh today, with a backtrace identical to what nvwarr showed above. The hosts.allow line that it got fetched up on is:

ALL:localhost

Yes, I think that is the correct explanation, Petr. I wrote a simple test program which uses the data extracted from the coredump and passed it to tcpd_warn. I did this yesterday, before you posted and wasn't able to reproduce the problem, but after I saw your post, I realised that I had included arpa/inet.h. Sure enough, when I removed that include I got the same crash as with sshd. So the sooner an update with this include added reaches the repositories, the better. Thanks for your effort.

tcp_wrappers-7.6-75.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/tcp_wrappers-7.6-75.fc19

*** Bug 987483 has been marked as a duplicate of this bug. ***

Package tcp_wrappers-7.6-75.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing tcp_wrappers-7.6-75.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-14882/tcp_wrappers-7.6-75.fc19
then log in and leave karma (feedback).

For some reason, it still hasn't reached my local mirror yet, but I've downloaded the rpm directly, installed it and restarted sshd. Now, we just have to wait until some evil person tests it:)

I can say, that with my own patch, which just included arpa/inet.h, I got the message "warning: /etc/hosts.allow, line 11: host name/address mismatch: 123.30.214.190 != static.vdc.vn" in the early hours of this morning, which is the point where it segfaulted without the patch. This is the first time I've got this message since I applied the patch. This again seems to be one of those nameserver misconfiguration things. 123.30.214.190 translates as static.vdc.vn, but if I look up static.vdc.vn, I get 203.162.0.78.

I won't give karma for your version yet, because I didn't install it until after this attempted breakin, but your tcp_wrappers-7.6-warnings.patch includes the important change, so I expect it to solve the issue. I guess compiler warnings do matter:)

tcp_wrappers-7.6-75.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Dears, 

I have the same problem but it causes the BackupExec Agent to shutdown 

my version is tcp_wrappers-7.6-77.el7.x86_64

kernel: beremote[]: segfault at 1f5840fc07d ip 00007f1554e1850c sp 00007f15531297c8 error 4 in libc-2.17.so[7f1554d98000+1b6000]


is there any other solution ?

Mahmoud, 

this bug was filled against Fedora 19 and it is already closed. If you have any problem with your RHEL-7, please raise a ticket through your regular Red Hat support channels to make certain  it receives the proper attention and prioritization to assure a timely resolution.