Bug 567711

Todd, do you use kerberos in your environment?  Do you possibly have a network dumps of the traffic causing crash?

(In reply to comment #1)
> Todd, do you use kerberos in your environment?  Do you possibly have a network
> dumps of the traffic causing crash?    

I do not have kerberos in my environment, nor do I have it enabled to my knowledge. Is there some setting you believe may be turned on by default that I can turn off? 

I think I do have a stack trace around with debug symbols up to the kerberos libraries. I'll look for that for you.

As for network dumps, if you could give me a quick hint as to how to capture this dump and/or what data you need to capture, I'll try to pull one for you.

This was relatively easy for my to re-produce once I found the pattern. Have you tried to do so yet?

Todd

Backtrace for a dovecot crash with debug symbols for openssl

#0  0x0000003adf4610a2 in krb5_is_referral_realm () from /usr/lib64/libkrb5.so.3
No symbol table info available.
#1  0x0000003adf448ade in krb5_kt_get_entry () from /usr/lib64/libkrb5.so.3
No symbol table info available.
#2  0x00002ba8d01c171e in kssl_keytab_is_available (kssl_ctx=0x150009f0) at kssl.c:1810
	krb5context = (krb5_context) 0x150187d0
	krb5keytab = (krb5_keytab) 0x15017aa0
	entry = {magic = 352312352, principal = 0x2ba8d01b6a54, timestamp = 0, vno = 0, key = {magic = -800400046, enctype = 1, length = 1, 
    contents = 0x100000000 <Address 0x100000000 out of bounds>}}
	princ = (krb5_principal) 0x0
	krb5rc = <value optimized out>
	rc = 1
#3  0x00002ba8d01a7345 in ssl3_choose_cipher (s=0x150184c0, clnt=<value optimized out>, srvr=<value optimized out>) at s3_lib.c:2239
	c = (SSL_CIPHER *) 0x2ba8d03cfc28
	prio = (STACK *) 0x14ffdc20
	allow = (STACK *) 0x14ff4a00
	i = 0
	j = <value optimized out>
	ok = <value optimized out>
	cert = (CERT *) 0x150008e0
	alg = 8224
	mask = 10545
	emask = 10545
#4  0x00002ba8d01a2aeb in ssl3_get_client_hello (s=0x150184c0) at s3_srvr.c:969
	i = <value optimized out>
	j = <value optimized out>
	ok = 1
	al = <value optimized out>
	ret = <value optimized out>
	cookie_len = <value optimized out>
	n = 1
	id = 1
	p = <value optimized out>
	c = <value optimized out>
	comp = <value optimized out>
	ciphers = (STACK *) 0x0
#5  0x00002ba8d01a3465 in ssl3_accept (s=0x150184c0) at s3_srvr.c:282
	buf = <value optimized out>
	l = <value optimized out>
	Time = 1267462235
	cb = (void (*)(const SSL *, int, int)) 0
	num1 = <value optimized out>
	ret = <value optimized out>
	state = 8464
	skip = <value optimized out>
#6  0x00002ba8d01ab602 in ssl23_get_client_hello (s=0x150184c0) at s23_srvr.c:577
	buf_space = "\026\003\000\000-\001\000\000)\003"
	p = <value optimized out>
	d = (unsigned char *) 0x0
	i = <value optimized out>
	csl = 8720
	cl = <value optimized out>
	n = 11
	j = <value optimized out>
	type = 3
#7  0x00002ba8d01abd99 in ssl23_accept (s=0x150184c0) at s23_srvr.c:203
	buf = (BUF_MEM *) 0x14ff8f10
	Time = 1267462235
	cb = (void (*)(const SSL *, int, int)) 0
	ret = <value optimized out>
	state = 8720
#8  0x000000000040b2f3 in ssl_step (proxy=0x14fff0a0) at ssl-proxy-openssl.c:415
No locals.
#9  0x0000000000414618 in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:208
	ctx = <value optimized out>
	event = (const struct epoll_event *) 0x14ff8f10
	list = (struct io_list *) 0x14ffd220
	io = (struct io_file *) 0x14ffc510
	tv = {tv_sec = 179, tv_usec = 999371}
	events_count = <value optimized out>
	t_id = 2
	msecs = <value optimized out>
	ret = 1
	i = 0
	call = <value optimized out>
#10 0x000000000041372d in io_loop_run (ioloop=0x14ff6fe0) at ioloop.c:335
No locals.
#11 0x00000000004089f5 in main (argc=1, argv=0x7fffbda05d48, envp=0x7fffbda05d58) at main.c:494
	group_name = <value optimized out>
	remote_ip = {family = 59264, u = {ip6 = {in6_u = {u6_addr8 = "\000\000\000\000k0@\000\000\000\000\000\000\220\026?", u6_addr16 = {0, 0, 12395, 64, 0, 0, 36864, 53270}, u6_addr32 = {0, 
          4206699, 0, 3491139584}}}, ip4 = {s_addr = 0}}}
	local_ip = {family = 1, u = {ip6 = {in6_u = {u6_addr8 = '\0' <repeats 12 times>, "???\024", u6_addr16 = {0, 0, 0, 0, 0, 0, 42400, 5373}, u6_addr32 = {0, 0, 0, 352167328}}}, ip4 = {
      s_addr = 0}}}
	remote_port = 58
	local_port = 0
	max_fds = 1021
	proxy = (struct ssl_proxy *) 0x0
	client = (struct client *) 0x0
	i = <value optimized out>
	fd = 1
	master_fd = -1
	ssl = 253

This is related to chroot, reproduced with dovecot and stunnel with chroot (stunnel does not crash without chroot).

Created attachment 397135 [details]
tcpdump -i eth0 tcp port 993

$>tcpdump -w /var/tmp/imap.pcap -i eth0 tcp port 993
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
315 packets captured
315 packets received by filter
0 packets dropped by kernel

Created attachment 397143 [details]
Check krb5_sname_to_principal() return value

Tested with stunnel.  Tomas, can you check this?

(In reply to comment #5)
> $>tcpdump -w /var/tmp/imap.pcap -i eth0 tcp port 993

Thank Todd.  It's not needed any more, we can reproduce now as mentioned in comment #4.  You may wish to use -s 1500 tcpdump option in the future, to avoid creating pcap file with truncated packets.

(In reply to comment #2)
> Is there some setting you believe may be turned on by default that I
> can turn off?

Both dovecot and stunnel offer configuration options to specify OpenSSL cipher list (see man ciphers) that can be used to disable support for specific ciphers (such as KRB5 ones), but that won't help here, as this crash happens before OpenSSL checks proposed cipher suite against its allowed list.

(In reply to comment #6)
> Created an attachment (id=397143) [details]
> Check krb5_sname_to_principal() return value

This patch was accepted by upstream and is now committed in upstream CVS:
  http://cvs.openssl.org/chngview?cn=19374

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2010-0162.html