Description of problem: Apache httpd with mod_ssl doesn't serve HTTPS correctly. Received requests (decrypted) are damaged. Version-Release number of selected component (if applicable): 2.12.90-15 though I think older releases are affected, too. Downgrading to F13 glibc-2.12-1.x86_64 (and glibc-common) fixes the problem. How reproducible: Install Fedora 14-current and httpd+mod_ssl and try to browse HTTPS. I'm actually using additional sw (RedMine as a mod_fcgid process) but I think it doesn't matter (I can check further if there are problems with reproducing). Steps to Reproduce: 1. Prepare a script similar to (substitute valid host name): GET /login?back_url=https%3A%2F%2Fredmine.osb.piap.pl%2Fprojects HTTP/1.1 Host: redmine.osb.piap.pl User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.10) Gecko/20100920 Fedora/3.6.10-1.fc13 Firefox/3.6.10 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Cookie: _redmine_session=BAh7BzoPc2Vzc2lvbl9pZCIlYjlkYzNmNTM5N2Q2ZTRiNjZiMTkxOWNkMjZkNWY2NTI6EF9jc3JmX3Rva2VuIjFhZjFmY0NkT3ozYlNUcXVWM3h3OTdmK1hoL3B2aEY4SVRaNEhrMWk4dW1ZPQ%3D%3D--4b3171d6ecf7e30496596352b3e51a4633d6ed61 Cache-Control: max-age=0 2. Adding the following entries to httpd config file may help debugging: LoadModule dumpio_module modules/mod_dumpio.so DumpIOInput On LogLevel debug 3. Feed the script to https: openssl s_client -quiet -connect $HTTPS_SERVER:443 < test Actual results: Https says it can't interpret the data. Http log file contains something like: dumpio_in (data-TRANSIENT): GET /login?back_url=https%3A%2F%2Fredmine.osb.piap.pl%2Fprojects HTTP/1.1\n dumpio_in (data-TRANSIENT): ntrol: max-age=0Q%3D%3D--4b3171d6ecf7e30496596352b3e51a4633d6ed61\n dumpio_in (data-TRANSIENT): rol: max-age=0\n dumpio_in (data-TRANSIENT): \n Sometimes the problem doesn't manifest itself and only does so after httpd is restarted. To be sure I restarted it several times. Expected results: HTTPs server should send HTML page. Additional info: It seems it's timing-dependent. When running under strace or ltrace the problem can be seen less frequently. Running the kernel with "nosmp" doesn't help. The hw is Asus Rampage II Gene + Core i7 CPU with VT-d (DMAR) disabled in BIOS (VT-d is broken on these boards).
It looks like glibc-2.12.90-16.x86_64 fixes this problem, I'm closing this.
Oops, it still happens with glibc-2.12.90-16.x86_64 and with -17, too. Sorry for the noise.
I cannot see any difference in the results between F13 and F14.
Perhaps a bit simpler test: - F13 w/ updates + only glibc and glibc-common from F14 (2.12.90-15.x86_64) - mod_ssl with default install (using default "localhost" cert etc.) - a completely different machine than the above, core 2 duo - based, running a custom kernel 2.6.35.4+ instead of 2.6.35.6-43.fc14.x86_64 (the above) - no extra software involved test file: GET / HTTP/1.1 Host: localhost User-Agent: 010203040506070809101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899 Cookie: test=010203040506070809101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899 Cache-Control: test=010203040506070809101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899 (6 lines, the last line is empty as required by the HTTP protocol, one may need to "unwrap" the long lines) openssl s_client -quiet -connect localhost:443 < test produces a valid HTML page half of the time, and the following the remaining half: HTTP/1.1 400 Bad Request ... <p>Your browser sent a request that this server could not understand.<br /> Request header field is missing ':' separator.<br /> <pre> 96979899</pre> ...
Works fine here.
I checked on the following machines running F13 w/ updates + glibc* 2.6.90-18: a) i686 Pentium III - can't observe the problem b) x86_64 Athlon 64 X2 - can't observe the problem c) x86_64 Pentium4 HT - can't observe the problem d) x86_64 Core2 duo - buggy e) x86_64 Core i7 - buggy Main difference between /proc/cpuinfo flags between b+c vs d+e is addition of: arch_perfmon aperfmperf vmx est tm2 ssse3 pdcm sse4_1 tpr_shadow vnmi flexpriority (all 4 machines can do sse2 + pni). Looks to me like some buggy SSE* string routine.
Please do followings: 1. Disable optimized string/memory functions one by one in glibc to find out which one causes the problem. 2. Write a wrapper for that function, which will a. Call both versions of the function. b. Compare their results. c. If they are different, dump all input parameters/contents.
Breakpoint 3, char_buffer_read (inctx=0x7fcb8b6a9898, buf=0x7fcb8b6a98e0 "Host: localhost\nUser-Agent: 0102030405060708091011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586"..., len=0x7fff86990f80) at /usr/src/debug/httpd-2.2.16/modules/ssl/ssl_engine_io.c:361 361 memcpy(in, buffer->value, buffer->length); (gdb) p in $7 = 0x7fcb8b6a98e0 "Host: localhost\nUser-Agent: 0102030405060708091011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586"... (gdb) p buffer.value $8 = 0x7fcb8b6a98f0 "User-Agent: 01020304050607080910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394"... (gdb) p buffer.length $9 = 643
Please use the 2.2.17 package in testing. *** This bug has been marked as a duplicate of bug 624609 ***