Bug 1263338

Created attachment 1073712 [details]
squid.conf

Note: this is not a regression, it's just 1st time of running the test on rhel-7 at big endian systems. The same problem is in squid-3.3.8-12.el7_0.
On RHEL-6 (squid-3.1.23-9.el6) there is no such problem.

Thanks Ondřej for helping with the debugging. The infinite loop is located in `auth_digest.cc` function `authenticateDigestNonceNew()`:

186         while ((temp = authenticateDigestNonceFindNonce((char const *) (newnonce->key)))) {
187             /* create a new nonce */
188             newnonce->noncedata.randomdata = squid_random();
189             /* Bug 3526 high performance fix: add 1 second to creationtime to avoid duplication */
190             ++newnonce->noncedata.creationtime;
191             authDigestNonceEncode(newnonce);
192         }

So far the most suspicious part is a typecast in `authenticateDigestNonceFindNonce()`:

354         nonce = static_cast < digest_nonce_h * >(hash_lookup(digest_nonce_cache, nonceb64));

As the problem depends on endianess, my first guess is endianess of `struct _digest_nonce_h` members.

Got it. The problem is not in the typecasts but rather in the base64 code turning a binary string into a base64 encoded string. The code not only checks the data pointer for being non-NULL but also the first byte for being non-zero and returns an empty string otherwise!

    base64.c:149

    if (!data || !*data || !result || result_size < 1 || data_size < 1)

It's binary data. I don't see any reason not to encode binary data starting with a zero byte. The first struct member is a relatively small number and therefore is encoded as a sequence of non-zero and zero bytes in little endian byte order. In big endian it is the other way round, starting with a zero byte.

Unless there is another issue, the following patch should fix it.

--- ./lib/base64.c.orig 2015-09-17 13:35:07.654617797 +0200
+++ ./lib/base64.c      2015-09-17 13:35:35.734700154 +0200
@@ -146,7 +146,7 @@
     int char_count = 0;
     int out_cnt = 0;
 
-    if (!data || !*data || !result || result_size < 1 || data_size < 1)
+    if (!data || !result || result_size < 1 || data_size < 1)
         return 0;
 
     if (!base64_initialized)

Let's see what upstream thinks about the patch.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-2378.html