Bug 914874

Description of problem:
As per http://www.gluster.org/community/documentation/index.php/Arch/BitRot_Detection
SHA256 is being considered as the hash algorithm for bitrot detection.

While this is the current standard, I have a few suggestions regarding this in order to be performant with low overhead on x86 platforms. There are 3 options:

1. BLAKE2 (64 and 32 bit) can be considered being designed to be used for these kind of use cases: https://blake2.net/. It is even faster than MD5 and is an enhanced version of BLAKE which was one the NIST SHA3 finalists with as large a security margin as KECCAK. Use cpuid detection to call the appropriate SSE2/4/AVX1/AVX2/XOP version.

2. If you still want to use SHA256 then there are two approaches to get the highest throughput possible:
   a. Use SHA512-256 for 64-bit platforms.
   b. Use Intel's Optimized SHA512-256 implementation (public domain) on x86 platforms: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/fast-sha512-implementations-ia-processors-paper.pdf
      This works well on AMD processors as well. Intel's implementation is the fastest possible on x86 processors even comparing with OpenSSL, CryptoPP and TrueCrypt.
   c. Use cpuid detection to use the appropriate SSE4/AVX1/AVX2 version.
   d. Intel also provides an optimized SHA256 if you do not want SHA512-256: http://www.intel.com.br/content/dam/www/public/us/en/documents/white-papers/sha-256-implementations-paper.pdf

3. Along with one of the above approaches use Merkle Tree-hashing (via OpenMP) to parallelize the work and get a significant benefit on multi-core boxes. In order not to completely load all the cores you can, for example, restrict parallelization to be upto 2 threads. However Tree-hashing cannot be mixed with 
serial hashing since the hash values generated are different.

4. For VM images you can logically split the image file into fixed-size chunks and calculate independent hash values for each. For sparse images hashing can be avoided for zero chunks.