Fastest hash for non-cryptographic uses?
fcn time generated hash
crc32: 0.03163 798740135
md5: 0.0731 0dbab6d0c841278d33be207f14eeab8b
sha1: 0.07331 417a9e5c9ac7c52e32727cfd25da99eca9339a80
xor: 0.65218 119
xor2: 0.29301 134217728
add: 0.57841 1105
And the code used to generate this is:
$loops = 100000;
$str = "ana are mere";
echo "<pre>";
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$x = crc32($str);
}
$tse = microtime(true);
echo "\ncrc32: \t" . round($tse-$tss, 5) . " \t" . $x;
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$x = md5($str);
}
$tse = microtime(true);
echo "\nmd5: \t".round($tse-$tss, 5) . " \t" . $x;
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$x = sha1($str);
}
$tse = microtime(true);
echo "\nsha1: \t".round($tse-$tss, 5) . " \t" . $x;
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$l = strlen($str);
$x = 0x77;
for($j=0;$j<$l;$j++){
$x = $x xor ord($str[$j]);
}
}
$tse = microtime(true);
echo "\nxor: \t".round($tse-$tss, 5) . " \t" . $x;
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$l = strlen($str);
$x = 0x08;
for($j=0;$j<$l;$j++){
$x = ($x<<2) xor $str[$j];
}
}
$tse = microtime(true);
echo "\nxor2: \t".round($tse-$tss, 5) . " \t" . $x;
$tss = microtime(true);
for($i=0; $i<$loops; $i++){
$l = strlen($str);
$x = 0;
for($j=0;$j<$l;$j++){
$x = $x + ord($str[$j]);
}
}
$tse = microtime(true);
echo "\nadd: \t".round($tse-$tss, 5) . " \t" . $x;
CRC32 is pretty fast and there's a function for it: http://www.php.net/manual/en/function.crc32.php
But you should be aware that CRC32 will have more collisions than MD5 or even SHA-1 hashes, simply because of the reduced length (32 bits compared to 128 bits respectively 160 bits). But if you just want to check whether a stored string is corrupted, you'll be fine with CRC32.
There's a speed comparison on the xxHash repository. This is what it shows, on January 12, 2021.
Hash Name | Width | Bandwidth (GB/s) | Small Data Velocity | Quality | Comment |
---|---|---|---|---|---|
XXH3 (SSE2) | 64 | 31.5 GB/s | 133.1 | 10 | |
XXH128 (SSE2) | 128 | 29.6 GB/s | 118.1 | 10 | |
RAM sequential read | N/A | 28.0 GB/s | N/A | N/A | for reference |
City64 | 64 | 22.0 GB/s | 76.6 | 10 | |
T1ha2 | 64 | 22.0 GB/s | 99.0 | 9 | Slightly worse [collisions] |
City128 | 128 | 21.7 GB/s | 57.7 | 10 | |
XXH64 | 64 | 19.4 GB/s | 71.0 | 10 | |
SpookyHash | 64 | 19.3 GB/s | 53.2 | 10 | |
Mum | 64 | 18.0 GB/s | 67.0 | 9 | Slightly worse [collisions] |
XXH32 | 32 | 9.7 GB/s | 71.9 | 10 | |
City32 | 32 | 9.1 GB/s | 66.0 | 10 | |
Murmur3 | 32 | 3.9 GB/s | 56.1 | 10 | |
SipHash | 64 | 3.0 GB/s | 43.2 | 10 | |
FNV64 | 64 | 1.2 GB/s | 62.7 | 5 | Poor avalanche properties |
Blake2 | 256 | 1.1 GB/s | 5.1 | 10 | Cryptographic |
SHA1 | 160 | 0.8 GB/s | 5.6 | 10 | Cryptographic but broken |
MD5 | 128 | 0.6 GB/s | 7.8 | 10 | Cryptographic but broken |
It seems xxHash is by far the fastest one, while many others beat older hashes, like CRC32, MD5 and SHA.