Dear Lazyweb: what’s a good hashing crate that’s very fast and lightweight and very unlikely to have conflicts, that’s not the sha family?
I’m not sure cryptographic security is important in this case, because the source of the data is already sha512-checked, but I still want something that can catch little things like cache corruption, and that can act as a unique key for usually-small data
I saw xxhash, but I guess I don’t understand enough to know if this is going to make the odds of conflicts practically nonexistent (I don’t want to have to check for conflicts at all) #rust#rustlang
@zkat we are currently playing with xxhash replacing md5 as the block-level checksumming algo for CouchDB. All we need is “did this block change on disk while we didn’t look”. It doesn’t matter to us if two blocks with different content get the same hash legitimately.
So far our experiments are going well, and IIRC I saw a rust crate since there is an erlang wrapper for one (the we didn’t end up using for boring reasons).
@zkat xxhash is all I can think of, yeah, but I don't know enough about conflict probability either. They claim at least that the 128-bit flavor of xxhash doesn't produce a collision in a sample of 100 billion hashes, but I don't know of any independent verification of that testing code: https://github.com/Cyan4973/xxHash/tree/dev/tests/collisions
Add comment