0

I want to compare two sha1 hashes for equality. What could be the most efficient way to do this? Currently, I am trying using memcmp. Thanks.

polapts
  • 5,493
  • 10
  • 37
  • 49
  • 3
    The real question should be: Why bother? Do you really *need* something that is faster than your current solution? I doubt that comparing SHA1s is the bottle neck in your application. – Ferdinand Beyer Mar 14 '12 at 08:46
  • @FerdinandBeyer excellent point – fredoverflow Mar 14 '12 at 08:50
  • I totally agree with Ferdinand and what you are trying to do is called 'premature optimization' which should be avoided... – Malkocoglu Mar 14 '12 at 08:52
  • Well.. @FredOverflow solution (is_same_sha1) seems to be faster than memcmp and it's not negligible. Thanks for it. I am just trying to see if I can make it even faster. What's wrong with it? – polapts Mar 14 '12 at 08:54
  • Did you just measure the hash comparisons, or did you measure the whole program performance? Because actually *computing* the hashes beforehand is likely to be several orders of magnitude slower, so it wouldn't matter at all how efficient the comparison is. – fredoverflow Mar 14 '12 at 08:56
  • 1
    @FredOverflow I measured performance for whole program and it turns out to be faster than memcmp. – polapts Mar 14 '12 at 09:02

3 Answers3

3

Well, since you already know at compile-time how large the blocks are, you could do this:

#include <cstdint>

bool is_same_sha1(const char* p, const char* q)
{
    const std::uint32_t* a = (const std::uint32_t*)p;
    const std::uint32_t* b = (const std::uint32_t*)q;
    return a[0] == b[0] && a[1] == b[1] && a[2] == b[2]
        && a[3] == b[3] && a[4] == b[4];
}

But don't take my advice blindly, you should measure any custom solution against the memcmp solution and only use it if it gives you a significant performance advantage. I wouldn't even be surprised if memcmp was still faster because it did something extremely clever and dirty.

fredoverflow
  • 256,549
  • 94
  • 388
  • 662
  • This seems like a good solution. It's faster than memcmp for my use. Thanks. Though I have to see if std::equal makes any difference. – polapts Mar 14 '12 at 08:55
  • This may fail if the buffers aren't aligned correctly. (It's also formally undefined behavior, but it is clear that the intent is for it to work _if_ external constraints such as alignment are met, and it would be a very poor implementation where it didn't.) – James Kanze Mar 14 '12 at 09:38
  • Also, of course, `uint32_t` might not be available. (In the end, it depends on your portability constraints. `uint32_t` _will_ be available under Windows and under Posix compliant systems. And there are an awful lot of applications where that's portable enough.) – James Kanze Mar 14 '12 at 09:40
  • @JamesKanze Could you please elaborate? Where does the buffer alignment comes into picture here? – polapts Mar 14 '12 at 09:42
  • @polapts: in C and C++, types may have alignment requirements, meaning that the address of a pointer to that type *must* be divisible by the required alignment. It's common for the alignment requirement of `uint32_t` to be `4`. So, if `p` or `q` happen not to be 4-aligned, then it's undefined behavior to cast them to `uint32_t*`. For example if I write `const char *foo == "11..1"`, with 21 digits in there, then `is_same_sha1(foo, foo+1)` is UB on an implementation that requires 4-alignment for `uint32_t`, since `foo` and `foo+1` can't both be 4-aligned. – Steve Jessop Mar 14 '12 at 10:23
2

What's wrong with memcmp()? You have to compare every byte of both hashes; memcmp() will quickly fail on the first difference it finds; and memcmp() can be written by the library authors to work in chunk sizes that are good choices for the platform.

sarnold
  • 102,305
  • 22
  • 181
  • 238
  • I would expect `std::equal` to be faster than `memcpy`; the compiler generates it for the precise type involved, and can possibly even take into account things like alignment. (Of course, `memcpy` is likely just a `#define` for something like `__builtin_memcpy`, so that the compiler can do similar optimizations.) – James Kanze Mar 14 '12 at 08:43
  • True, but your advice to align on boundaries is definitely worth while; if they aligned on 32 byte blocks, it might be a single SSE-something instruction to check them... – sarnold Mar 14 '12 at 08:48
  • @sarnold Is there an SSE instruction that compares exactly 160 bits? :) You would probably have to overread from memory (dangerous?) and then mask off the extra bits. – fredoverflow Mar 14 '12 at 08:58
  • @Fred: A single op might be optimistic, but you presumably could do it as a 128 and a 32 *if* you had the right alignment. Unfortunately, checking the alignment might well take longer than unconditionally doing it in smaller chunks. – Steve Jessop Mar 14 '12 at 10:30
  • @Fred: I was thinking of zeroing the next twelve bytes so they do not influence the comparison. I did get a little overzealous and assume that newer SSE* instructions might introduce 256 bit comparison routines, but you're right: it looks like most of the CPU's comparison instructions are intended for 128 bit blocks at most. Though [SSE 4.2 introduced some variable-length string comparison routines](http://en.wikipedia.org/wiki/SSE4.2#SSE4.2), it might be a little while before libraries include them. – sarnold Mar 14 '12 at 10:45
2

std::equal would seem the best bet, but memcmp would also work. With regards to efficiency, it will depend on the implementation, but also (possibly) how the data is defined and represented.

James Kanze
  • 150,581
  • 18
  • 184
  • 329