Apologies if this is the wrong list. If so, please direct me to the
appropriate one.
Consider the following C code:
include <openssl/hmac.h>
#include <stdio.h>
int main(int argc, char** argv) {
unsigned char foo[10] = "boo";
printf("%s\n", SHA1(foo, 3, 0));
}
in file test-hmac.c.
gcc -static test-hmac.c -lcrypto; ./a.out
This works correctly, spewing garbage to the terminal.
gcc test-hmac.c -lcrypto; ./a.out
This segmentation faults.
Why? What is wrong here? So far, my best guess is that it has to do with
how SHA1 allocates the return value when passed the null pointer (the third
argument, 0). The SHA1 function creates a static pointer large enough to
hold the result which it then returns. Does the fact that this operation
occurs in a shared library change the allocation to the static pointer so
that accessing after it returns is outside the allowed memory for the
calling program? Any help is appreciated.
Thanks.
P.S. libcrypto version 0.9.8k, Debian version squeeze/sid.
