Force OpenSSL's RNGs to return a repeatable byte sequence

You can force the FIPS ANSI X9.31 RNG into a test mode at runtime, but not the SSLeay RNG (the default). If you recompile OpenSSL with -DPREDICT, the default RNG will output a predictable sequence of numbers, but that's not very convenient.

The RAND_pseudo_bytes function generates a predictable series of numbers, meaning it does not add entropy to itself automatically like RAND_bytes. But like you noticed it's only possible to add entropy to the seed, not provide the seed explicitly, so between runs of the program you'll get different numbers. Also not helpful.

But writing your own predictable RNG engine is not difficult. In fact, I'll take you through it by making a rand engine with stdlib's rand() at its core:

#include <cstdio>
#include <cstdlib>
#include <cassert>
#include <openssl/rand.h>

// These don't need to do anything if you don't have anything for them to do.
static void stdlib_rand_cleanup() {}
static void stdlib_rand_add(const void *buf, int num, double add_entropy) {}
static int stdlib_rand_status() { return 1; }

// Seed the RNG.  srand() takes an unsigned int, so we just use the first
// sizeof(unsigned int) bytes in the buffer to seed the RNG.
static void stdlib_rand_seed(const void *buf, int num)
{
        assert(num >= sizeof(unsigned int));
        srand( *((unsigned int *) buf) );
}

// Fill the buffer with random bytes.  For each byte in the buffer, we generate
// a random number and clamp it to the range of a byte, 0-255.
static int stdlib_rand_bytes(unsigned char *buf, int num)
{
        for( int index = 0; index < num; ++index )
        {
                buf[index] = rand() % 256;
        }
        return 1;
}

// Create the table that will link OpenSSL's rand API to our functions.
RAND_METHOD stdlib_rand_meth = {
        stdlib_rand_seed,
        stdlib_rand_bytes,
        stdlib_rand_cleanup,
        stdlib_rand_add,
        stdlib_rand_bytes,
        stdlib_rand_status
};

// This is a public-scope accessor method for our table.
RAND_METHOD *RAND_stdlib() { return &stdlib_rand_meth; }

int main()
{
        // If we're in test mode, tell OpenSSL to use our special RNG.  If we
        // don't call this function, OpenSSL uses the SSLeay RNG.
        int test_mode = 1;
        if( test_mode )
        {
                RAND_set_rand_method(RAND_stdlib());
        }

        unsigned int seed = 0x00beef00;
        unsigned int rnum[5];

        RAND_seed(&seed, sizeof(seed));
        RAND_bytes((unsigned char *)&rnum[0], sizeof(rnum));
        printf("%u %u %u %u %u\n", rnum[0], rnum[1], rnum[2], rnum[3], rnum[4]);

        return 0;
}

Every time you run this program, it seeds srand() with the same number and therefore gives you the same sequence of random numbers every time.

corruptor:scratch indiv$ g++ rand.cpp -o r -lcrypto -g
corruptor:scratch indiv$ ./r
1547399009 981369121 2368920148 925292993 788088604
corruptor:scratch indiv$ ./r
1547399009 981369121 2368920148 925292993 788088604
corruptor:scratch indiv$