How to chain BCryptEncrypt and BCryptDecrypt calls using AES in GCM mode?

@Codeguard's answer got me through the project I was working on which lead me to find this question/answer in the first place; however, there were still a number of gotchas I struggled with. Below is the process I followed with the tricky parts called out. You can view the actual code at the link above:

1. Use BCryptOpenAlgorithmProvider to open the algorithm provider using BCRYPT_AES_ALGORITHM.
2. Use BCryptSetProperty to set the BCRYPT_CHAINING_MODE to BCRYPT_CHAIN_MODE_GCM.
3. Use BCryptGetProperty to get the BCRYPT_OBJECT_LENGTH to allocate for use by the BCrypt library for the encrypt/decrypt operation. Depending on your implementation, you may also want to:
   • Use BCryptGetProperty to determine BCRYPT_BLOCK_SIZE and allocate scratch space for the IV. The Windows API updates the IV with each call, and the caller is responsible for providing the memory for that usage.
   • Use BCryptGetProperty to determine BCRYPT_AUTH_TAG_LENGTH and allocate scratch space for the largest possible tag. Like the IV, the caller is responsible for providing this space, which the API updates each time.
4. Initialize the BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO struct:
   • Initialize the structure with BCRYPT_INIT_AUTH_MODE_INFO()
   • Initialize the pbNonce and cbNonce field. Note that for the first call to BCryptEncrypt/BCryptDecrypt, the IV is ignored as an input and this field is used as the "IV". However, the IV parameter will be updated by that first call and used by subsequent calls, so space for it must still be provided. In addition, the pbNonce and cbNonce fields must remain set (even though they are unused after the first call) for all calls to BCryptEncrypt/BCryptDecrypt or those calls will complain.
   • Initialize pbAuthData and cbAuthData. In my project, I set these fields just before the first call to BCryptEncrypt/BCryptDecrypt and immediately reset them to NULL/0 immediately afterward. You can pass NULL/0 as the input and output parameters during these calls.
   • Initialize pbTag and cbTag. pbTag can be NULL until the final call to BCryptEncrypt/BCryptDecrypt when the tag is retrieved or checked, but cbTag must be set or else BCryptEncrypt/BCryptDecrypt will complain.
   • Initialize pbMacContext and cbMacContext. These point to a scratch space for the BCryptEncrypt/BCryptDecrypt to use to keep track of the current state of the tag/mac.
   • Initialize cbAAD and cbData to 0. The APIs use these fields, so you can read them at any time, but you should not update them after initially setting them to 0.
   • Initialize dwFlags to BCRYPT_AUTH_MODE_CHAIN_CALLS_FLAG. After initialization, changes to this field should be made by using |= or &=. Windows also sets flags within this field that the caller needs to take care not to alter.
5. Use BCryptGenerateSymmetricKey to import the key to use for encryption/decryption. Note that you will need to supply the memory associated with BCRYPT_OBJECT_LENGTH to this call for use by BCryptEncrypt/BCryptDecrypt during operation.
6. Call BCryptEncrypt/BCryptDecrypt with your AAD, if any; no input nor space for output need be supplied for this call. (If the call succeeds, you can see the size of your AAD reflected in the cbAAD field of the BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO structure.)
   1. Set pbAuthData and cbAuthData to reflect the AAD.
   2. Call BCryptEncrypt or BCryptDecrypt.
   3. Set pbAuthData and cbAuthData back to NULL and 0.
7. Call BCryptEncrypt/BCryptDecrypt "N - 1" times
   • The amount of data passed to each call must be a multiple of the algorithm's block size.
   • Do not set the dwFlags parameter of the call to anything other than 0.
   • The output space must be equal to or greater than the size of the input
8. Call BCryptEncrypt/BCryptDecrypt one final time (with or without plain/cipher text input/output). The size of the input need not be a multiple of the algorithm's block size for this call. dwFlags is still set to 0.
   1. Set the pbTag field of the BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO structure either to the location at which to store the generated tag or to the location of the tag to verify against, depending on whether the operation is an encryption or decryption.
   2. Remove the BCRYPT_AUTH_MODE_CHAIN_CALLS_FLAG from the dwFlags field of the BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO structure using the &= syntax.
9. Call BCryptDestroyKey
10. Call BCryptCloseAlgorithmProvider

It would be wise, at this point, to wipe out the space associated with BCRYPT_OBJECT_LENGTH.

I managed to get it to work. It seems that the problem is in MSDN, it should mention setting BCRYPT_AUTH_MODE_CHAIN_CALLS_FLAG instead of BCRYPT_AUTH_MODE_IN_PROGRESS_FLAG.

#include <windows.h>
#include <assert.h>
#include <vector>
#include <Bcrypt.h>
#pragma comment(lib, "bcrypt.lib")

std::vector<BYTE> MakePatternBytes(size_t a_Length)
{
    std::vector<BYTE> result(a_Length);
    for (size_t i = 0; i < result.size(); i++)
    {
        result[i] = (BYTE)i;
    }

    return result;
}

std::vector<BYTE> MakeRandomBytes(size_t a_Length)
{
    std::vector<BYTE> result(a_Length);
    for (size_t i = 0; i < result.size(); i++)
    {
        result[i] = (BYTE)rand();
    }

    return result;
}

int _tmain(int argc, _TCHAR* argv[])
{
    NTSTATUS bcryptResult = 0;
    DWORD bytesDone = 0;

    BCRYPT_ALG_HANDLE algHandle = 0;
    bcryptResult = BCryptOpenAlgorithmProvider(&algHandle, BCRYPT_AES_ALGORITHM, 0, 0);
    assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptOpenAlgorithmProvider");

    bcryptResult = BCryptSetProperty(algHandle, BCRYPT_CHAINING_MODE, (BYTE*)BCRYPT_CHAIN_MODE_GCM, sizeof(BCRYPT_CHAIN_MODE_GCM), 0);
    assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptSetProperty(BCRYPT_CHAINING_MODE)");

    BCRYPT_AUTH_TAG_LENGTHS_STRUCT authTagLengths;
    bcryptResult = BCryptGetProperty(algHandle, BCRYPT_AUTH_TAG_LENGTH, (BYTE*)&authTagLengths, sizeof(authTagLengths), &bytesDone, 0);
    assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptGetProperty(BCRYPT_AUTH_TAG_LENGTH)");

    DWORD blockLength = 0;
    bcryptResult = BCryptGetProperty(algHandle, BCRYPT_BLOCK_LENGTH, (BYTE*)&blockLength, sizeof(blockLength), &bytesDone, 0);
    assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptGetProperty(BCRYPT_BLOCK_LENGTH)");

    BCRYPT_KEY_HANDLE keyHandle = 0;
    {
        const std::vector<BYTE> key = MakeRandomBytes(blockLength);
        bcryptResult = BCryptGenerateSymmetricKey(algHandle, &keyHandle, 0, 0, (PUCHAR)&key[0], key.size(), 0);
        assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptGenerateSymmetricKey");
    }

    const size_t GCM_NONCE_SIZE = 12;
    const std::vector<BYTE> origNonce = MakeRandomBytes(GCM_NONCE_SIZE);
    const std::vector<BYTE> origData  = MakePatternBytes(256);

    // Encrypt data as a whole
    std::vector<BYTE> encrypted = origData;
    std::vector<BYTE> authTag(authTagLengths.dwMinLength);
    {
        BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO authInfo;
        BCRYPT_INIT_AUTH_MODE_INFO(authInfo);
        authInfo.pbNonce = (PUCHAR)&origNonce[0];
        authInfo.cbNonce = origNonce.size();
        authInfo.pbTag   = &authTag[0];
        authInfo.cbTag   = authTag.size();

        bcryptResult = BCryptEncrypt
            (
            keyHandle,
            &encrypted[0], encrypted.size(),
            &authInfo,
            0, 0,
            &encrypted[0], encrypted.size(),
            &bytesDone, 0
            );

        assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptEncrypt");
        assert(bytesDone == encrypted.size());
    }

    // Decrypt data in two parts
    std::vector<BYTE> decrypted = encrypted;
    {
        DWORD partSize = decrypted.size() / 2;

        std::vector<BYTE> macContext(authTagLengths.dwMaxLength);

        BCRYPT_AUTHENTICATED_CIPHER_MODE_INFO authInfo;
        BCRYPT_INIT_AUTH_MODE_INFO(authInfo);
        authInfo.pbNonce = (PUCHAR)&origNonce[0];
        authInfo.cbNonce = origNonce.size();
        authInfo.pbTag   = &authTag[0];
        authInfo.cbTag   = authTag.size();
        authInfo.pbMacContext = &macContext[0];
        authInfo.cbMacContext = macContext.size();

        // IV value is ignored on first call to BCryptDecrypt.
        // This buffer will be used to keep internal IV used for chaining.
        std::vector<BYTE> contextIV(blockLength);

        // First part
        authInfo.dwFlags = BCRYPT_AUTH_MODE_CHAIN_CALLS_FLAG;
        bcryptResult = BCryptDecrypt
            (
            keyHandle,
            &decrypted[0*partSize], partSize,
            &authInfo,
            &contextIV[0], contextIV.size(),
            &decrypted[0*partSize], partSize,
            &bytesDone, 0
            );

        assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptDecrypt");
        assert(bytesDone == partSize);

        // Second part
        authInfo.dwFlags &= ~BCRYPT_AUTH_MODE_CHAIN_CALLS_FLAG;
        bcryptResult = BCryptDecrypt
            (
            keyHandle,
            &decrypted[1*partSize], partSize,
            &authInfo,
            &contextIV[0], contextIV.size(),
            &decrypted[1*partSize], partSize,
            &bytesDone, 0
            );

        assert(BCRYPT_SUCCESS(bcryptResult) || !"BCryptDecrypt");
        assert(bytesDone == partSize);
    }

    // Check decryption
    assert(decrypted == origData);

    // Cleanup
    BCryptDestroyKey(keyHandle);
    BCryptCloseAlgorithmProvider(algHandle, 0);

    return 0;
}