How to unit test deliberate compilation errors of template code

Testing for a negative feature, hence provide a guarantee that certain construct will fail to compile is possible using c++20 requires expressions as follows:

Simple example

Below, I check if overloads to the function func exist in static assertions, when used with a test framework, the boolean should be used on one of the run time tests in order to not block the other tests from compiling:

#include <concepts>
/// Arbitrary restrictions in order to test:
/// if T != double -> zero args
template <typename T> void func(){};
/// if T == double -> arbitrary args.
template<std::same_as<double> ...T> void func(T... as){};
template <typename T, typename... a> constexpr bool applies_to_func = requires(a... as) {
  func<T>(as...);
};
/// compiles:
static_assert(applies_to_func<int>);
static_assert(applies_to_func<double, double>);
static_assert(applies_to_func<double, double, double>);
/// function fails to compile:
static_assert(!applies_to_func<int, int>);

The code is available on Compiler Explorer: https://godbolt.org/z/direWo

C++17

I recently tried tried to do a similar thing for a project in which I can only use c++17. In my code I also check if the function's return type matches the caller's expectations. Aside from some limitations regarding the non-type template parameters, a similiar thing can be achieved as demonstrated below. In this case I could not enfroce double as input to the overload, due to the implicit conversion, applies_to_func(void, int, int) will evaluate to true in the code snipplet below.

#include <utility>
#include <string>

/// Create compile-time tests that allow checking a specific function's type  
#define COMPILE_TIME_TEST(func) COMPILE_TIME_TEST_FUNCTION(func, func)
#define COMPILE_TIME_TEST_FUNCTION(name, func)                                                              \
namespace detail {                                                                                          \
  template<typename R, auto... args> struct name ## FromArgs:std::false_type{};                             \
  template<auto... args> struct name ## FromArgs<decltype(func(args...)), args...> : std::true_type{};      \
  template<typename R, typename... Args> struct name ## FromType:std::false_type{};                         \
  template<typename... Args> struct name ## FromType<decltype(func(std::declval<Args>()...)), Args...> : std::true_type{};\
}                                                                                                           \
template<typename R, auto ...Args>                                                                          \
static constexpr auto name ## _compiles = detail::name ## FromArgs<R, Args...>::value;                      \
template<typename ...Args> \
static constexpr auto name ## _compiles_from_type = detail::name ## FromType<Args...>::value;\

int func();
template <typename T> void func(T);
void func(double);
void func(double, double );
void func(double, double, double);

// create the structs from above macros for the function `func`
COMPILE_TIME_TEST(func);

static_assert(!func_compiles<void>);
static_assert(func_compiles<int>);
static_assert(func_compiles_from_type<void, double, double>);
static_assert(!func_compiles_from_type<void, double, double, double, double>);

static_assert(func_compiles<void, 1>);
static_assert(!func_compiles<void, 1, nullptr>);

Do it in the similar way compiler tests are written. You will have a bit of testing code in some scripting language (shell, perl, tcl etc.) that will run compiler on given snippets of code and check whether the right ones compiled and the right ones did not.

• gcc uses DejaGnu, which is built on top of expect, which is itself built on top of Tcl.
• If you use shell script (probably easier, DejaGnu is probably overkill), you might want to look at shUnit2.
• Perl's Test::Harness system should be mostly easy to use as is.
• After all, it's not that much more work to run process from C++, so writing a function to try to call compiler on a given string and check whether it outputs error for line where you expect it would not be that hard and than you can integrate it into the other boost.test-based tests.