Avoid exponential grow of const references and rvalue references in constructor

This is exactly the use case for "pass by value and move" technique. Although slighly less efficient than lvalue/rvalue overloads, it not too bad (one extra move) and saves you the hassle.

LinearClassifier(Loss loss, Optimizer optimizer) 
    : _loss(std::move(loss)), _optimizer(std::move(optimizer)) {}

In the case of lvalue argument, there will be one copy and one move, in the case of rvalue argument, there will be two moves (provided that you classes Loss and Optimizer implement move constructors).

Update: In general, perfect forwarding solution is more efficient. On the other hand, this solution avoids templated constructors which are not always desirable, because it will accept arguments of any type when not constrained with SFINAE and lead to hard errors inside the constructor if arguments are not compatible. In other words, unconstrained templated constructors are not SFINAE-friendly. See Barry's answer for a constrained template constructor which avoids this problem.

Another potential problem of a templated constructor is the need to place it in a header file.

Update 2: Herb Sutter talks about this problem in his CppCon 2014 talk "Back to the Basics" starting at 1:03:48. He discusses pass by value first, then overloading on rvalue-ref, then perfect forwarding at 1:15:22 including constraining. And finally he talks about constructors as the only good use case for passing by value at 1:25:50.

Actually, this is the precise reason why perfect forwarding was introduced. Rewrite the constructor as

template <typename L, typename O>
LinearClassifier(L && loss, O && optimizer)
    : _loss(std::forward<L>(loss))
    , _optimizer(std::forward<O>(optimizer))
{}

But it will probably be much simpler to do what Ilya Popov suggests in his answer. To be honest, I usually do it this way, since moves are intended to be cheap and one more move does not change things dramatically.

As Howard Hinnant has told, my method can be SFINAE-unfriendly, since now LinearClassifier accepts any pair of types in constructor. Barry's answer shows how to deal with it.

For the sake of completeness, the optimal 2-argument constructor would take two forwarding references and use SFINAE to ensure that they're the correct types. We can introduce the following alias:

template <class T, class U>
using decays_to = std::is_convertible<std::decay_t<T>*, U*>;

And then:

template <class L, class O,
          class = std::enable_if_t<decays_to<L, Loss>::value &&
                                   decays_to<O, Optimizer>::value>>
LinearClassifier(L&& loss, O&& optimizer)
: _loss(std::forward<L>(loss))
, _optimizer(std::forward<O>(optimizer))
{ }

This ensures that we only accept arguments that are of type Loss and Optimizer (or are derived from them). Unfortunately, it is quite a mouthful to write and is very distracting from the original intent. This is pretty difficult to get right - but if performance matters, then it matters, and this is really the only way to go.

But if it doesn't matter, and if Loss and Optimizer are cheap to move (or, better still, performance for this constructor is completely irrelevant), prefer Ilya Popov's solution:

LinearClassifier(Loss loss, Optimizer optimizer)
: _loss(std::move(loss))
, _optimizer(std::move(optimizer))
{ }