decltype with function template which has default argument make the confused result(a funny problem or gcc's bug)

Looking at the "update".

The functions #11 and #22 are overloaded with respect to each other. As a template they both exist, and they differ on the first parameter (int versus float). Thus getvalue(0, state<2>{}) will always match #22, no matter the expression it is in (decltype or otherwise).

For example:

int main() {
  using t = decltype(getvalue(0, state<2>{}));
  std::cout << typeid(t).name() << std::endl;
  auto result = getvalue(0, state<2>{});
  std::cout << typeid(decltype(result)).name() << std::endl;
}

When compiled and invoked:

$ g++ -std=c++17 main.cpp -o main && ./main | c++filt -t
unsigned long
unsigned long

If you would fix #11 to use int instead, it gets worse. The compiler now sees both template functions with the same signature and throws an ambiguous call error:

main.cpp: In function ‘int main()’:
main.cpp:29:44: error: call of overloaded ‘getvalue(int, state<2>)’ is ambiguous
   using t = decltype(getvalue(0, state<2>{}));
                                            ^
main.cpp:21:6: note: candidate: void getvalue(int, state<N>, int) [with int N = 2; U = state<1>]
 void getvalue(int, state<N>, int res = generate_state<N>::value) {
      ^~~~~~~~
main.cpp:25:13: note: candidate: std::size_t getvalue(int, state<N>, int) [with int N = 2; U = state<2>; std::size_t = long unsigned int]
 std::size_t getvalue(int, state<N>, int r = 0) {
             ^~~~~~~~

The thing is - when you invoke a function, it tries to instantiate all possible alternatives, including all default arguments, default template arguments etc. as needed. When, after instantiation, an alternative is valid - it is considered.

There is no possibility in C++ to reject an alternative just because a given template with arguments was not instantiated yet.

What is possible, is to reject an alternative, because such instantiation failed, as already suggested by Stian Svedenborg.

A quick example on what is possible:

#include <iostream>

template<int N>
struct state
{
    static constexpr int value = N;
    friend auto create(state<N>);
};

template<int N>
struct generate_state
{
    friend auto create(state<N>) {
        return state<N>{};
    }
    static constexpr int value = N;
};
template struct generate_state<1>;

template<int N>
struct is_zero{};

template<>
struct is_zero<0> {
    using type = void;
};

//typename `is_zero<N>::type` is valid only for N=0,
//otherwise the expression leads to an error

template<int N>
struct is_nonzero{
    using type = void;

};

template<>
struct is_nonzero<0> {
};

//typename `is_nonzero<N>::type` is valid for N!=0.
//For N=0 the expression leads to an error

template<int N, typename U = typename is_zero<N>::type > // #11
void getvalue(int, state<N>, int res = generate_state<N>::value) {
}

template<int N, typename U = typename is_nonzero<N>::type > // #22
std::size_t getvalue(int, state<N>, int r = 0) {
    return N;
}

int main() {
  //This tries to instantiate both #11 and #22.
  //#11 leads to an error during default argument instantiation and is silently rejected.
  //Thus #22 is used
  using t = decltype(getvalue(0, state<2>{}));
  std::cout << typeid(t).name() << std::endl;

  //This also tries to instantiate both #11 and #22.
  //#22 leads to an error during default argument instantiation and is silently rejected.
  //Thus #11 is used
  using u = decltype(getvalue(0, state<0>{}));
  std::cout << typeid(u).name() << std::endl;
}

When invoked this gives the expected:

$ g++ -std=c++17 main.cpp -o main && ./main | c++filt -t
unsigned long
void

In general SFINAE - the mechanism which allows an error to be silently rejected during instantiation, rather than actually throwing an error and terminating your compilation process - is really tricky. But the explanation would be big and is beyond the scope of this question/answer.


Update:

Understanding the problem:

This is some interesting code! As you state in the comments to my original answer, the crux here is the friend auto declarations inside the state<N> and generate_state<N> classes.

If I understand your idea, the point is to declare the classes in such a way that create(state<x>) is only defined if generate_state<x> has also been declared in this scope.

Digging further into your code, I believe I have understood what is going on.

What is happening

To understand what is happening, let us take a look at your second example.

Let us change main to the following:

int main() {
    using t = decltype(getvalue(0, state<1>{})); // Line 1
    using u = decltype(getvalue(0, state<2>{})); // Line 2
    using v = decltype(getvalue(0, state<3>{})); // Line 3

    std::cout << typeid(t).name() << std::endl;
    std::cout << typeid(u).name() << std::endl;
    std::cout << typeid(v).name() << std::endl;
}

This also compiles and produces

std::size_t (actually it is just 'm' on my machine, but anyhow...)
std::size_t
std::size_t

What is happening here is the following:

On line 1, #11 will fail to resolve, since create(state<0>) does not exist, this is a substitution failure and is therefore not an error. #22 will resolve and is therefore used.

On line 2, #11 will resolve, and in resolving it will resolve generate_state<2>::value. This statement adds create(state<2>) to the compiler's symbol table.

Following this, line 2 will try to resolve #22. And intuitively we would expect this to fail. However, since #11 just resolved, create(state<2>) is now available, and #22 resolves as well. int is a better match than float so #22 is chosen.

The same thing now happens for line 3, since create<(state<2>) is available.

It is made even more clear if you again alter main to the following:

int main() {
    using t = decltype(getvalue(0, state<1>{})); 
    using v = decltype(getvalue(0, state<3>{})); // Line 2 and 3 are swapped.
    using u = decltype(getvalue(0, state<2>{})); 

    std::cout << typeid(t).name() << std::endl;
    std::cout << typeid(u).name() << std::endl;
    std::cout << typeid(v).name() << std::endl;
}

As doing so will cause the compiler to fail.

The compiler fails because on (the new) line 2, create(state<2>) is not yet available, so #11 fails to resolve. As #11 fails to resolve,create(state<3>)` is never added to the symbol table and thus #22 also fails to resolve, resulting in a compilation error.

Likewise, changing the default parameter in #11 to state<N>::value will cause the #11 to be picked over #22 for get_value(0, state<2>). If you do this, all states other than 1 and 2 will fail (as expected).


Original Answer: Kept to explain comments.

To my eye your examples behave as expected. You seem to have misunderstood parts of the fundamentals about template instantiations. I'll go through them in turn:

When you write:

It means the complier chose the #22 ,but at this point of decltype(getvalue(0, state<2>{})),the defination of create(state<2>{}) does not exsite at all

This statement is false. One of the characteristics of a template class/struct is that the type will be declared whenever it is needed.

This means that the statement:

template struct generate_state<1>;

Is not really doing anything in this example. You can safely remove it and the code will still work in exactly the same way. The only reason to use the statement above, is when you want a certain version of a template to be referenced in the given compilation unit (and thus type-substituted and written to code).

The other thing I think you have misunderstood is how the template functions are compiled.

As you already know, when writing a normal template function, there are two stages to its calling. First, during compilation, the template parameters are substituted and the function is written to code. Second, when the function is called, the previously written code is executed with the given arguments, normally this only happens at runtime, but when invoking the function is a constexpr context the function may be executed at compile-time.

This is the core of metaprogramming: To design logic which is executed at compile-time. The output from the metaprogramming execution is the code that will execute.

So the reason your static_asserts fail is because the compiler cannot prove that the assertion is always true, for any and all instantiation of the template, it has nothing to do with how that function is called.

What I believe you are trying to do is to use a feature popularly called "SFINAE" (Substitution Failure Is Not An Error). But that only works for methods inside a template class/struct. (Read more about SFINAE here)