Odd behavior in pattern matching for functional form

Even Equal doesn't work, although it seems to work. What you expect is a comparison, that gives True when two symbols are the same and False otherwise. Look at this:

f == g
(* f == g *)

This is surely not what you expect. While it seems OK in your first example, because f==f indeed simplifies to True, the flaw becomes obvious in your second example:

g != f
(* g != f *)

Mathematica tries to decide whether f and g are unequal. The problem is that unequal is not equivalent to they are not the same. So the rule of thumb is, if you do numeric comparisons, then you often want to use == or != except you are comparing exact numbers. If you want to decide whether expressions are the same or not, then you need to use === or =!=.

Here are some examples as a brain teaser:

1 === 1.0
(* False *)

1 == 1.0
(* True *)

Something "ugly" also:

DeleteCases[{f[x], g[x], h[x]}, x_[_] /; TrueQ[x == f]]

(*{g[x], h[x]}*)

DeleteCases[{f[x], g[x], h[x]}, x_[_] /; !TrueQ[x == f]]

(*{f[x]}*)