Derivative with prime for vector-valued function

Let me try to clarify the mystery here. The way you defined soln[t], without the underscore, means that the delayed expansion will only work when you use symbol t as the argument:

soln[u]
(* soln[u] - it returns unevaluated *)

When you type D[soln[t],t], you are just lucky that the first argument evaluates to {Sin[t], Cos[t]}, which gets differentiated. If you try any other letter, e.g.

D[soln[u],u]
(* Derivative[1][soln][u] *)

it returns unchanged.

This works in Mathematica 12.0

Remove[soln, t, x, y]

soln[tau_] := DSolveValue[ {
   Derivative[1][x][t] == y[t], 
   Derivative[1][y][t] == -x[t], 
   x[0] == 0, y[0] == 1 }, 
   { x[tau], y[tau] }, t]

soln[t]
(*  {Sin[t], Cos[t]}  *)

soln'[t]
(*  {Cos[t], -Sin[t]}  *)

Don't know why your example doesn't work.

Besides the underscore, the problem is the SetDelayed := .

{soln[t_] = {x[t], y[t]} /. 
First@DSolve[{Derivative[1][x][t] == y[t], 
  Derivative[1][y][t] == -x[t], x[0] == 0, y[0] == 1}, {x[t], 
  y[t]}, t],
soln[t],
soln'[t]}

(*   {{Sin[t], Cos[t]}, {Sin[t], Cos[t]}, {Cos[t], -Sin[t]}}   *)

Let me say, many user here think, SetDelayed is the best way to define nearly everything. The opposite is true. Use it as less as absolutly necessary.