How to apply functions on elements of list
Apply[{1, -#} . #2^2 &, t3, {2}]
{{-2, -3}, {1, 1}, {-2, -3}, {1, 1}, {-2, -3}}
Also
☺ = {1, -#}. #2^2 & @@@ # & /@ # &;
☺ @ t3
{{-2, -3}, {1, 1}, {-2, -3}, {1, 1}, {-2, -3}}
and
Map[{1, - First@#} . Last[#]^2 &, t3, {2}]
{{-2, -3}, {1, 1}, {-2, -3}, {1, 1}, {-2, -3}}
and
ReplaceAll[{n_?NumericQ, {x_, y_}} :> x^2 - n y^2] @ t3
{{-2, -3}, {1, 1}, {-2, -3}, {1, 1}, {-2, -3}}
Maybe this:
Partition[#, 2]& @ (#[[2, 1]]^2 - #[[1]]*#[[2, 2]]^2 & /@ Flatten[t3, 1])
{{-2, -3}, {1, 1}, {-2, -3}, {1, 1}, {-2, -3}}